# Fix an ASP.NET Core App Bug From a Datadog Monitor Alert

Yes. When a Datadog monitor fires, an agent reads its logs and traces, finds the failing ASP.NET Core code, fixes it with an xUnit test and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/fix-datadog-alerts/aspnet-core/), in a Project that has your repository.

## What the agent does

The run starts from the alert itself. Your server receives Datadog's webhook for the monitor and sends the payload as part of the prompt, so the agent knows the monitor id, the query that tripped and the time it fired. Through the Datadog MCP server it reads the monitor definition, then the error logs and APM traces from the 30 minutes around that time, filtered by the service and environment tags the monitor names. It groups the errors by exception type and stack frame, so a burst of identical `NullReferenceException`s counts as one problem with a sample, not a page of noise.

Next it maps what Datadog shows onto the code. In a solution shaped like bitwarden/server, a trace's resource name such as `POST /accounts/prelogin` points at `src/Identity/Controllers/AccountsController.cs`, and a `PUT /ciphers/{id}` span points at `src/Api/Vault/Controllers/CiphersController.cs`, a file of nearly 2,000 lines. The stack frames in the log lines narrow it to the method, and from there the agent follows the call into the command or service it delegates to, because the controller is often only where the exception surfaced. It reads the request models in `src/Api/Vault/Models` and the validators under `src/Identity/IdentityServer/RequestValidators` before deciding where the fix belongs.

Before changing anything it reproduces the failure as a test. The repository pins the .NET SDK in `global.json` (`10.0.103`, `rollForward: latestFeature`), so the sandbox uses that SDK, not whatever is newest. Unit tests in `test/Api.Test` and `test/Identity.Test` use xUnit with AutoFixture and NSubstitute: a class marked `[ControllerCustomize(typeof(CiphersController))]` and `[SutProviderCustomize]`, and `[Theory, BitAutoData]` methods that take a `SutProvider<T>`. The agent writes the regression test in that style, with the inputs the failing log lines show, runs it to watch it fail for the same reason the alert did, then makes the fix and runs it again.

When the path spans the auth pipeline, a unit test cannot show the bug, so the agent uses the integration projects instead (`test/Identity.IntegrationTest`, `test/Api.IntegrationTest`). Their guide, `test/INTEGRATION_TEST.md`, describes a `WebApplicationFactory` per host with an in-memory SQLite database, which runs in the sandbox without a SQL Server. It finishes with `dotnet test` on the affected test projects, then the wider suite, and opens the pull request.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "A Datadog monitor on this ASP.NET Core service fired; its webhook payload is at the end of this prompt. Use the Datadog connection to read the monitor, then the error logs and APM traces for its service and environment from 15 minutes before to 15 minutes after it fired. Group the errors by exception and stack frame, map the trace resource names to controllers in src/Identity and src/Api/Vault, and find the code path that produced them. Write an xUnit regression test in the matching test project that reproduces the failure with the inputs the logs show, following the AutoFixture and SutProvider conventions already used there, and confirm it fails. Fix the cause, run dotnet test on the affected projects, and open a pull request that quotes the log lines and trace ids the fix addresses. If the logs do not point to a code fault, do not change code; answer with what you found instead.\n\nPayload: <Datadog webhook body>",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

When the cause is in the code, a pull request comes back with two parts: the fix, in the service, command or controller method the trace led to, and a regression test beside the existing ones, such as a new `[Theory, BitAutoData]` in `test/Api.Test/Vault/Controllers/CiphersControllerTests.cs`. The description quotes the Datadog log lines and trace ids it reproduced, names the monitor, and lists the `dotnet test` commands it ran with their result. Every one of those commands and its output is also in the session's trajectory.

When the logs point somewhere else, such as a database timeout, an expired certificate or a downstream service returning 503, the agent does not invent a code change. The run's answer says what the logs and traces show and which component to look at, and that answer is what Rush posts to your webhook, so the on-call channel still gets an explanation.

A reviewer checks three things before merging: that the test fails on the old code for the reason the alert did, that the fix handles the input class and not only the one value in the log, and that nothing in the change touches authentication or encryption behavior beyond what the failing path needs.

## Run it on every Datadog monitor alert

In Datadog, add a webhook integration that posts to an endpoint on your server, and add `@webhook-<name>` to the monitors you want handled. When one alerts, your server appends the payload to this prompt and sends the request with the service's repository in `resources`. Filter on the transition you care about (Triggered, not Recovered) and on the tags that mark an application error, so a disk-space alert does not start a code fix.

Datadog retries a webhook it did not see answered, and one incident can trigger the same monitor more than once, so record the alert's id with the `execution_id` Rush returns and skip a repeat. A key runs at most `max_concurrent` sessions at once (4 by default) and answers `429` past that; queue the rest rather than drop them.

## Why run it on Rush?

An alert at 3 a.m. usually means someone opens Datadog, copies a stack trace, checks out the service and starts a debug build. On Rush the agent does that in its own sandbox with its own clone, the SDK `global.json` names and its own Datadog session, so nobody's laptop or local database is involved, and the investigation starts the moment the webhook arrives rather than when someone wakes.

The result is evidence, not a summary. The pull request quotes the exact log lines and trace ids, the trajectory keeps every Datadog query and every `dotnet test` run with its output, and the regression test proves the fix against the failing input. If several monitors fire in one incident, each gets its own run in its own sandbox, and the reviewer decides what merges.

## Frequently asked questions

### How does the agent reach Datadog?

Through Datadog's MCP server, which Rush lists among its connectors. Connect it once with `POST /me/connections` (`"type": "mcp"`, its URL and `"auth": {"mode": "oauth"}`) and sign in, then add `"connections": ["datadog"]` to the request so this run can use it. The listed server is Datadog's US1 site; for another site, add that site's MCP URL as your own server.

### Can it change behavior in the identity or vault code without anyone noticing?

No. Everything it changes is in the pull request diff, and nothing reaches your default branch until a person merges it. The prompt also tells it to answer without changing code when the logs do not point to a code fault.

### Does it need SQL Server to run the tests?

Not for unit tests, which use substitutes for repositories, and not for integration tests that use the in-memory SQLite factory the repository documents. When a fix needs a provider-specific database test, the pull request says which one it could not run.

### What if Datadog has no traces, only logs?

It works from the log lines alone: the exception, its stack frames and the request path in the log attributes. Traces make the mapping to a controller faster and more certain, and the pull request says which it had.