# Add Slack Alerts on Failed Runs to a Dagster Project

Yes. An agent adds a run failure sensor that posts the job, run and error to Slack, routes by the alert_team tag, tests it with mocked requests and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-slack-failure-alerts/dagster/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent clones the repository into its own cloud sandbox and reads how the Dagster code is organised before adding anything. In a workspace like dagster-io/hooli-data-eng-pipelines, `dg.toml` lists six projects, `dagster_cloud.yaml` maps each to a Dagster+ code location built from its `src/` directory, and `.github/workflows/deploy-dagster-cloud.yml` deploys a branch deployment per pull request. The main project, `hooli-data-eng`, loads its definitions with `load_defs(hooli_data_eng.defs)` in `definitions.py`, so every subpackage under `defs/` with its own `definitions.py` is picked up: `pandas`, `dbt`, `files`, `databricks`, `kubernetes` and the rest. `defs/pandas/schedules.py` already defines `predict_job` with `tags={"alert_team": "ml"}`, an `asset_sensor` and a freshness-check schedule, and `defs/files/sensors.py` shows the house style for a sensor that reads the environment through `utils.get_env()`, which returns `PROD`, `BRANCH` or `LOCAL` from `DAGSTER_CLOUD_DEPLOYMENT_NAME` and `DAGSTER_IS_DEV_CLI`. `pyproject.toml` depends on `requests` and `responses` but not on `dagster-slack`, and nothing in the repository mentions Slack.

It adds a new `defs/alerts/` package in the same shape as its neighbours: `sensors.py` and `definitions.py`. The sensor is a `@dg.run_failure_sensor` with `monitor_all_code_locations=True`, so one definition in this location covers failures in `basics`, `batch_enrichment`, `hooli_data_ingest` and the other locations too, and `default_status=dg.DefaultSensorStatus.RUNNING` so it is on after deploy without a click in the UI. Its body reads the run's job name, run id and the failure event's message from the sensor context, reads the `alert_team` tag the repository already puts on `predict_job`, and posts a short message through `requests` to a Slack incoming webhook. The webhook URL comes from an environment variable, `SLACK_ALERTS_WEBHOOK_URL`, read the way `resources.py` reads `SNOWFLAKE_ACCOUNT`, and a per-team mapping lets the `ml` tag reach its own channel while untagged jobs reach the default. When `get_env()` returns `LOCAL`, the sensor logs the message instead of posting it, so `dagster dev` on a laptop never pages a channel.

The message carries what an on-call engineer needs to act: job name, run id, the deployment name from `DAGSTER_CLOUD_DEPLOYMENT_NAME`, the first line of the error, and a link to the run built from the deployment URL, with the team tag as a prefix so Slack search finds a team's failures. The agent does not add `dagster-slack`: the repository already has `requests`, an incoming webhook needs no bot token or scopes, and one fewer package in a `pyproject.toml` that already lists more than twenty keeps the uv lock change to nothing. It documents the alternative, Dagster+ alert policies configured outside the code, in the pull request and says why a sensor in the repository was chosen: it is versioned, tested and visible in the same place as the jobs it watches.

Then it proves the sensor works without a Slack workspace. A new `hooli_data_eng_tests/test_alerts.py` builds a context with `dg.build_run_status_sensor_context` and an ephemeral `DagsterInstance`, uses the `responses` library the project already depends on to intercept the webhook call, and asserts on the posted JSON: job name, run id and the routed channel for a run tagged `alert_team: ml` and for one with no tag. The existing `test_repo_loads.py` pattern gets one more assertion that the definitions load with the sensor present. In the sandbox the agent runs `make dependencies`, `pytest hooli-data-eng/hooli_data_eng_tests`, `ruff check` as the deploy workflow does, and `dg check defs` from the dev extras to confirm the location still loads, and pastes the output into the pull request.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add Slack alerts on failed runs to this Dagster project. Read dg.toml, dagster_cloud.yaml, pyproject.toml, definitions.py, every defs subpackage's definitions.py, the existing sensors and schedules, utils, resources.py and the tests first, and note any alert-related tag the jobs already carry. Add a defs/alerts package with a run_failure_sensor that monitors all code locations, is running by default, reads the job name, run id, deployment name and failure message from the context, routes by the alert_team tag to a per-team channel with a default, posts through requests to a Slack incoming webhook read from the SLACK_ALERTS_WEBHOOK_URL environment variable, and only logs when the environment is LOCAL. Do not add dagster-slack. Add tests that build a run status sensor context, mock the webhook with the responses library and assert on the payload for a tagged and an untagged run, and extend the repo-loads test. Add a README note on setting the variable in Dagster+. Run the dependency install, pytest, ruff check and dg check defs in your sandbox, and open a pull request with the output and a note on Dagster+ alert policies as the alternative.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request adding `hooli-data-eng/src/hooli_data_eng/defs/alerts/sensors.py` and `definitions.py`, a `test_alerts.py` with the mocked webhook assertions, one more line in `test_repo_loads.py`, and a README paragraph on where `SLACK_ALERTS_WEBHOOK_URL` is set for each Dagster+ location. No dependency changes, so `uv.lock` is untouched. The description shows the exact JSON the sensor posts for a tagged and an untagged failure.

The run's record keeps the `pytest`, `ruff check` and `dg check defs` output from the sandbox, so a reviewer sees the definitions load with the new sensor and the payload assertions pass before the branch deployment the deploy workflow will create for the pull request.

Before merging, a reviewer creates the incoming webhook in Slack, sets the variable on the production location in Dagster+, picks the channel names for the team mapping, and decides whether a failure in a branch deployment should post to the same channel or be silenced by treating `BRANCH` like `LOCAL`.

## Run it on every Linear issue labelled data-platform

A data team that tracks platform work in Linear starts one run per issue from an automation with a `linear` trigger on the `data-platform` label, in the Project that holds the Dagster repository. The trigger adds the issue after this prompt, so an issue that asks for a second channel, alerts on `run_status_sensor` success for a critical job, or a message that includes the partition key is what the agent reads.

Each run checks out the workspace in its own sandbox, runs the tests and `dg check defs`, and opens its own pull request linking the issue, which the deploy workflow turns into a branch deployment a reviewer can watch. An automation run has its own `max_runs_per_hour` cap.

## Why run it on Rush?

A failure sensor is twenty lines of Python and an afternoon of questions: which locations it covers, how the environment is detected so laptops stay quiet, where the tag the jobs already carry can route a message, and how to test a Slack call without Slack. The agent answers each from the repository's own conventions, `get_env()`, the `alert_team` tag, the `responses` dependency, and leaves the proof in the pull request as test output rather than a screenshot of a channel.

The same request runs across every Dagster workspace an organisation deploys, each in its own sandbox with its own `definitions.py` read first, so a dozen teams get the same alert shape with their own routing and their own tests, and the record of every command is kept alongside the pull request.

## Frequently asked questions

### Why not use Dagster+ alert policies?

They work, and the pull request says so. A sensor in the repository is versioned, reviewed and tested with the jobs it watches, runs the same way under `dagster dev`, and can read run tags for routing. Alert policies live outside the code and are configured per deployment; a team can use both.

### Does it need dagster-slack or a bot token?

No. An incoming webhook accepts a JSON post with no token, and the project already depends on `requests`. `dagster-slack` is the right choice when the message needs a bot identity, threads or channel lookup; for a failure notice to a fixed channel it adds a dependency for nothing.

### Will it alert on every code location?

Yes. `monitor_all_code_locations=True` makes one sensor in the `data-eng-pipeline` location watch failures in every location in the deployment, so the other five projects need no code change. A team that wants per-location routing can add a location name to the message, which the context provides.

### What stops it posting from my laptop?

The same `get_env()` helper the rest of the project uses. When `DAGSTER_IS_DEV_CLI` is set the sensor logs the message and returns, so `dagster dev` never reaches Slack. Branch deployments count as `BRANCH` and post by default; a reviewer can change that to log-only in one line.

### How is it tested without a Slack workspace?

With `dg.build_run_status_sensor_context` and an ephemeral instance to construct a failure, and the `responses` library to intercept the webhook post and capture its body. The tests assert on the job name, run id and channel in the JSON, so a change to the message format shows up as a failing test.

## More Dagster jobs

- [Review Pull Requests on a Dagster Project](https://getrush.ai/run/review-pull-requests/dagster/)
- [Upgrade Python Dependencies Weekly for a Dagster Project](https://getrush.ai/run/upgrade-python-dependencies/dagster/)
- [Write Dagster Asset Tests for a Project](https://getrush.ai/run/write-asset-tests/dagster/)

## The same job on other stacks

- [Post Slack Alerts When GitHub Actions Workflows Fail](https://getrush.ai/run/add-slack-failure-alerts/github-actions/)
- [Add Slack Alerts on Task Failure to Airflow DAGs](https://getrush.ai/run/add-slack-failure-alerts/airflow/)

## More jobs like this: Turn a Linear label into a PR

- [Add Slash Commands to a discord.py Bot](https://getrush.ai/run/add-slash-command/discord-py/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
