# Review Pull Requests on a Dagster Project

Yes. For each pull request the agent loads the changed code locations, runs Ruff, a dbt parse and the tests, and posts one review with the output quoted.

[Run this on Rush](https://cloud.getrush.ai/run/review-pull-requests/dagster/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

A Dagster pull request can pass every check a repository has and still break at the next deploy, because most of what holds a pipeline together is strings: asset keys, partition start dates, resource names, cron expressions. So the agent starts by learning how this workspace is wired. In a workspace like dagster-io/hooli-data-eng-pipelines it reads `dg.toml`, which lists six projects and comments out a seventh, `hooli-airlift`, with a note that it is currently failing; `dagster_cloud.yaml`, which names seven code locations and the directory each one builds from; each changed project's `pyproject.toml`; and `.github/workflows/deploy-dagster-cloud.yml`. That workflow runs Ruff with `check --fix`, validates the cloud config, builds an image for each location whose files changed and deploys it. It never runs pytest and never loads the definitions before the image is built.

Then it checks out the pull request and does what the workflow leaves out. In each changed project it runs `uv sync --extra dev` against that project's `uv.lock`, which brings in `dagster-dg-cli`, pytest and Ruff. For `hooli-data-eng` it runs `dbt deps` and `dbt parse --project-dir=dbt_project --target LOCAL` first, because the dbt component needs a manifest and the Makefile's own `manifest` target parses with `--target BRANCH`, which `dbt_project/profiles.yml` defines as Snowflake; `LOCAL` is DuckDB and needs no credentials. With `DAGSTER_IS_DEV_CLI` set, so that `get_env()` in `utils/__init__.py` resolves to `LOCAL`, it runs `dg check defs` and quotes any error. It runs `ruff check` without `--fix` and `ruff format --check`: the workflow's `--fix` repairs what it can inside the runner and commits nothing, so a fixable violation passes CI and stays in the branch.

It runs pytest on the base branch before the pull request branch. In this repository that matters: `hooli_data_eng_tests/test_repo_loads.py` imports a `hooli_data_eng` object from a package whose `__init__.py` is empty, and `test_assets.py` imports `hooli_data_eng.assets.raw_data`, while the assets now live under `defs/custom_ingest/assets.py`. A suite that cannot be collected on the base branch is reported once, as a standing problem, and not charged to the author.

Last it reads the diff against rules the code implies. Every `resources.py` under `defs/` builds a `resource_def` dictionary keyed `LOCAL`, `BRANCH` and `PROD`, so a resource added under one key loads on a laptop and fails in the branch deployment. `DailyPartitionsDefinition(start_date="2023-05-25")` is declared in both `custom_ingest/assets.py` and `dbt/component.py`, so a start date changed in one file is checked against the other. A renamed asset or `key_prefix` is searched for in every place that refers to it by string: the `AssetIn(key_prefix=["ANALYTICS"])` inputs in `pandas/assets.py`, the `orders_sensor` asset sensor on `["ANALYTICS", "orders_augmented"]`, the selection in `predict_job`, and the `asset_selection` lines in `defs/dbt/component.yaml`. A new code location has to appear in `dg.toml`, in `dagster_cloud.yaml`, and three times in the workflow: the `files_yaml` filter, the block that builds the `LOCATIONS` variable, and an image build step.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Review this pull request to our Dagster workspace. Read dg.toml, dagster_cloud.yaml, the Makefile, .github/workflows/deploy-dagster-cloud.yml, dbt_project/profiles.yml and the pyproject.toml, definitions.py and utils of every project the diff touches first. In each changed project run uv sync --extra dev from its own uv.lock. For a project that loads dbt, run dbt deps and dbt parse against the LOCAL target so no warehouse credentials are needed. Set DAGSTER_IS_DEV_CLI so the code resolves its LOCAL resources, run dg check defs, ruff check without --fix, ruff format --check, and pytest, and quote every error with file and line. Run pytest on the base branch too and report failures that exist there separately from failures the pull request introduces. Then read the diff: every resource must exist under the LOCAL, BRANCH and PROD keys of its resource_def; a changed partitions definition must match every other definition of the same partitions; a renamed asset key, key_prefix, job, schedule or sensor must be updated everywhere it is referred to by string, including asset sensors, AssetIn, asset selections and component YAML; a changed cron or automation condition is stated in plain words; a new code location must be added to dg.toml, dagster_cloud.yaml and the workflow's file filter, location list and image build. Do not materialize assets against a warehouse. Post one review on the pull request with file and line for every finding and the command output quoted. Do not push commits.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app/pull/42"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

One review on the pull request. It opens with a short results block: dependency sync, dbt parse, `dg check defs`, Ruff lint, Ruff format and pytest, each marked pass or fail with the lines that failed, and a separate line for anything that already fails on the base branch. Findings follow by file, each with the line, what breaks and where the other end of the broken reference is, for example the sensor that still watches an asset key the diff renamed. A change to a cron string or an automation condition is restated as a sentence ("midnight on the odd days of the month" for `0 0 1-31/2 * *`) so a reviewer can confirm the schedule is the one intended.

The review comments; it does not approve or block. The full evidence stays on the session: `GET /o/{org}/sessions/{id}/messages` returns every command with its output, including the full `dg check defs` traceback when a location does not load.

A person still decides what the agent cannot see from code. Whether a new partition start date means a backfill, and who pays for it, depends on the warehouse. Whether the resource the pull request adds under `PROD` points at the right account is a credentials question the sandbox has no access to. The review lists these as open questions for the author and does not guess.

## Run it on every pull request opened

In the Project that holds the repository, create an automation with this prompt, read-only switched on, and a `github` trigger on `pull_request.opened` and `pull_request.synchronize`. Read-only runs can read the repository and comment but cannot push, which is the right permission for a reviewer. The trigger puts the pull request after the prompt and fills in nothing else; set its `branch_patterns` to the base branch you deploy from.

In a repository that already creates a Dagster+ branch deployment for each pull request, the two run side by side: the deployment shows the changed assets in the Dagster UI, and the review says whether the code loads and what the diff breaks. The trigger fires for work by an owner, member or collaborator only. A pull request from a fork never fires it, so a maintainer who wants one reviewed pushes its commits to a branch in the repository and opens the pull request from there.

## Why run it on Rush?

Loading a Dagster code location is not free. Each project here has its own lockfile, the main one needs a parsed dbt project before its definitions import, and the environment variable that selects resources changes what gets constructed. A reviewer who wants to know whether a branch loads has to rebuild that on their own machine, usually on top of the virtual environment of whatever they were working on. The agent builds it from the lockfile in a fresh sandbox for each pull request, and when several pull requests are open each gets its own sandbox at the same time.

It also gives the author something a green deploy does not: the exact command and output behind every finding. A branch deployment that fails to load shows an error in a UI after an image build; the review shows the same traceback against the diff, next to the string reference that caused it, and the whole record is readable from the session afterwards.

## Frequently asked questions

### Does the agent need Snowflake or Dagster+ credentials?

No. It loads the definitions with the project's local resources and parses dbt against the DuckDB target, neither of which opens a warehouse connection. Anything that can only be confirmed against the real warehouse or the deployed code location is listed in the review as not checked, with the reason.

### Will it materialize my assets?

Not against real storage. The prompt forbids it, and loading definitions does not run them. Tests that execute a job in process against a temporary DuckDB file, as the existing asset tests in this kind of project do, run as part of pytest.

### What does it check in dbt model changes?

It parses the project, so a model that references a missing source or macro fails with dbt's own error. It then reads the changed SQL and YAML against the Dagster side: a renamed model is an asset key change, and a model moved between the groups in the dbt component's YAML changes its partitioning and schedule.

### Can it fix what it finds?

Not in a read-only automation, by design. To have a finding fixed, start a separate run on the pull request's branch with a prompt that names the finding, and it pushes a commit there for the author to review.

## More Dagster jobs

- [Upgrade Python Dependencies Weekly for a Dagster Project](https://getrush.ai/run/upgrade-python-dependencies/dagster/)
- [Write Dagster Asset Tests for a Project](https://getrush.ai/run/write-asset-tests/dagster/)
- [Add Asset Checks to a Dagster Project from a Linear Issue](https://getrush.ai/run/add-asset-checks/dagster/)

## The same job on other stacks

- [Review Pull Requests on a dbt Project](https://getrush.ai/run/review-pull-requests/dbt/)
- [Review Pull Requests in a Deno App](https://getrush.ai/run/review-pull-requests/deno/)
- [Review Pull Requests for a discord.py Bot](https://getrush.ai/run/review-pull-requests/discord-py/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
