# Write Dagster Asset Tests for a Project

Yes. An agent writes pytest tests for the Dagster assets, ops and resources that have none, runs them in each project's own uv environment and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/write-asset-tests/dagster/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent clones the repository into its own cloud sandbox and maps which code locations have tests. In a workspace like dagster-io/hooli-data-eng-pipelines, `dg.toml` lists six projects, each with its own `pyproject.toml`, `uv.lock` and a `pytest` entry under the `dev` extra. Only `hooli-data-eng` has real tests: `hooli_data_eng_tests/test_assets.py` and `test_repo_loads.py`. The `test_assets.py` files in `hooli-bi` and `hooli-airlift` are empty, and `hooli-basics`, `hooli-batch-enrichment` and `hooli-data-ingest` have no test directory at all. The deploy workflow, `.github/workflows/deploy-dagster-cloud.yml`, runs Ruff with `check --fix` and deploys the code locations whose files changed; it never runs pytest. The one existing test module also imports `hooli_data_eng.assets.raw_data`, but the package now keeps its assets under `defs/` (`pandas`, `dbt`, `custom_ingest`, `scikit_learn` and others) and has no `assets` module, so that file cannot import as it stands.

For `hooli-batch-enrichment` the agent reads `assets.py`, `api.py`, `warehouse.py` and `definitions.py`. `raw_data` is an asset that reads a CSV through `MyWarehouse`, a `ConfigurableResource` with a `path` that runs `duckdb.sql(...).df()`, logs the row count and attaches `preview`, `nrows` and `user_input` metadata from an `experimentConfig` whose `experiment_name` defaults to `default_config`. `enriched_data` is a `graph_asset` of three ops: `split_rows` yields `DynamicOutput` chunks of `number_records_per_batch` rows (default 50), `process_chunk` carries `RetryPolicy(max_retries=2)` and calls `EnrichmentAPI.get_order_details`, which fakes `http://api.jaffleshop.co/v1/order_details` with the `responses` library and raises `API time out` on roughly one call in five hundred, and `concat_chunk_list` joins the frames. The tests the agent writes in `hooli-batch-enrichment/hooli_batch_enrichment_tests/` build `raw_data` with `build_asset_context` and a `MyWarehouse` pointed at a small CSV in `tmp_path`, assert the frame and the metadata keys, check that `split_rows` yields three mapping keys for a 120-row frame at the default batch size, run `process_chunk` with a stub `api` resource that returns a fixed `order_center` so the random time-out cannot make the suite flaky, materialize `raw_data` and `enriched_data` together with `materialize` and assert the `order_center` column, and load `defs` to check that `run_etl_pipeline` and the `run_assets_30min` schedule with its `*/30 * * * *` cron exist.

For `hooli-basics` the whole project is `definitions.py`: `country_stats` calls `read_html` on a web page with the `html5lib` flavor, renames six columns and computes `pop_change` as `(pop_2023 / pop_2022 - 1) * 100`; `check_country_stats` is an `asset_check` that always passes; `change_model` fits a scikit-learn `LinearRegression` on continent dummies and returns a `MaterializeResult` whose metadata holds `r2_score`, `rmse`, `mae`, `num_training_samples`, `intercept` and `coefficients`; `continent_stats` groups by continent. The tests patch `read_html` to return a six-column frame so nothing touches the network, assert the renamed columns and the `pop_change` arithmetic on known numbers, check that `num_training_samples` equals the rows with a `pop_change`, that `coefficients` has one key per continent dummy, that the check returns `passed=True`, and that `defs` loads three assets and one check. `hooli-data-ingest` is Sling replication from `component.yaml` and `replication.yaml` with no Python logic beyond a schedule, so the agent adds only a definitions-load test there and says why.

The agent runs `uv sync --extra dev` in each project, since every one has a `uv.lock`, then `uv run pytest` there, and formats the new files with `ruff format` the way the `Makefile`'s `ruff` target does. It also repairs the stale import in `hooli-data-eng/hooli_data_eng_tests/test_assets.py` by finding where `orders` and `users` now live under `defs/`, or marks the module skipped with the reason when the assets changed shape, and says which in the pull request. Finally it adds a `pytest` step to the deploy workflow before the deploy job, scoped to the projects whose files changed using the `changed-files` outputs the workflow already computes.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Write pytest tests for the Dagster projects in this workspace that have none. Read dg.toml, every project's pyproject.toml and definitions, and the existing tests under hooli-data-eng first, and follow their naming for the new test directories. For hooli-batch-enrichment, cover raw_data with a temporary CSV, split_rows chunking, process_chunk with a stubbed api resource so the random time-out cannot flake, a full materialize of both assets, and a definitions load that finds the job and schedule. For hooli-basics, patch read_html so no test touches the network, cover the column renames and pop_change math, the change_model metadata, the asset check and a definitions load. Fix or skip the existing hooli-data-eng test whose import no longer resolves and explain which you did. Run uv sync and uv run pytest in each project, format with ruff, add a pytest step to the deploy workflow for changed projects, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that adds `hooli-batch-enrichment/hooli_batch_enrichment_tests/`, `hooli-basics/hooli_basics_tests/` and a definitions-load test for `hooli-data-ingest`, changes `hooli-data-eng/hooli_data_eng_tests/test_assets.py`, and adds a pytest step to `.github/workflows/deploy-dagster-cloud.yml`. The description lists each test by name with the behaviour it pins, pastes the `uv run pytest` output per project, and explains the one stub, the `api` resource, and the one patch, `read_html`.

The run's record keeps every `uv sync` and `pytest` invocation with its output, so a reviewer can see each project's tests pass in an environment built from its own lockfile, not a shared one.

Before merging, a reviewer checks that the stub for `EnrichmentAPI` does not hide the retry behaviour worth testing, decides whether the `read_html` patch should become a fixture file committed under the tests, and confirms the new workflow step runs on branch deployments as well as on `main`.

## Run it on every new code location added to dg.toml

In the console, an automation in the Project that has the repository, with a `github` trigger on `pull_request.opened` and `pull_request.synchronize` and `branch_patterns` for your default branch, starts one run per pull request with the pull request as its context. The prompt tells the run to look for a project in `dg.toml` whose `*_tests` directory is missing or empty, write its first tests on the pull request's branch, and comment with nothing to do when every project is covered.

From your own server, the same work is one `POST /o/{org}/sessions` with this body and the repository in `resources`, sent with a `request_id` that is a UUID v5 of the pull request URL so a retried start returns the run already begun. A webhook URL on the API key reports `run.completed` with the `session_url` to read the pull request link from.

## Why run it on Rush?

Dagster tests fail for reasons that have nothing to do with the assets: a resource that reaches the network, a random failure baked into a fake API, an import that moved when the project adopted the `defs/` layout. The agent reads the resources and the ops before it writes a test, stubs exactly the two calls that leave the process, and runs every suite in the project's own `uv` environment, so the pull request carries tests that pass for the reason they should.

The sandbox is its own computer, each `uv sync` and `pytest` run stays in the record with its output, and a workspace with six code locations gets all six covered in one run rather than one afternoon per project.

## Frequently asked questions

### Why stub the enrichment API instead of testing it as written?

Because `EnrichmentAPI.get_order_details` throws `API time out` at random, about once in five hundred calls, to demonstrate retries. A test that calls it directly fails now and then for no change in the code. The stub returns a fixed `order_center`; the `RetryPolicy` on `process_chunk` is asserted separately by reading the op definition.

### What happens to the existing test that no longer imports?

The agent looks for `orders` and `users` under `hooli_data_eng/defs/` and repoints the import when the assets still take the same context and resources. When their shape changed too, it marks the module with `pytest.mark.skip` and the reason, so the suite is green and the pull request names the test that needs a human.

### Does the run need Dagster Cloud credentials or a warehouse?

No. The tests build contexts with `build_asset_context` and `materialize` against in-process resources: a DuckDB read of a temporary CSV and a stubbed API. `DAGSTER_CLOUD_API_TOKEN` and the Snowflake, Databricks and Power BI resources in the other projects are never used.

### Will the tests run in CI after this?

Yes. The deploy workflow gains a `pytest` step before the deploy job, limited to the projects whose files changed, using the `changed-files` outputs the workflow already computes for the deploy itself. A pull request that breaks an asset now fails before a branch deployment is built.

## More Dagster jobs

- [Add Asset Checks to a Dagster Project from a Linear Issue](https://getrush.ai/run/add-asset-checks/dagster/)
- [Add Concurrency Limits to a Dagster Deployment](https://getrush.ai/run/add-concurrency-limits/dagster/)
- [Set Up Dependabot for a Dagster Workspace with uv](https://getrush.ai/run/add-dependabot/dagster/)

## More jobs like this: Write the missing tests

- [Write bUnit Tests for a Blazor Web App](https://getrush.ai/run/write-bunit-tests/blazor/)
- [Write Tests for an AWS CDK App](https://getrush.ai/run/write-cdk-assertion-tests/aws-cdk/)
- [Write Tests for a Go CLI](https://getrush.ai/run/write-cli-tests/go-cli/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
