# Set Up GitHub Actions CI for a Dagster Project

Yes. An agent adds a GitHub Actions workflow that installs each Dagster code location with uv, loads its definitions, runs its pytest suite per PR, and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-github-actions-ci/dagster/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent reads the workspace layout first. In a repository like dagster-io/hooli-data-eng-pipelines, `dg.toml` declares a workspace of six projects, `hooli-data-eng`, `hooli-data-ingest`, `hooli-bi`, `hooli-basics`, `hooli-snowflake-insights` and `hooli-batch-enrichment`, with a seventh, `hooli-airlift`, commented out with a note that it currently fails on an import error. Each project has its own `pyproject.toml` built with hatchling, a `dev` extra, a `uv.lock` and a `Dockerfile`; `hooli-data-eng`, `hooli-bi` and `hooli-basics` declare `requires-python >= 3.9, < 3.13` and list `pytest` in that extra, and `hooli-data-eng/Dockerfile` starts from the `uv:python3.12-bookworm` image and runs `uv sync --frozen`. `dagster_cloud.yaml` names each code location's module, such as `hooli_data_eng.definitions`, and the `Makefile` has `dependencies`, `deps` for `dbt deps`, `manifest` for `dbt parse --project-dir=dbt_project --target BRANCH`, and `ruff`.

Tests exist but nothing runs them. `hooli-data-eng/hooli_data_eng_tests/test_assets.py` materializes the `orders` and `users` partitioned assets with `RawDataAPI(flaky=False)` and a DuckDB IO manager writing `test.duckdb`; `test_repo_loads.py` asserts a job can be found; `test_databricks_pipes.py` calls `materialize` at import time and says in a comment that it needs `DAGSTER_CLOUD_DEPLOYMENT_NAME`, `DATABRICKS_HOST` and `DATABRICKS_TOKEN` to mean anything; `hooli-bi` and `hooli-airlift` have `test_assets.py` files of their own. The only workflow, `deploy-dagster-cloud.yml`, is a Dagster+ hybrid deploy: it runs `ruff check --fix` through `astral-sh/ruff-action`, detects which locations changed, builds a Docker image per changed location, pushes it to ECR and deploys a branch deployment, with a `dagster-cloud ci check` on the configuration. It never runs pytest, and a `ruff check --fix` step reformats rather than fails.

The agent writes `.github/workflows/test.yml` on `pull_request` and push to `main` with a matrix over the six active projects from `dg.toml`, so a failure names its location. Each job uses `astral-sh/setup-uv` with caching, Python 3.12 to match the Dockerfile, `uv sync --frozen --extra dev` in the project directory, then two checks: a definitions load, importing the module `dagster_cloud.yaml` names for that location and constructing its `Definitions`, and `uv run pytest` over the project's tests directory. For `hooli-data-eng` the dbt assets need a manifest before definitions load, so that job runs `dbt deps` and `dbt parse` first the way the Makefile does; the `BRANCH` and `PROD` targets in `dbt_project/profiles.yml` are Snowflake with credentials read from environment variables that default to empty strings, and the `LOCAL` target is DuckDB, so the agent checks in the sandbox whether `dbt parse` against `BRANCH` needs a connection and uses `LOCAL` if it does. `test_databricks_pipes.py` is deselected in CI with a `pytest` marker and a skip when `DATABRICKS_HOST` is unset, since it talks to a real workspace. A separate `lint` job runs `ruff check` and `ruff format --check` without `--fix`.

Before writing the YAML the agent runs every step itself: the sync for each project, the manifest build, the definitions load and pytest for the locations that have tests. Where `test_repo_loads.py` imports a name that no longer exists in the package, the agent fixes the test to load definitions the way `dagster_cloud.yaml` does rather than deleting it, and says so. Projects without a tests directory get the definitions load only, and the pull request names them so the team knows where tests are missing.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add GitHub Actions CI to this Dagster workspace. Read dg.toml, dagster_cloud.yaml, the Makefile, every project's pyproject.toml, uv.lock and Dockerfile, the dbt project's profiles.yml, every existing test file and the existing workflow first. Write .github/workflows/test.yml that runs on pull requests and pushes to the default branch with one matrix job per active code location: set up uv with caching and the Python version the Dockerfiles use, sync that project's locked dependencies with its dev extra, load the location's definitions module as dagster_cloud.yaml names it, and run pytest over its tests. For the location with dbt assets, build the dbt manifest first, using a target that needs no warehouse credentials. Skip any test that needs external credentials with a marker and a clear skip reason. Add a lint job that runs ruff check and ruff format --check without fixing. Run every step yourself before writing the workflow, fix tests that fail for reasons unrelated to credentials, and open a pull request that lists what each job runs and which locations have no tests.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request with `.github/workflows/test.yml`, a `conftest.py` or marker change in `hooli-data-eng/hooli_data_eng_tests/` that skips the Databricks test when its credentials are absent, a fix to `test_repo_loads.py` if its import was stale, and nothing else: `deploy-dagster-cloud.yml`, the Dockerfiles and `dagster_cloud.yaml` are unchanged, so deploys behave exactly as before.

The description has a table with one row per code location: the sync command, whether definitions loaded, the pytest summary line from the sandbox or the note that the project has no tests. The dbt manifest step and which profile target it used are called out. Every command and its output is in the transcript at `GET /o/{org}/sessions/{id}/messages`.

The workflow runs on the pull request itself, so a reviewer sees seven jobs on the checks tab before reading the YAML. They confirm the matrix matches `dg.toml`, read the skip reason on the Databricks test, and decide whether `hooli-airlift` should join the matrix once its import error is fixed, which a follow-up on the same session can attempt.

## Run it on every terminal hand-off

A data engineer who has been running pytest by hand before each deploy starts this from the terminal with this body and goes back to the pipeline that needs attention. The run reads the real `dg.toml`, lock files and `dagster_cloud.yaml`, so the matrix, the Python version and the manifest step are taken from the repository rather than from a template.

Send a `request_id`, a UUID, with the body so a retried start returns the run it already began. The pull request is on the run's `GET /o/{org}/sessions/{id}` answer once it is open, and a key with a `webhook_url` hears `run.completed` within about 30 seconds of the turn ending.

## Why run it on Rush?

A Dagster workspace with several code locations, a dbt project and a cloud deploy is exactly where a hand-written workflow fails on its first run: a manifest that is not there yet, a test that imports a Databricks client, a profile target that wants a warehouse. The agent runs the whole matrix in a sandbox first, with uv and the locked dependencies of each project, so the jobs it writes have each passed once before the pull request opens.

The run keeps every command and its output, so the pull request's table of what passed per location is backed by the transcript. Teams with several Dagster repositories can start one run each, in parallel, each in its own sandbox.

## Frequently asked questions

### Does CI need Snowflake or Databricks credentials?

No. The dbt manifest is built with a target that does not connect, the DuckDB-backed tests write a local `test.duckdb`, and the Databricks pipes test is skipped with a stated reason when `DATABRICKS_HOST` is unset. Adding the secrets later turns that skip into a real run.

### Will this slow down or change the Dagster+ deploy?

No. `deploy-dagster-cloud.yml` is untouched and keeps building and deploying changed locations. The new workflow is separate and runs in parallel; if you want deploys to wait for green tests, that is a one-line `needs` change the pull request describes but does not make.

### Why one job per code location?

Each project has its own `uv.lock` and Python dependencies, which is why each has its own Dockerfile. Installing them together would resolve a different set than production runs. A matrix also means a failure in `hooli-bi` is labelled as such rather than buried in one long log.

### What about the commented-out hooli-airlift project?

It stays out of the matrix, because `dg.toml` says it fails to import today, and the pull request notes that. Once the import error is fixed, adding it is one line in the matrix.

## More Dagster jobs

- [Add Retry Policies to the Flaky Assets in a Dagster Project](https://getrush.ai/run/add-retry-policies/dagster/)
- [Add a Schedule to a Dagster Project](https://getrush.ai/run/add-schedules/dagster/)
- [Add a Sensor to a Dagster Project](https://getrush.ai/run/add-sensors/dagster/)

## The same job on other stacks

- [Add a GitHub Actions CI Job That Runs dbt build on Every Pull Request](https://getrush.ai/run/add-github-actions-ci/dbt/)
- [Set Up GitHub Actions CI for an Eleventy Site](https://getrush.ai/run/add-github-actions-ci/eleventy/)
- [Set Up GitHub Actions CI for an Expo App](https://getrush.ai/run/add-github-actions-ci/expo/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
