# Add GitHub Actions CI to an Airflow Project

Yes. An agent adds a GitHub Actions workflow that parses every DAG with a DagBag test, lints them and builds your Astro image on each pull request, in a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-github-actions-ci/airflow/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent starts from what the project already uses to check itself, which in an Astro project like astronomer/airflow-example-dags is nothing. There is no `.github/` directory and no tests. The only pipeline is `.drone.yml`, four Drone steps that run `docker build` and `docker push` against a registry under `astronomer-trials.com` with an `astro_api_key` secret, on pushes to `master`, `dev` and `release-*` branches, through the `astronomerio/ap-build:0.0.7` image. Nothing checks a pull request, and nothing checks that the DAG files import at all. The `Dockerfile` is one line, `FROM quay.io/astronomer/astro-runtime:6.0.0`, a runtime built on Airflow 2.4; `requirements.txt` is empty and `packages.txt` adds `postgresql`. `.astro-registry.yaml` lists the five DAG files meant for the Astronomer Registry, and `airflow_settings.yaml` holds empty connection, pool and variable placeholders.

It then reads `dags/` to learn what a parse of this project needs. Six files: `s3_upload_dag.py`, `s3_upload_copy_dag.py` and `s3_upload_copy_delete.py` import `S3Hook` and the S3 copy and delete operators from the Amazon provider; `databricks_sample.py` imports `DatabricksRunNowOperator`, `S3KeySensor` and `PostgresOperator`; `kube_pod.py` imports `KubernetesPodOperator` from the Kubernetes provider and, at import time, calls `conf.get('kubernetes', 'NAMESPACE')`, so the config section has to exist when the file is parsed; and `example_external_task_sensor.py` is commented out from the first line to the last. Four files import from the `airflow.operators.dummy_operator` shim, three of them from `airflow.operators.python_operator` as well, and all use `schedule_interval=`, which parse on Airflow 2.4 with deprecation warnings. The workflow has to install Airflow at the runtime's version with the Amazon, Databricks, Kubernetes and Postgres providers, or every DAG fails to import for a reason that has nothing to do with the code.

The agent writes `tests/dags/test_dag_integrity.py`: it loads `dags/` through `DagBag(include_examples=False)`, asserts `import_errors` is empty, asserts the DAG ids it finds are the five the registry file names, since the commented-out file contributes none, and asserts every DAG has at least one task and an `owner`. It writes `.github/workflows/ci.yml` on `pull_request` and on pushes to the default branch with three jobs. `dags` sets up the Python the runtime image ships, installs the Airflow release the image carries with the matching constraints file plus those four providers, pinned in a `requirements-ci.txt` so the job is reproducible, and runs `pytest tests/dags`. `lint` runs `ruff check dags tests` with a short `ruff.toml` that leaves the deprecation-shim imports to the test rather than failing on style. `image` runs `docker build .` so a change to `Dockerfile`, `packages.txt` or `requirements.txt` that breaks the Astro image is caught before `astro deploy`, which is what the Drone pipeline was really checking.

Before it opens anything the agent proves the workflow in its sandbox: it installs that same Airflow and the providers, runs the DagBag test and reads its output, and runs `ruff`. The sandbox has no Docker, so for the image job it installs it first (`sudo apt-get install -y docker.io`, then starts `dockerd`) and runs `docker build .` once to see the runtime image pull and the `postgresql` package install. It does not touch `.drone.yml`: whether the Drone build and push stay or go is a deployment decision for the team, and the pull request says so.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add GitHub Actions CI to this Astro Airflow project. Read the Dockerfile, requirements.txt, packages.txt, airflow_settings.yaml, .astro-registry.yaml, .drone.yml and every file under dags/ first, and note the Airflow version the astro-runtime image carries and which provider packages the DAG imports need. Write tests/dags/test_dag_integrity.py that loads dags/ with DagBag(include_examples=False), asserts there are no import errors, asserts the DAG ids match the DAGs the registry file lists, and asserts every DAG has at least one task and an owner. Write .github/workflows/ci.yml on pull_request and on pushes to the default branch with three jobs: install Airflow at the runtime's version with its constraints file and the needed providers from a pinned requirements-ci.txt and run pytest tests/dags; run ruff check over dags and tests with a ruff.toml that does not fail on the deprecated import shims; and run docker build . to prove the Astro image still builds. Run the test, ruff and the docker build yourself in the sandbox before opening the pull request, leave .drone.yml unchanged, and describe in the pull request what each job checks and what it found.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request on its own branch with `.github/workflows/ci.yml`, `tests/dags/test_dag_integrity.py`, `requirements-ci.txt`, `ruff.toml` and nothing else changed; `Dockerfile`, `packages.txt`, `airflow_settings.yaml` and `.drone.yml` are as they were. Its description says which Airflow release and providers the test job installs and why, lists the five DAG ids the integrity test expects, and quotes the DagBag result, the ruff result and the tail of the `docker build` output from the sandbox.

The three jobs run on the pull request itself, so the first evidence that the workflow works is the checks on that PR. The commands the agent ran and their full output are in the session's transcript at `GET /o/{org}/sessions/{id}/messages`, and the session read names the pull request once it is open.

A reviewer checks three things before merging: that the pinned Airflow and provider versions in `requirements-ci.txt` match the runtime image the project deploys, so the parse test exercises what production runs; that the DAG id list in the test is the one they want enforced, since adding a DAG will now mean adding it there too; and whether the Drone pipeline should be removed in a follow-up now that the image build is covered. A follow-up message to the same session can add a `tests/dags/test_dag_conventions.py` that fails on the deprecated shim imports once the team is ready to migrate them.

## Run it on every terminal hand-off

A data engineer who just inherited a DAG repository with no checks starts this from the terminal with this body, then goes back to the pipeline that is late. The run reads the actual `Dockerfile` and `dags/`, so the Airflow version, the provider list and the DAG ids in the workflow come from the project, not from a template.

Send a `request_id`, a UUID, with the body so a retried start returns the run it already began instead of a second pull request. When the run finishes, its `GET /o/{org}/sessions/{id}` answer names the pull request.

## Why run it on Rush?

Writing an Airflow CI job on a laptop means installing a specific Airflow release and four providers into a clean environment, discovering which DAG fails to import and why, and then pushing to see the same thing happen on a runner. The agent does that in its own sandbox, with the real DagBag output in front of it, and installs Docker there to see the image build go through before it asks anyone to review a workflow file.

Every command the run executed, from the pip install with the constraints file to the `docker build`, is kept with its output in the session record, so the pull request's claims about what passes are checkable. A team with several Astro deployments can send the same request for each repository at once and get one pull request per project, each built from that project's own runtime version and imports.

## Frequently asked questions

### Why install Airflow in the workflow instead of running astro dev parse?

`astro dev parse` needs the Astro CLI and Docker, and it parses inside the runtime image, which is slower on a runner and harder to read when it fails. A pip install of the same Airflow release with its constraints file gives the same import check in plain pytest output, and the `image` job still proves the Dockerfile builds. The pull request names both options so the team can switch if they prefer the CLI.

### What happens to the commented-out DAG file?

Nothing. `example_external_task_sensor.py` has every line commented out, so DagBag loads no DAG from it and the test expects exactly the five ids the registry file lists. If someone uncomments it, the id assertion fails and tells them to add it to the list, which is the point of the check.

### Does this replace the Drone pipeline?

Not in this pull request. `.drone.yml` builds and pushes an image to a registry with a secret the workflow cannot see, which is deployment, not CI. The new `image` job covers the build half; whether to drop Drone and deploy from GitHub Actions is left to the team and named as a follow-up.

### Will the deprecated operator imports fail the build?

No. `airflow.operators.dummy_operator` and `airflow.operators.python_operator` still import on Airflow 2.4 with a warning, and the ruff config does not turn warnings into failures. The DagBag test passes as the DAGs are written today. A later run can migrate them and add a warnings-as-errors check once the code is ready.

## More Apache Airflow jobs

- [Add Pools to Limit Concurrent Tasks in Airflow](https://getrush.ai/run/add-pools/airflow/)
- [Add Retry Policies With Exponential Backoff to Airflow DAGs](https://getrush.ai/run/add-retry-policies/airflow/)
- [Move Airflow Connections to a Secrets Backend](https://getrush.ai/run/add-secrets-backend/airflow/)

## The same job on other stacks

- [Set Up GitHub Actions CI for an AWS CDK App](https://getrush.ai/run/add-github-actions-ci/aws-cdk/)
- [Set Up GitHub Actions CI for a Bazel Repository](https://getrush.ai/run/add-github-actions-ci/bazel/)
- [Set Up GitHub Actions CI for a Blazor Web App](https://getrush.ai/run/add-github-actions-ci/blazor/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
