What the agent does
The agent starts from what the project already uses to check itself, which in an Astro project like astronomer/airflow-example-dags is nothing. There is no .github/ directory and no tests. The only pipeline is .drone.yml, four Drone steps that run docker build and docker push against a registry under astronomer-trials.com with an astro_api_key secret, on pushes to master, dev and release-* branches, through the astronomerio/ap-build:0.0.7 image. Nothing checks a pull request, and nothing checks that the DAG files import at all. The Dockerfile is one line, FROM quay.io/astronomer/astro-runtime:6.0.0, a runtime built on Airflow 2.4; requirements.txt is empty and packages.txt adds postgresql. .astro-registry.yaml lists the five DAG files meant for the Astronomer Registry, and airflow_settings.yaml holds empty connection, pool and variable placeholders.
It then reads dags/ to learn what a parse of this project needs. Six files: s3_upload_dag.py, s3_upload_copy_dag.py and s3_upload_copy_delete.py import S3Hook and the S3 copy and delete operators from the Amazon provider; databricks_sample.py imports DatabricksRunNowOperator, S3KeySensor and PostgresOperator; kube_pod.py imports KubernetesPodOperator from the Kubernetes provider and, at import time, calls conf.get('kubernetes', 'NAMESPACE'), so the config section has to exist when the file is parsed; and example_external_task_sensor.py is commented out from the first line to the last. Four files import from the airflow.operators.dummy_operator shim, three of them from airflow.operators.python_operator as well, and all use schedule_interval=, which parse on Airflow 2.4 with deprecation warnings. The workflow has to install Airflow at the runtime's version with the Amazon, Databricks, Kubernetes and Postgres providers, or every DAG fails to import for a reason that has nothing to do with the code.
The agent writes tests/dags/test_dag_integrity.py: it loads dags/ through DagBag(include_examples=False), asserts import_errors is empty, asserts the DAG ids it finds are the five the registry file names, since the commented-out file contributes none, and asserts every DAG has at least one task and an owner. It writes .github/workflows/ci.yml on pull_request and on pushes to the default branch with three jobs. dags sets up the Python the runtime image ships, installs the Airflow release the image carries with the matching constraints file plus those four providers, pinned in a requirements-ci.txt so the job is reproducible, and runs pytest tests/dags. lint runs ruff check dags tests with a short ruff.toml that leaves the deprecation-shim imports to the test rather than failing on style. image runs docker build . so a change to Dockerfile, packages.txt or requirements.txt that breaks the Astro image is caught before astro deploy, which is what the Drone pipeline was really checking.
Before it opens anything the agent proves the workflow in its sandbox: it installs that same Airflow and the providers, runs the DagBag test and reads its output, and runs ruff. The sandbox has no Docker, so for the image job it installs it first (sudo apt-get install -y docker.io, then starts dockerd) and runs docker build . once to see the runtime image pull and the postgresql package install. It does not touch .drone.yml: whether the Drone build and push stay or go is a deployment decision for the team, and the pull request says so.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add GitHub Actions CI to this Astro Airflow project. Read the Dockerfile, requirements.txt, packages.txt, airflow_settings.yaml, .astro-registry.yaml, .drone.yml and every file under dags/ first, and note the Airflow version the astro-runtime image carries and which provider packages the DAG imports need. Write tests/dags/test_dag_integrity.py that loads dags/ with DagBag(include_examples=False), asserts there are no import errors, asserts the DAG ids match the DAGs the registry file lists, and asserts every DAG has at least one task and an owner. Write .github/workflows/ci.yml on pull_request and on pushes to the default branch with three jobs: install Airflow at the runtime's version with its constraints file and the needed providers from a pinned requirements-ci.txt and run pytest tests/dags; run ruff check over dags and tests with a ruff.toml that does not fail on the deprecated import shims; and run docker build . to prove the Astro image still builds. Run the test, ruff and the docker build yourself in the sandbox before opening the pull request, leave .drone.yml unchanged, and describe in the pull request what each job checks and what it found.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request on its own branch with .github/workflows/ci.yml, tests/dags/test_dag_integrity.py, requirements-ci.txt, ruff.toml and nothing else changed; Dockerfile, packages.txt, airflow_settings.yaml and .drone.yml are as they were. Its description says which Airflow release and providers the test job installs and why, lists the five DAG ids the integrity test expects, and quotes the DagBag result, the ruff result and the tail of the docker build output from the sandbox.
The three jobs run on the pull request itself, so the first evidence that the workflow works is the checks on that PR. The commands the agent ran and their full output are in the session's transcript at GET /o/{org}/sessions/{id}/messages, and the session read names the pull request once it is open.
A reviewer checks three things before merging: that the pinned Airflow and provider versions in requirements-ci.txt match the runtime image the project deploys, so the parse test exercises what production runs; that the DAG id list in the test is the one they want enforced, since adding a DAG will now mean adding it there too; and whether the Drone pipeline should be removed in a follow-up now that the image build is covered. A follow-up message to the same session can add a tests/dags/test_dag_conventions.py that fails on the deprecated shim imports once the team is ready to migrate them.
Run it on every terminal hand-off
A data engineer who just inherited a DAG repository with no checks starts this from the terminal with this body, then goes back to the pipeline that is late. The run reads the actual Dockerfile and dags/, so the Airflow version, the provider list and the DAG ids in the workflow come from the project, not from a template.
Send a request_id, a UUID, with the body so a retried start returns the run it already began instead of a second pull request. When the run finishes, its GET /o/{org}/sessions/{id} answer names the pull request.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
Writing an Airflow CI job on a laptop means installing a specific Airflow release and four providers into a clean environment, discovering which DAG fails to import and why, and then pushing to see the same thing happen on a runner. The agent does that in its own sandbox, with the real DagBag output in front of it, and installs Docker there to see the image build go through before it asks anyone to review a workflow file.
Every command the run executed, from the pip install with the constraints file to the docker build, is kept with its output in the session record, so the pull request's claims about what passes are checkable. A team with several Astro deployments can send the same request for each repository at once and get one pull request per project, each built from that project's own runtime version and imports.
Frequently asked questions
Why install Airflow in the workflow instead of running astro dev parse?
astro dev parse needs the Astro CLI and Docker, and it parses inside the runtime image, which is slower on a runner and harder to read when it fails. A pip install of the same Airflow release with its constraints file gives the same import check in plain pytest output, and the image job still proves the Dockerfile builds. The pull request names both options so the team can switch if they prefer the CLI.
What happens to the commented-out DAG file?
Nothing. example_external_task_sensor.py has every line commented out, so DagBag loads no DAG from it and the test expects exactly the five ids the registry file lists. If someone uncomments it, the id assertion fails and tells them to add it to the list, which is the point of the check.
Does this replace the Drone pipeline?
Not in this pull request. .drone.yml builds and pushes an image to a registry with a secret the workflow cannot see, which is deployment, not CI. The new image job covers the build half; whether to drop Drone and deploy from GitHub Actions is left to the team and named as a follow-up.
Will the deprecated operator imports fail the build?
No. airflow.operators.dummy_operator and airflow.operators.python_operator still import on Airflow 2.4 with a warning, and the ruff config does not turn warnings into failures. The DagBag test passes as the DAGs are written today. A later run can migrate them and add a warnings-as-errors check once the code is ready.