What the agent does
The agent clones the repository into its own cloud sandbox and reads the Collector configuration before adding anything, because the demo layers several files and the order matters. In a system like open-telemetry/opentelemetry-demo, compose.yaml runs the otel-collector service from COLLECTOR_CONTRIB_IMAGE, pinned in .env to the contrib distribution at 0.160.0, with --config=/etc/otelcol-config.yml followed by --config=/etc/otelcol-config-extras.yml, a 400M memory limit, GOMEMLIMIT=160MiB, and an explicit environment list of the variables the config may read. compose.observability.yaml replaces that command with four files in order: base, otelcol-config-full.yml, otelcol-config-observability.yml and extras. The base file's traces pipeline exports to debug and the span_metrics connector; the observability layer adds otlp_grpc/jaeger, otlp_http/prometheus, opensearch and otlp_grpc/firepit, each with sending_queue.batch, and rewrites every pipeline's exporters list. otelcol-config-extras.yml is an empty stub whose comments explain the one rule that matters here: the Collector merges maps across files but replaces arrays, so a later file that names a pipeline's exporters must repeat every exporter the earlier files set.
The agent writes the exporter into the extras file, which is the seam the repository reserved for forks. It defines otlp_http/backend with endpoint: ${env:OTLP_BACKEND_ENDPOINT}, a headers map carrying the authentication header the backend documents from ${env:OTLP_BACKEND_API_KEY}, tls left at its secure default, retry_on_failure enabled and sending_queue.batch to match the neighbouring exporters. It then repeats the full exporter lists for traces, metrics and logs and appends the new name to each, choosing the observability layout the stub's own example assumes: [debug, otlp_grpc/jaeger, span_metrics, otlp_http/backend] for traces. Because the extras file is also loaded in the minimal layout where otlp_grpc/jaeger does not exist, the agent adds a second file, otelcol-config-extras-minimal.yml, with lists that repeat only the base exporters, and documents in the file header which compose layout each one belongs to.
The environment has to carry the two values to the container. The agent adds OTLP_BACKEND_ENDPOINT and OTLP_BACKEND_API_KEY to the environment list of the otel-collector service in compose.yaml and nothing to .env: the Makefile already passes --env-file .env --env-file .env.override to every compose command and creates .env.override with touch when it is missing, so the pull request tells the reader to put both values there or in the shell and never to commit the key. The config reads them with the ${env:...} form, the same way the base file reads POSTGRES_MONITORING_PASSWORD. The redaction and transform/redact_sensitive_data processors already in the traces pipeline run before any exporter, so what reaches the vendor is what reaches Jaeger today.
The sandbox has no Docker, so the agent proves the configuration with the Collector itself. It downloads the otelcol-contrib 0.160.0 release binary for Linux, exports the environment variables the files reference, and runs otelcol-contrib validate once per layout with the same ordered --config flags as each compose file, which catches an exporter named in a pipeline but never defined, or a repeated list that dropped span_metrics. It also reads compose.observability.yaml and the Makefile to confirm the extras file is mounted in every layout the repository supports, and runs the markdownlint and misspell targets from make check over the README it adds.
Start it with one API call
POST https://api.phoenix.so/o/{org}/sessions with this body:
{
"agent": "claude",
"input": "Add an OTLP exporter for an external observability backend to this repository's OpenTelemetry Collector configuration. Read every file under src/otel-collector, the otel-collector service in compose.yaml and compose.observability.yaml, and .env first, and note the order in which the config files are loaded and the warning that later files replace pipeline arrays. Put the exporter in the extras layer with the endpoint and the authentication header read from environment variables, keep TLS at its default, enable retries and the batching sending queue like the neighbouring exporters, and repeat every existing exporter in each pipeline before appending the new one; provide a second extras file for the minimal layout where the observability exporters do not exist. Pass the two variables through the collector service's environment list, read them from the env override file the Makefile already loads, and keep the key out of the repository. Download the pinned otelcol-contrib release and run validate for each layout with the same ordered config flags as the compose files. Open a pull request that names the layouts, the variables to set and the validate output.",
"resources": [
{
"url": "https://github.com/your-org/your-app"
}
]
}- Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
- Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
- Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.
What comes back
What comes back is a pull request that fills in src/otel-collector/otelcol-config-extras.yml, adds a sibling extras file for the minimal layout, adds two variables to the otel-collector service in compose.yaml, and adds a short README.md in src/otel-collector/ on choosing the extras file for each layout and where the two variables go. The description has the final exporters list per pipeline per layout, so a reviewer can check that nothing that exported before stops exporting.
The run's record keeps the otelcol-contrib validate output for each ordered set of config files, the download of the pinned release, and the Markdown checks that ran, so the review is about the exporter's settings rather than about whether the YAML loads.
Before merging, a reviewer sets the endpoint and key in their own .env.override, which the Makefile already loads, brings the stack up with the observability layout, and checks that traces, metrics and logs appear in the vendor while Jaeger and Prometheus still receive theirs. The redaction processors are unchanged, so what the vendor receives is what Jaeger already received.
Run it on every Collector repository
A platform engineer hands this off from the terminal: one POST /o/{org}/sessions with this body and the repository in resources. A team that runs several Collector deployments sends one request per repository, each with a request_id that is a UUID v5 of the repository path so a retry returns the run it already started, and the key's webhook_url reports each finished run with its pull request.
Each run reads that deployment's own layering and exporter names, so a repository with one config file gets the exporter written straight into its pipelines, and one with layers gets the repeated lists its layering demands.
The use case this job follows, with its automation and trigger: Hand off a long task.
Why run it on Rush?
The failure this change invites is silent: a pipeline whose exporters list was replaced instead of extended, so span metrics or Jaeger quietly stop while the new vendor receives everything. The agent reads every layer in its own sandbox, writes the lists out in full for each layout, and runs the real Collector's validate against the same ordered flags the compose files use, so the mistake is caught before a reviewer reads the diff.
Every validate run and every file read is in the run's record. The reviewer checks the exporter settings and the variable names, which is the part that needs a human, and trusts the loading because they can see it.
Frequently asked questions
Why does the exporter go in the extras file instead of the base config?
Because the repository reserved that file for exactly this: it is always loaded last, and its header says to override it in a fork. Writing the exporter there keeps the base and observability files identical to upstream, which makes future upstream merges painless.
Why two extras files?
A later file replaces a pipeline's exporter array, so the list must name exactly the exporters earlier files defined. The minimal layout defines debug and span_metrics; the observability layout also defines Jaeger, Prometheus and OpenSearch. One list cannot be right for both, so each layout points at its own extras file.
Where does the API key live?
In the environment of the collector container only. The config reads it with ${env:OTLP_BACKEND_API_KEY}, compose.yaml passes the variable through, and the pull request asks for it in .env.override, which the Makefile already loads, or in the shell. Nothing committed holds the value.
Will the vendor receive personal data?
No more than Jaeger does today. The traces pipeline runs transform/redact_sensitive_data and redaction before any exporter, and the new exporter is appended after them in the same pipeline, so the same hashed and masked attributes reach every backend.
How is the config checked without Docker?
With the Collector binary itself. The agent downloads the same contrib release the .env pins and runs otelcol-contrib validate with the ordered --config flags each compose layout uses, which is the same check the container does at start, minus the containers.