# Add an OTLP Exporter for Your Own Backend to the OpenTelemetry Collector

Yes. An agent adds an OTLP exporter for your vendor to the Collector's extras layer, keeps the existing exporters, validates every config layering and opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/add-otlp-exporter/opentelemetry/). Opens the console with this prompt ready to send, in a Project that has your repository.

## What the agent does

The agent clones the repository into its own cloud sandbox and reads the Collector configuration before adding anything, because the demo layers several files and the order matters. In a system like open-telemetry/opentelemetry-demo, `compose.yaml` runs the `otel-collector` service from `COLLECTOR_CONTRIB_IMAGE`, pinned in `.env` to the contrib distribution at 0.160.0, with `--config=/etc/otelcol-config.yml` followed by `--config=/etc/otelcol-config-extras.yml`, a 400M memory limit, `GOMEMLIMIT=160MiB`, and an explicit `environment` list of the variables the config may read. `compose.observability.yaml` replaces that `command` with four files in order: base, `otelcol-config-full.yml`, `otelcol-config-observability.yml` and extras. The base file's `traces` pipeline exports to `debug` and the `span_metrics` connector; the observability layer adds `otlp_grpc/jaeger`, `otlp_http/prometheus`, `opensearch` and `otlp_grpc/firepit`, each with `sending_queue.batch`, and rewrites every pipeline's `exporters` list. `otelcol-config-extras.yml` is an empty stub whose comments explain the one rule that matters here: the Collector merges maps across files but replaces arrays, so a later file that names a pipeline's `exporters` must repeat every exporter the earlier files set.

The agent writes the exporter into the extras file, which is the seam the repository reserved for forks. It defines `otlp_http/backend` with `endpoint: ${env:OTLP_BACKEND_ENDPOINT}`, a `headers` map carrying the authentication header the backend documents from `${env:OTLP_BACKEND_API_KEY}`, `tls` left at its secure default, `retry_on_failure` enabled and `sending_queue.batch` to match the neighbouring exporters. It then repeats the full exporter lists for `traces`, `metrics` and `logs` and appends the new name to each, choosing the observability layout the stub's own example assumes: `[debug, otlp_grpc/jaeger, span_metrics, otlp_http/backend]` for traces. Because the extras file is also loaded in the minimal layout where `otlp_grpc/jaeger` does not exist, the agent adds a second file, `otelcol-config-extras-minimal.yml`, with lists that repeat only the base exporters, and documents in the file header which compose layout each one belongs to.

The environment has to carry the two values to the container. The agent adds `OTLP_BACKEND_ENDPOINT` and `OTLP_BACKEND_API_KEY` to the `environment` list of the `otel-collector` service in `compose.yaml` and nothing to `.env`: the Makefile already passes `--env-file .env --env-file .env.override` to every compose command and creates `.env.override` with `touch` when it is missing, so the pull request tells the reader to put both values there or in the shell and never to commit the key. The config reads them with the `${env:...}` form, the same way the base file reads `POSTGRES_MONITORING_PASSWORD`. The `redaction` and `transform/redact_sensitive_data` processors already in the traces pipeline run before any exporter, so what reaches the vendor is what reaches Jaeger today.

The sandbox has no Docker, so the agent proves the configuration with the Collector itself. It downloads the `otelcol-contrib` 0.160.0 release binary for Linux, exports the environment variables the files reference, and runs `otelcol-contrib validate` once per layout with the same ordered `--config` flags as each compose file, which catches an exporter named in a pipeline but never defined, or a repeated list that dropped `span_metrics`. It also reads `compose.observability.yaml` and the Makefile to confirm the extras file is mounted in every layout the repository supports, and runs the `markdownlint` and `misspell` targets from `make check` over the README it adds.

## Start it with one API call

`POST https://api.phoenix.so/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Add an OTLP exporter for an external observability backend to this repository's OpenTelemetry Collector configuration. Read every file under src/otel-collector, the otel-collector service in compose.yaml and compose.observability.yaml, and .env first, and note the order in which the config files are loaded and the warning that later files replace pipeline arrays. Put the exporter in the extras layer with the endpoint and the authentication header read from environment variables, keep TLS at its default, enable retries and the batching sending queue like the neighbouring exporters, and repeat every existing exporter in each pipeline before appending the new one; provide a second extras file for the minimal layout where the observability exporters do not exist. Pass the two variables through the collector service's environment list, read them from the env override file the Makefile already loads, and keep the key out of the repository. Download the pinned otelcol-contrib release and run validate for each layout with the same ordered config flags as the compose files. Open a pull request that names the layouts, the variables to set and the validate output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

What comes back is a pull request that fills in `src/otel-collector/otelcol-config-extras.yml`, adds a sibling extras file for the minimal layout, adds two variables to the `otel-collector` service in `compose.yaml`, and adds a short `README.md` in `src/otel-collector/` on choosing the extras file for each layout and where the two variables go. The description has the final `exporters` list per pipeline per layout, so a reviewer can check that nothing that exported before stops exporting.

The run's record keeps the `otelcol-contrib validate` output for each ordered set of config files, the download of the pinned release, and the Markdown checks that ran, so the review is about the exporter's settings rather than about whether the YAML loads.

Before merging, a reviewer sets the endpoint and key in their own `.env.override`, which the Makefile already loads, brings the stack up with the observability layout, and checks that traces, metrics and logs appear in the vendor while Jaeger and Prometheus still receive theirs. The redaction processors are unchanged, so what the vendor receives is what Jaeger already received.

## Run it on every Collector repository

A platform engineer hands this off from the terminal: one `POST /o/{org}/sessions` with this body and the repository in `resources`. A team that runs several Collector deployments sends one request per repository, each with a `request_id` that is a UUID v5 of the repository path so a retry returns the run it already started, and the key's `webhook_url` reports each finished run with its pull request.

Each run reads that deployment's own layering and exporter names, so a repository with one config file gets the exporter written straight into its pipelines, and one with layers gets the repeated lists its layering demands.

## Why run it on Rush?

The failure this change invites is silent: a pipeline whose `exporters` list was replaced instead of extended, so span metrics or Jaeger quietly stop while the new vendor receives everything. The agent reads every layer in its own sandbox, writes the lists out in full for each layout, and runs the real Collector's `validate` against the same ordered flags the compose files use, so the mistake is caught before a reviewer reads the diff.

Every validate run and every file read is in the run's record. The reviewer checks the exporter settings and the variable names, which is the part that needs a human, and trusts the loading because they can see it.

## Frequently asked questions

### Why does the exporter go in the extras file instead of the base config?

Because the repository reserved that file for exactly this: it is always loaded last, and its header says to override it in a fork. Writing the exporter there keeps the base and observability files identical to upstream, which makes future upstream merges painless.

### Why two extras files?

A later file replaces a pipeline's exporter array, so the list must name exactly the exporters earlier files defined. The minimal layout defines `debug` and `span_metrics`; the observability layout also defines Jaeger, Prometheus and OpenSearch. One list cannot be right for both, so each layout points at its own extras file.

### Where does the API key live?

In the environment of the collector container only. The config reads it with `${env:OTLP_BACKEND_API_KEY}`, `compose.yaml` passes the variable through, and the pull request asks for it in `.env.override`, which the Makefile already loads, or in the shell. Nothing committed holds the value.

### Will the vendor receive personal data?

No more than Jaeger does today. The traces pipeline runs `transform/redact_sensitive_data` and `redaction` before any exporter, and the new exporter is appended after them in the same pipeline, so the same hashed and masked attributes reach every backend.

### How is the config checked without Docker?

With the Collector binary itself. The agent downloads the same contrib release the `.env` pins and runs `otelcol-contrib validate` with the ordered `--config` flags each compose layout uses, which is the same check the container does at start, minus the containers.

## More OpenTelemetry jobs

- [Add Tail-Based Sampling to an OpenTelemetry Collector Pipeline](https://getrush.ai/run/add-tail-sampling/opentelemetry/)
- [Drop Health Check Spans from OpenTelemetry Traces](https://getrush.ai/run/drop-health-check-spans/opentelemetry/)
- [Investigate a Microservices Outage from OpenTelemetry Data](https://getrush.ai/run/investigate-outage-with-grafana/opentelemetry/)

## More jobs like this: Hand off a long task

- [Add Output Caching to an ASP.NET Core Minimal API](https://getrush.ai/run/add-output-caching/dotnet/)
- [Add OWASP Dependency-Check to a Gradle Build](https://getrush.ai/run/add-owasp-dependency-check/gradle/)
- [Add OWASP Dependency-Check to a Maven Build](https://getrush.ai/run/add-owasp-dependency-check/maven/)

## Keep reading

- [How a Rush run works on your repository, from request to pull request](https://getrush.ai/cloud/)
- [Compare cloud agents, agent sandboxes and AI code review](https://getrush.ai/compare/)
- [Rush pricing: Free, Pro and Max](https://getrush.ai/pricing/)
