All jobs

Add OpenTelemetry Tracing to a Ktor Server

Yes. An agent installs the KtorServerTelemetry plugin in a Ktor 3 server, exports OTLP traces with the service name set, adds a test and opens a pull request.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the server before it adds anything. In a repository shaped like ktorio/ktor-samples the target is the kweet sample: kweet/build.gradle.kts applies io.ktor.plugin 3.6.0 and Kotlin 2.4.20, pins jvmToolchain(17), and depends on ktor-server-netty-jvm, FreeMarker, Sessions, Resources, ktor-server-call-logging and Exposed on an H2 file database through a c3p0 pool, with mockk and ktor-server-test-host-jvm for tests. kweet/gradle.properties holds kotlin_version, logback_version 1.5.21 and exposed_version, the wrapper is Gradle 9.2.1, and application.conf starts io.ktor.samples.kweet.KweetApplicationKt.main on port 8080. In KweetApplication.kt, mainWithDependencies(dao) installs DefaultHeaders, CallLogging, ConditionalHeaders, PartialContent, Resources, FreeMarker and Sessions before the routing block registers the index, post, delete, user, view, login and register routes from their own files.

The same repository has an opentelemetry sample, and the agent reads it as the reference for versions and shape rather than guessing: its gradle.properties pins opentelemetry_version 2.14.0 for io.opentelemetry.instrumentation:opentelemetry-ktor-3.0 (with the -alpha suffix), opentelemetry-sdk-extension-autoconfigure and opentelemetry-exporter-otlp at 1.56.0 and opentelemetry-semconv at 1.37.0. Its shared/.../utils.kt builds an AutoConfiguredOpenTelemetrySdk with a resource customizer that sets ServiceAttributes.SERVICE_NAME, and its server/.../setupServerTelemetry.kt installs KtorServerTelemetry with setOpenTelemetry, capturedRequestHeaders, capturedResponseHeaders and a spanStatusExtractor. The root build.gradle.kts there sets OTEL_METRICS_EXPORTER=none and OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/ as environment for JavaExec tasks and starts Jaeger with the docker-compose plugin.

The change to kweet is small and follows that shape. The agent adds the four OpenTelemetry coordinates to kweet/build.gradle.kts and their versions to kweet/gradle.properties, writes a Telemetry.kt next to KweetApplication.kt that builds the autoconfigured SDK with the service name kweet and installs KtorServerTelemetry with the User-Agent request header and Content-Type response header captured, and calls it at the top of mainWithDependencies so every route is traced, including the FreeMarker renders and the redirects after a post. It does not instrument Exposed or c3p0: database spans would need the JDBC instrumentation agent, and the pull request says so instead of half-doing it. The exporter stays OTLP over gRPC on the default endpoint, configured by the standard OTEL_* environment variables, so no collector address is hardcoded in the repository.

For a test the agent extends kweet/src/test/kotlin/KweetApplicationTest.kt, which already builds the app with testApplication and a relaxed mockk DAOFacade. It adds a case that installs an in-memory span exporter from opentelemetry-sdk-testing through the same Telemetry.kt entry point, requests /, and asserts one server span named for the GET with http.response.status_code 200 and the kweet service name on its resource. It runs ./gradlew test from the kweet directory until the suite passes, then pushes a branch and opens the pull request. The repository's .github/workflows/build-all.yml runs ./build-all.sh over every sample on pull requests to main under JDK 24, so the new test runs there as well.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add OpenTelemetry tracing to the kweet Ktor server in this repository. Use the opentelemetry sample in the same repository as the reference for plugin and SDK versions. Install KtorServerTelemetry in mainWithDependencies with an autoconfigured SDK whose service name is kweet, keep the exporter configured by the standard OTEL_* environment variables, do not hardcode a collector address, add a test with an in-memory span exporter that asserts a server span for GET /, run ./gradlew test in kweet, and open a pull request that says what is traced and what is not.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

What comes back is a pull request whose diff touches kweet/build.gradle.kts and kweet/gradle.properties for the dependencies and versions, a new kweet/src/main/kotlin/io/ktor/samples/kweet/Telemetry.kt, one call added in KweetApplication.kt, and the new case in kweet/src/test/kotlin/KweetApplicationTest.kt. The description names the plugin version, the headers captured, the environment variables that select the exporter and endpoint, and the one thing it leaves out, database spans.

The checks are the proof. The build-all workflow runs on the pull request as it does for every change in the repository, and the trajectory at GET /o/{org}/sessions/{id}/messages keeps each ./gradlew test invocation with its output, so a reviewer can see the span assertion pass rather than take the description's word for it.

Before merging, a reviewer checks that OTEL_SERVICE_NAME or the resource customizer is the one source of the service name, that no span attribute captures the SESSION cookie or a form password, since the plugin captures only the headers named in capturedRequestHeaders, and that the test's in-memory exporter never leaks into the main source set. A reviewer running the sample locally sets OTEL_EXPORTER_OTLP_ENDPOINT to a collector and watches a request to / arrive as a trace.

Run it on every service onboarded to tracing

A platform team bringing services onto one collector sends POST /o/{org}/sessions once per repository with this prompt and that repository in resources. Each run reads its own build file, so a server already on ktor-server-call-logging keeps it and gains spans beside it, a server on Maven gets the same coordinates in its pom.xml, and each run ends in its own pull request.

Set a webhook_url on the API key and Rush POSTs run.completed or run.failed when each turn finishes, with session_url to read the pull request from. Past its max_concurrent runs, 4 by default, the key answers 202 with status: "queued", so a script that onboards twenty services can send all twenty and Rush starts each waiting run when a slot frees.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Wiring OpenTelemetry into a Ktor server by hand means finding the right instrumentation artifact for Ktor 3, matching SDK and semconv versions that resolve together, and then running a Gradle build long enough to see whether they do. On Rush the run does that in its own cloud sandbox with the sample's Gradle 9.2.1 wrapper and the JDK 17 toolchain the build file declares, and it reads the repository's own opentelemetry sample for the versions instead of a blog post.

The result comes with its evidence. The pull request shows exactly where the plugin is installed and what it captures, the repository's build-all workflow runs the test on the pull request, and the trajectory keeps every Gradle command and its output. Nothing reaches main until the pull request merges.

Frequently asked questions

Does it send traces anywhere during the run?

No. The exporter is selected by the standard OTEL_* environment variables, which the sandbox does not set, and the test uses an in-memory exporter. Where traces go is decided in your deployment environment, not in the repository.

Will it trace the database too?

Not in this pull request. Exposed and the c3p0 pool need the JDBC instrumentation, which is a separate dependency with its own configuration, and the pull request description says so. A follow-up on the same session can add it once the HTTP spans are reviewed.

Which versions does it pick?

The ones the repository already uses in its opentelemetry sample: opentelemetry-ktor-3.0 2.14.0-alpha, the SDK autoconfigure and OTLP exporter at 1.56.0 and semconv 1.37.0. If your repository has no such sample, the agent reads the Ktor version from the build file and picks the matching instrumentation artifact, and names the versions in the description.

Does it replace CallLogging?

No. CallLogging stays installed and keeps writing one Logback line per request; the telemetry plugin adds spans beside it. Linking the two with a trace id in the log pattern is a one-line change the pull request mentions but does not make.

How do I confirm the test ran?

Open the session's trajectory at GET /o/{org}/sessions/{id}/messages and read the ./gradlew test output, or wait for the build-all workflow on the pull request. A run whose suite could not pass opens no pull request and says so in its result, which the run.completed webhook carries.

More Ktor jobs

The same job on other stacks

Keep reading