All jobs

Add Structured Logging to a Phoenix LiveView App

Yes. An agent turns a Phoenix LiveView app's production logs into JSON with a request id and user id on every line, at the release its Elixir allows, and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the logging the app already has before it changes the format, because a LiveView app logs from three places: plugs, LiveView processes and plain modules. In an app like fly-apps/live_beats, config/config.exs formats the console as $time $metadata[$level] $message with metadata: [:request_id], config/dev.exs shortens it to [$level] $message, config/test.exs sets the level to warn, and config/prod.exs sets info. lib/live_beats_web/endpoint.ex runs Plug.RequestId and Plug.Telemetry with the [:phoenix, :endpoint] prefix before the router, and Phoenix.LiveDashboard.RequestLogger. The code calls Logger.info with interpolated strings in four places: oauth_callback_controller.ex on a failed GitHub exchange or insert, file_controller.ex when serving or proxying a file, and media_library.ex when a song file cannot be deleted; lib/live_beats.ex logs a Logger.error when an attached event handler raises. The current user is set in user_auth.ex by fetch_current_user for plugs and by on_mount(:current_user) and on_mount(:ensure_authenticated) for LiveViews, and router.ex puts both in live_session blocks. fly.toml deploys to Fly, whose log shipper reads one line per record from stdout.

The toolchain decides the library release. mix.exs allows elixir: "~> 1.12" and the Dockerfile builds on hexpm/elixir:1.14.0-erlang-24.0.1, while on Hex logger_json 6.0.0 and 7.0.0 require ~> 1.16 or ~> 1.15.1 and 5.1.4 allows ~> 1.9. So the agent adds logger_json at the newest 5.x, which keeps the Elixir 1.14 image working, and says in the pull request that moving the image to 1.15 or later makes the 7.x line and its :default_handler formatter available. With 5.x, config/prod.exs sets config :logger, backends: [LoggerJSON] and config :logger_json, :backend with the basic formatter and metadata: :all, while dev and test keep their text formats so mix test output stays readable.

Then every line gets the ids that make it searchable. Plug.RequestId already puts request_id into Logger metadata for the plug pipeline, so fetch_current_user adds Logger.metadata(user_id: user.id) when a session holds a user. A LiveView runs in its own process with its own metadata, so the two on_mount clauses in user_auth.ex set user_id there, and LiveBeatsWeb.Nav's on_mount adds the region it already assigns, so a slow handle_event in player_live.ex logs with the same keys as the request that mounted it. The agent turns config :phoenix, :logger off in prod so Phoenix's two-line text request log stops, and emits one JSON request line from the [:phoenix, :endpoint, :stop] event Plug.Telemetry already fires, with method, path, status and duration. The four interpolated Logger.info calls keep their messages and move the song id, file path, IP and changeset errors into metadata keys, so a search can group them.

The check runs with the app's own test setup. The sandbox has no Docker, so the agent installs Postgres with the postgres/postgres credentials config/test.exs expects, installs the Erlang and Elixir pair the Dockerfile pins, sets BUMBLEBEE_CACHE_DIR because application.ex loads the Whisper model at boot, and runs mix test, which the test alias precedes with ecto.create and ecto.migrate. It adds a test under test/live_beats_web/ that configures the JSON backend for one test with ExUnit.CaptureLog at info, requests /signin through ConnCase, decodes the captured line with Jason and asserts request_id, method, path and status are present, and a second case that mounts SignInLive and checks a log from the LiveView process carries user_id. mix format --check-formatted runs with the project's .formatter.exs before the pull request opens.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add structured JSON logging to this Phoenix LiveView app. Read mix.exs, the Dockerfile, config/config.exs, config/dev.exs, config/test.exs, config/prod.exs, config/runtime.exs, lib/live_beats_web/endpoint.ex, lib/live_beats_web/router.ex, lib/live_beats_web/controllers/user_auth.ex, lib/live_beats_web/live/nav.ex, fly.toml and every Logger call under lib first. Check on Hex which logger_json release the project's pinned Elixir allows and add that one; say what a newer Elixir would allow. Configure the JSON backend in prod only, keep dev and test text, turn Phoenix's request logger off in prod and emit one JSON request line from the endpoint's stop telemetry event with method, path, status and duration. Put user_id into Logger metadata in fetch_current_user and in both on_mount clauses, and the region from Nav's on_mount. Move the interpolated values in the existing Logger.info calls into metadata keys without changing their messages. Install Postgres and the pinned Erlang and Elixir in your sandbox, set BUMBLEBEE_CACHE_DIR, run mix test, add tests that capture a request log and a LiveView log with the JSON backend and decode them, run mix format --check-formatted, and open a pull request with the results.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

A pull request that changes mix.exs and mix.lock (one dependency), config/prod.exs (the backend and the Phoenix logger switch), endpoint.ex or a new telemetry handler module (the request line), user_auth.ex and nav.ex (metadata), the four modules with interpolated logs, and adds a test file. The description names the logger_json release and the Elixir constraint behind it, quotes one decoded request line and one LiveView line, and shows the mix test and mix format results.

A reviewer checks that no metadata key carries a token or an email, that dev output is unchanged, and that the request line has the fields their log search needs, then merges. After the next deploy, Fly's log stream carries one JSON object per line, and a request_id or user_id finds the plug request and the LiveView events that followed it.

Run it on every service onboarded to central logging

A developer onboarding the app to a log search starts this from the terminal with this body and the repository in resources, and goes on to the dashboards. The run reads the real Elixir pin and the real mount hooks, so the library release compiles on the image the app ships from and the ids land on LiveView lines, which is where a text-format setup usually loses them.

Send a request_id, a UUID, with the body so a retried start returns the run it already began. When the turn ends, GET /o/{org}/sessions/{id} names the pull request, and a follow-up to the same session can ask for the Elixir image bump and the 7.x formatter, or for Ecto query logs in the same shape.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Structured logging in a LiveView app is mostly about which process a line comes from, and the only way to know the ids survived the socket is to mount a view under the JSON backend and read what came out. The agent does that in its own sandbox with a real Postgres and the pinned toolchain, and keeps every command and the decoded lines in the run's record, so the pull request shows a LiveView log with a user_id rather than promising one.

The same run fits any Phoenix application: a JSON API where the request line is the whole story, an umbrella where each app configures the backend once in the shared config, or an app already on Elixir 1.15 where the :default_handler formatter replaces the backend. Each gets the release its own constraints allow and a test that decodes its own output.

Frequently asked questions

Why not the newest logger_json?

The 6.x and 7.x releases require Elixir 1.15.1 or newer, and the Dockerfile builds on 1.14.0. The run adds the newest 5.x so the image keeps building, and the pull request says that bumping the image is the one change that admits the newer line.

Will dev and test output change?

No. The JSON backend is configured in config/prod.exs only; dev.exs keeps [$level] $message and test.exs keeps the warn level. The new tests switch the backend on for their own process and restore it afterwards.

How does user_id reach LiveView logs?

A LiveView runs in its own process, so metadata set in a plug does not follow it. The on_mount clauses in user_auth.ex run inside the LiveView process at mount, and that is where the run sets Logger.metadata(user_id: ...), so every later handle_event and handle_info log carries it.

Does mix test need the Whisper model?

application.ex starts an Nx.Serving that loads openai/whisper-tiny through Bumblebee at boot, so the first mix test downloads it into BUMBLEBEE_CACHE_DIR. The run sets that directory and reports the download in the transcript; it is unrelated to logging and unchanged by the pull request.

More Phoenix LiveView jobs

The same job on other stacks

Keep reading