All jobs

Add Connection Timeouts to a Tokio Server

Yes. An agent adds an idle-connection timeout to a Tokio TCP server behind a CLI flag, keeps pub/sub streams alive, tests it with cargo and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads the accept loop and the per-connection handler before it adds a timer anywhere. In a server shaped like tokio-rs/mini-redis, src/server.rs sets MAX_CONNECTIONS to 250 and hands each accepted socket a permit from a Semaphore of that size; Listener::run acquires the permit, calls accept (which backs off from one second and gives up past sixty-four), builds a Handler with a Connection, a Shutdown receiver and a clone of the shutdown_complete_tx sender, and spawns handler.run(), dropping the permit when it returns. Handler::run loops while the shutdown flag is clear and does a tokio::select! between self.connection.read_frame() and self.shutdown.recv(), then parses the frame with Command::from_frame and calls cmd.apply. Nothing in that loop ever gives up on a silent client, so a peer that connects and sends nothing holds one of the 250 permits until the process exits.

src/connection.rs wraps the TcpStream in a BufWriter with a 4 KiB BytesMut read buffer; read_frame loops on read_buf and returns Ok(None) on a clean close or the error connection reset by peer on a half frame. src/bin/server.rs parses one clap flag, --port, defaults it to DEFAULT_PORT (6379 in lib.rs), binds 127.0.0.1 and calls server::run(listener, signal::ctrl_c()). Cargo.toml is mini-redis 0.4.1 on tokio 1 with full, clap 4.2.7 with derive, tracing and tracing-subscriber, and a test-util feature of Tokio in dev-dependencies. The CI workflow builds and tests on Ubuntu, macOS and Windows, once plain and once with the otel feature, and runs cargo fmt --all --check.

The change is contained. The agent adds an idle_timeout: Option<Duration> to the Listener and Handler structs and a public server::run_with that takes it (the existing run calls it with None, so the public API and every current test stay as they are). In Handler::run the read_frame arm of the select! becomes tokio::time::timeout(idle, self.connection.read_frame()) when a timeout is set; on Elapsed the handler logs at info with the peer address through tracing, returns Ok(()), and the socket, the Connection buffer and the semaphore permit are released as they would be on a normal close. src/cmd/subscribe.rs runs its own select! over the subscription StreamMap, dst.read_frame() and shutdown.recv(), and a subscriber is supposed to sit silent for as long as it likes, so that loop is left untimed and the pull request says why. src/bin/server.rs gains --idle-timeout-secs, parsed into the Option<Duration>; with the flag absent, nothing changes.

Tests go in tests/server.rs next to key_value_get_set, pub_sub and send_error_unknown_command, which already start a server on 127.0.0.1:0 through a start_server helper. The agent adds a second helper that starts one with a short idle timeout in milliseconds, then a test that opens a raw TcpStream, writes nothing, waits past the timeout and asserts read returns zero bytes, and a second test that subscribes to a channel, waits the same time, publishes from another connection and still receives the message. These use real short durations rather than the paused clock, because the timeout races a socket read. The agent runs cargo test, cargo test --features otel, cargo fmt --all --check and cargo clippy in its sandbox, and starts mini-redis-server --idle-timeout-secs 2 to show with nc that an idle connection is closed while mini-redis-cli ping still answers.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add an idle-connection timeout to this Tokio server. Read Cargo.toml, src/server.rs, src/connection.rs, src/shutdown.rs, src/cmd/mod.rs, src/cmd/subscribe.rs, src/bin/server.rs, tests/server.rs and .github/workflows/ci.yml first. Add an Option<Duration> idle timeout to the listener and handler, a server::run_with that takes it while run keeps its signature, and wrap the read_frame arm of the handler's select in tokio::time::timeout when it is set; on expiry log the peer at info and return so the socket and the semaphore permit are released. Leave the subscribe command's loop untimed and explain why in a comment. Add an --idle-timeout-secs flag to src/bin/server.rs that defaults to no timeout. Add tests in tests/server.rs: an idle raw TcpStream is closed after the timeout, and a silent subscriber still receives a published message after it. Run cargo test with and without the otel feature, cargo fmt --check and cargo clippy, then open a pull request with the output.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request changes src/server.rs, src/bin/server.rs and tests/server.rs, and adds a line to README.md describing the flag. Its description quotes the cargo test summary with and without --features otel, the cargo fmt and cargo clippy results, and an nc session against a server started with --idle-timeout-secs 2 showing the connection closed while a mini-redis-cli ping sent on another connection answers PONG.

The CI workflow runs the same build and test matrix on Ubuntu, macOS and Windows on the pull request, so the reviewer sees the new tests pass on all three. The run's record keeps the full command history, including the tracing line the server prints when it closes an idle peer, so a reviewer can match the log to the test.

Before merging, a reviewer chooses the production value for the flag and decides whether the timeout should also cover a slow writer by wrapping write_frame, which the pull request names as a follow-up rather than doing unasked. Nothing in the protocol, the Db or the client changes.

Run it on every Tokio service that accepts raw TCP

A team with several Tokio services starts one run per repository from its own server: a POST /o/{org}/sessions each, with the repository in resources and a request_id that is a UUID v5 of the repository's full name, so a retried call returns the run already started. Each run reads its own accept loop and handler, so a service that already uses tokio::time::timeout around its reads gets a report and no pull request, and one built on tower gets the timeout proposed as a tower::timeout layer instead of a hand-written select!.

To keep the guard in place, an automation in the Project with a github trigger on pull_request.opened and pull_request.synchronize and the prompt "if this pull request adds a loop that awaits a socket read, check it is bounded by the idle timeout or say why it must not be, and leave a comment" reviews each change. The trigger adds the pull request after the prompt and fires only for work by owners, members and collaborators.

The use case this job follows, with its automation and trigger: Hand off a long task.

Why run it on Rush?

Timeouts in an async server are a question of where, not whether: wrapped around the whole select! they fire during a graceful shutdown, placed inside the subscribe loop they disconnect every quiet subscriber, and set as a global default they change behaviour for anyone who upgrades. The agent reads the actual loops, puts the timer on the one read that should be bounded, leaves the one that should not, and proves both with a test that opens real sockets.

Every command stays with the run, from cargo clippy to the nc session, and the pull request quotes them. Each run has its own machine with its own Rust toolchain, so a dozen services get the same treatment at once, each with its own pull request and its own test output.

Frequently asked questions

Why is the subscribe loop exempt?

Because a subscriber's job is to wait. src/cmd/subscribe.rs runs its own select! over the channel stream, incoming frames and shutdown, and a client that has subscribed and sent nothing for an hour is behaving correctly. The idle timeout applies to the command loop in Handler::run, where silence means a client that connected and never spoke. The test that publishes to a silent subscriber after the timeout is there to prove the exemption holds.

Does this change behaviour for anyone who upgrades?

No. server::run keeps its signature and passes no timeout, the new flag defaults to none, and every existing test in tests/server.rs runs unchanged. Only a caller who passes --idle-timeout-secs or uses run_with gets the new behaviour.

Why not use Tokio's paused clock in the tests?

tokio::time::pause auto-advances the clock only when every task is idle, and a test that waits on a real socket read is not idle in that sense, so the result depends on scheduling. A real timeout of a few hundred milliseconds is deterministic enough and keeps the two new tests under a second.

Does the timeout free the connection limit?

Yes. The permit from the Semaphore is dropped when handler.run() returns, which is exactly what happens on expiry, so an idle peer no longer holds one of the 250 slots. The pull request points at the drop(permit) in Listener::run so a reviewer can confirm it.

More Tokio jobs

The same job on other stacks

Keep reading