OSWorld hard-20: computer-cli's MCP server (0.1.12) against Cua's MCP server, 20 tasks x 3 trials.
Post: https://getrush.ai/blog/computer-cli-osworld/

Files, in the order they were written:
  RULE.txt         The selection and scoring rule, written at 09:46 UTC on 2026-10-20, before any agent ran.
                   Kept exactly as written (file names inside it say .md; they ship here as .txt).
  Dockerfile.txt   The one image change RULE.txt describes ("Dockerfile in this folder"), for both tools.
  walk.txt         The no-agent baseline walk over the seeded order, 09:47-09:54 UTC.
  tasks.txt        The 20 tasks, frozen at 09:54:44 UTC.
                   sha256: d1520f7f777797bfae64f0d5c47aea2baac996c2ba63f526847532c562894c5a
  results.txt      One line per run as each lane finished: task, tool, trial, OSWorld reward (ERR = unscored),
                   wall seconds, Claude Code turns, end state, cost.
  results-aborted-zsh.txt  The first launch, which failed at setup before any agent started (DEVIATIONS.txt, 1).
  runs.csv         One row per run with the metrics the scores use, including model calls (distinct model
                   responses in the run's transcript).
  score.py         Scores runs.csv: python3 score.py runs.csv > score.txt
                   and python3 score.py runs.csv --capped-as-zero > score-capped-as-zero.txt
  score.txt, score-capped-as-zero.txt  Its output.
  DEVIATIONS.txt   What did not go to plan.

tasks.txt columns: order in the seeded draw, cua-bench variant id (the task's index in cua-bench's
OSWorld test_all task list, passed to `cb run task ... --variant-id`), domain, OSWorld task id.

Image: ghcr.io/trycua/bench-osworld@sha256:e55cf3a314793f89be7efbffdc043aab29bf85990c6213a9ebbb204825504439