All jobs

Add Test Coverage to CI for a Lit Web Component

Yes. An agent turns on web-test-runner V8 coverage for your Lit component, writes the tests the report shows missing, sets a floor from a real run and opens a PR.

Run this on RushOpens the console with this prompt ready to send, in a Project that has your repository.

What the agent does

The agent reads how tests run before it measures them. In a project laid out like lit/lit-element-starter-ts, package.json depends on lit 3.2, @web/test-runner 0.15, @web/test-runner-playwright 0.9 and @open-wc/testing 4, with TypeScript 5.9 compiling src/**/*.ts to the repository root, so my-element.js and test/my-element_test.js are build output that .gitignore lists. web-test-runner.config.js points files at ./test/**/*_test.js, launches Chromium, Firefox and WebKit through Playwright unless a BROWSERS variable narrows the set, uses mocha's tdd interface with a 60 second timeout, and switches exportConditions on a MODE variable; npm test runs it twice, once as test:dev and once as test:prod. There is no coverage key anywhere, no lockfile, and no .github/ directory.

The component under test is small, which makes the coverage numbers honest rather than impressive. src/my-element.ts has a name property, a numeric count, a render that calls sayHello, a private _onClick that increments count and dispatches count-changed, and static styles. src/test/my-element_test.ts has five tests: definition, default render, render with a name, a click, and the computed padding. The agent enables coverage in the config with coverage: true and a coverageConfig that writes the report to a coverage/ directory, adds lcov and text-summary reporters alongside the default, and restricts include to the compiled my-element.js so test files and polyfills do not inflate the total. V8 coverage is native in Chromium only, so the coverage run sets BROWSERS=chromium, while the three-browser run keeps its job.

Then it runs the suite with coverage once to see the real numbers and reads what the report marks as missed. In this component the uncovered paths are the count-changed event detail, which no test listens for, sayHello called directly rather than through render, and a second click to prove the count keeps incrementing instead of resetting; the agent adds those cases to src/test/my-element_test.ts in the same suite and test style the file already uses. It sets coverageConfig.threshold for statements, branches, functions and lines to whole numbers just under the figures it measured after the new tests, so the floor fails a pull request that removes a test without failing the current suite.

Because the repository has no workflow, the agent adds .github/workflows/ci.yml with one job that installs with npm install (there is no lockfile to npm ci against), runs npx playwright install --with-deps chromium, builds with tsc, runs BROWSERS=chromium npm run test:dev -- --coverage, uploads coverage/ as an artifact and appends the text-summary lines to the job summary. If a workflow already exists, it adds the coverage step to that workflow's test job instead. It runs npm run lint, both test:dev and test:prod across the three browsers, and the coverage run, and keeps every output for the pull request.

Start it with one API call

POST https://api.phoenix.so/o/{org}/sessions with this body:

{
  "agent": "claude",
  "input": "Add test coverage to CI for this Lit web component project and write the tests the report shows missing. Read package.json, tsconfig.json, web-test-runner.config.js, .gitignore, .eslintrc.json, src/my-element.ts and src/test/my-element_test.ts first, check whether a lockfile and a .github directory exist, and keep the existing MODE and BROWSERS behaviour. Enable web-test-runner coverage with a coverageConfig that reports to coverage/, adds lcov and text-summary reporters and includes only the compiled component file. Run the suite with coverage in Chromium, read the uncovered lines, add tests for them in the existing suite style, then set threshold values just under the measured numbers. Add a GitHub Actions workflow, or a step in the existing one, that installs the Playwright Chromium browser, builds, runs the coverage run, uploads the report and writes the summary to the job summary. Run lint, the full three-browser test:dev and test:prod runs and the coverage run, and open a pull request with the coverage summary before and after your tests.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
  1. Get an API key. In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
  2. Send the request. POST the body in request.json to https://api.phoenix.so/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
  3. Read what came back. Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

What comes back

The pull request changes web-test-runner.config.js, adds tests to src/test/my-element_test.ts, and adds .github/workflows/ci.yml. Its description shows the text-summary block from the first coverage run and from the final one, the threshold values it chose and why they sit where they do, and the output of lint and of the three-browser runs in both modes.

Once merged, every pull request gets a coverage artifact and a summary line, and a drop under the floor fails the run with web-test-runner's own threshold message naming the metric. The floor is a number the team can raise in one line as the component grows; it is not a target the agent invented.

Before merging, a reviewer checks that the new tests describe behaviour the component is meant to keep, in particular the count-changed event, picks whether Firefox and WebKit should run in CI on every push or only on main, and decides whether to commit a lockfile so the workflow can move to npm ci.

Run it on every nightly

In the console, a scheduled automation in the Project that has the component repository starts one run each night with this prompt, using a schedule trigger with every set to 1d in the team's timezone. Each run reads the latest coverage report, writes tests for whatever the component gained since the last run, and raises the threshold only when the new tests pass, so the floor follows the suite instead of blocking it. An automation run has its own max_runs_per_hour cap.

A team with many component packages built from the same starter can instead send this body once per repository to POST /o/{org}/sessions from a script, each with its own repository in resources and a request_id that is a UUID v5 of the repository name, so a retry returns the run that already started; the key's webhook_url then receives one run.completed per repository.

The use case this job follows, with its automation and trigger: Write the missing tests.

Why run it on Rush?

Coverage on a browser-tested component has details a template skips: V8 coverage comes from Chromium only, the measured files are compiled output rather than the TypeScript sources, and a threshold copied from somewhere else either fails on day one or never fails at all. The agent installs the Playwright browsers in its own sandbox, measures the real numbers, writes the tests that move them, and sets the floor from what it measured.

The run keeps every command and its output, so the pull request can show the summary before and after the new tests rather than a claim about them. Several component repositories can be handled at once, each in its own sandbox with its own browsers.

Frequently asked questions

Why collect coverage from Chromium only?

web-test-runner reads V8's native coverage, which Chromium exposes and Firefox and WebKit do not. Covering those browsers needs Babel instrumentation and is slower, so the agent keeps the three-browser run for correctness and the Chromium run for the numbers, and says so in the pull request.

Will the threshold block my next pull request?

Only if it lowers coverage below the floor, which the agent sets just under the number it measured with the new tests in place. A pull request that adds code with tests passes; one that deletes a test fails with the metric named.

Does it measure the TypeScript files or the compiled JavaScript?

The compiled files, because that is what the browser runs and what the config's files glob points at. Source maps are on in tsconfig.json, so the lcov report and any viewer map lines back to src/my-element.ts.

What if I already have a CI workflow?

Then the agent adds the coverage step, the artifact upload and the summary to the existing test job rather than creating a second workflow, and keeps the install and build steps that job already has.

More Lit jobs

The same job on other stacks

Keep reading