# Write Integration Tests for a VS Code Extension

Yes. An agent adds @vscode/test-cli integration tests that boot a real VS Code, open your tree view and run its commands under xvfb, then opens a PR.

[Run this on Rush](https://cloud.getrush.ai/run/write-extension-tests/vscode-extension/), in a Project that has your repository.

## What the agent does

Extension tests run inside a real VS Code: `@vscode/test-electron` downloads a VS Code build, `@vscode/test-cli` starts it with your extension loaded, and Mocha tests run in its extension host with the full `vscode` API. The agent starts by reading what the extension declares. In microsoft/vscode-extension-samples' `tree-view-sample`, `package.json` names the extension `custom-view-samples` from publisher `vscode-samples`, requires `vscode` `^1.100.0`, compiles with `tsc -p ./` into `out/`, and has no test script and no test dependencies. It contributes a `package-explorer` activity bar container holding the `nodeDependencies` view, five explorer views, and commands such as `nodeDependencies.refreshEntry`, `addEntry`, `editEntry` and `deleteEntry`.

Two details in that manifest decide how the tests are written. `activationEvents` lists only `onLanguage:json` and `onLanguage:jsonc`, so the extension does not activate when VS Code starts; a test must open a JSON document or call `extension.activate()` first, or every command lookup fails. And `DepNodeProvider` in `src/nodeDependencies.ts` builds its tree from the first workspace folder's `package.json` and that folder's `node_modules`, showing "No dependency in empty workspace" or "Workspace has no package.json" otherwise, so the tests need a fixture workspace with known contents.

The agent copies the setup from the repository's own `helloworld-test-cli-sample`, which uses `@vscode/test-cli` `^0.0.8`, `@vscode/test-electron` `^2.3.9`, Mocha and a `.vscode-test.mjs` with `files: 'out/test/**/*.test.js'`. It adds those to `devDependencies`, a `test` script that runs `vscode-test`, and a `.vscode-test.mjs` whose `workspaceFolder` points at `src/test/fixtures/workspace`: a `package.json` with two dependencies and matching stub folders under `node_modules`. Tests go in `src/test/*.test.ts`, which the existing `tsconfig.json` (`rootDir: src`, `outDir: out`) compiles to `out/test`.

The tests activate the extension through `vscode.extensions.getExtension('vscode-samples.custom-view-samples')`, check every contributed command is registered with `vscode.commands.getCommands(true)`, open the view with the `workbench.view.extension.package-explorer` command, and call `DepNodeProvider.getChildren()` against the fixture to assert the tree lists the two dependencies with the versions in its `package.json`. They run `nodeDependencies.refreshEntry` and check the provider fires `onDidChangeTreeData`. The agent runs `npm run compile` and `xvfb-run -a npm test` in its sandbox until every test passes.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "codex",
  "input": "Pick the tree-view-sample extension. Add integration tests with @vscode/test-cli and @vscode/test-electron, set up the way helloworld-test-cli-sample in this repository does (devDependencies, a test script, .vscode-test.mjs). Give the tests a fixture workspace with a package.json and node_modules entries, activate the extension explicitly (it only activates on JSON files), check every contributed command is registered, open the Package Explorer view, assert the Node Dependencies tree lists the fixture's dependencies, and run the refresh command. Compile, run them under xvfb-run until they pass, and open a pull request.",
  "resources": [
    {
      "url": "https://github.com/your-org/your-app"
    }
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The pull request adds to `tree-view-sample/`: `.vscode-test.mjs`, `src/test/extension.test.ts` and `src/test/nodeDependencies.test.ts`, the fixture under `src/test/fixtures/workspace/`, the test dependencies and `@types/mocha` in `package.json` with the updated `package-lock.json`, a `test` script, and `.vscode-test/` in `.gitignore` so the downloaded VS Code is not committed. `extension.ts` and the providers do not change.

The description lists each test and what it proves, quotes the Mocha summary from the `xvfb-run -a npm test` run in the sandbox, and names the VS Code version the tests ran against. It also notes what is not covered: the FTP explorer needs a live FTP server, and drag and drop in `testViewDragAndDrop` has no programmatic API to drive. The session's record keeps every compile and test run.

Before merging, a reviewer checks the fixture is small and committed, that the tests do not depend on the order Mocha runs files in, and whether to add a CI job, which the pull request includes only if the repository already has a workflow for this sample.

## Run it on every command contributed

Each new command or view in `package.json` is a new thing that can silently fail to register. A GitHub trigger on `pull_request.opened` can start this run for pull requests that change an extension's `package.json` or `src/extension.ts`, asking for a test that the new contribution registers and works, so it is covered when it lands.

In a repository with dozens of extensions, send one request per extension directory that has no tests yet. Each run boots its own VS Code under its own virtual display in its own sandbox, so they proceed side by side and each opens a separate pull request.

## Why run it on Rush?

The integration test setup is the part extension authors give up on: a downloaded VS Code build, a headless display on Linux, a fixture workspace, and an activation event that quietly prevents the extension from loading in the test window. In a Rush sandbox the agent works through those failures on a disposable Linux machine with `xvfb-run`, and no extra VS Code window ever opens on your desktop.

What comes back is checkable: the pull request's tests, the Mocha output from the sandbox in the session's record, and the exact VS Code version they passed against. Because each extension gets its own sandbox, a samples repository can get tests across many extensions at once, each reviewed on its own.

## Frequently asked questions

### Why does the test have to activate the extension itself?

VS Code only loads an extension when one of its `activationEvents` fires. This sample activates on opening a JSON or JSONC file, so a fresh test window would never load it. The tests call `extension.activate()` first, which is what VS Code does when the event fires.

### Can the tests read what the tree view shows?

The `vscode` API does not expose the rendered tree, so the tests call the `TreeDataProvider` the view uses, here `DepNodeProvider.getChildren()`, with the same fixture workspace. That checks the data the view displays, and a separate test opens the view to prove it is registered.

### Which VS Code version do the tests run against?

By default `@vscode/test-cli` downloads the current stable build. Set `version` in `.vscode-test.mjs` to pin one, or add a second configuration with the oldest version your `engines` field allows, here 1.100.

### Does it need a display?

VS Code is an Electron app, so on Linux it needs one, which `xvfb-run` provides. On macOS and Windows the same `npm test` runs without it.