# Run a Keyword Gap Analysis With Semrush

Yes. An agent reads organic keywords for your domain and two competitors in Semrush and returns gap.csv and gap.md, ranked by volume and difficulty.

[Run this on Rush](https://cloud.getrush.ai/run/keyword-gap-analysis/semrush/).

## What the agent does

The run reads Semrush through Semrush's remote MCP server, `https://mcp.semrush.com/v2/mcp`, the Semrush entry in the Rush connector catalog. It is read only and spends the Semrush API units of the account that signs in. Connect it once with `POST /me/connections {"type": "mcp", "name": "semrush", "url": "https://mcp.semrush.com/v2/mcp", "auth": {"mode": "oauth"}}`, open the `authorize_url` while signed in to the console, and name `semrush` on each run. No repository is involved, so the request has no `resources`.

The agent starts with your domain. It asks Semrush for the domain's organic search keywords in the US database (Semrush's Domain Organic Search Keywords report, `domain_organic`), with each keyword's position, search volume, keyword difficulty and the URL that ranks. It repeats the same read for each competitor named in the prompt. Because every line a report returns costs API units, the agent asks for the top-ranking rows first rather than a domain's whole keyword list, and the prompt can cap the number of rows per domain.

With three keyword lists in the sandbox, it does the comparison itself in a short script: a keyword is a gap when at least one competitor holds a position from 1 to 10 and your domain is absent from the results or ranks below 100. Keywords where you rank 11 to 30 and a competitor is in the top 10 go to a separate list, because those are pages to improve rather than pages to write. Branded keywords that contain a competitor's name are dropped, since you cannot win them.

Last, it ranks the gaps. Each keyword gets a score from its volume and difficulty, then the agent groups near-duplicates ("invoice template", "free invoice template") under one topic so the top 20 in `gap.md` are 20 different pieces of content, not 20 spellings of one. For each it writes the competitor URL that ranks today and one line on why it is worth targeting.

## Start it with one API call

`POST https://api.prix.dev/o/{org}/sessions`

```json
{
  "agent": "claude",
  "input": "Use the Semrush connection to read the organic keywords of example.com and of its two competitors competitor-one.com and competitor-two.com in the US database, top 2,000 rows per domain by traffic. Write gap.csv with every keyword where a competitor ranks in the top 10 and example.com does not rank in the top 100, with search volume, keyword difficulty, each competitor's position and ranking URL. Drop keywords containing a competitor's brand. Write gap.md naming the 20 to target first, grouping near-duplicate keywords into one topic, with why each one is worth it, and a second list of keywords where example.com ranks 11 to 30 and a competitor is in the top 10. Read only.",
  "connections": [
    "semrush"
  ]
}
```

1. **Get an API key.** In the console, open Settings, then API keys, and create a key for your org. Send it in the Authorization header as a Bearer token.
2. **Send the request.** POST the body in request.json to https://api.prix.dev/o/{org}/sessions, with your repository as the resources URL. The run starts in its own cloud sandbox.
3. **Read what came back.** Follow the run on its live stream or wait for the run.completed webhook, then read the result and its trajectory.

## What comes back

The session carries two artifacts. `gap.csv` has one row per gap keyword with its volume, difficulty, each competitor's position and the competitor URL that ranks, sorted by the agent's score, so it opens straight into a spreadsheet filter. `gap.md` is the plan: the 20 topics to write first, each with its main keyword, the near-duplicates folded under it, the page to beat and a sentence on why, then the shorter list of pages you already have that sit on page two or three.

Before acting on it, a marketer checks three things. The database: a domain that sells mostly outside the US needs the prompt changed to its own market. The score: the agent states the formula it used at the top of `gap.md`, so a team that weighs difficulty more heavily can ask for a re-rank in a follow-up. And the brand filter: a competitor whose name is also a common word may have had real keywords dropped, and the transcript lists every filtered row.

## Run it on every first of the month

Rankings move every month, and so does the gap. A Project automation on a monthly schedule can send the same request on the first of the month, or a cron job on your server can post it with an `Idempotency-Key` built from the month so a retry never starts a second analysis.

Keep each month's `gap.csv` and the next run can be told to compare: send last month's file as a `document` block in `input` and ask for keywords that closed, opened or moved. The `run.completed` webhook tells your server when the new files are ready to read from the session.

## Why run it on Rush?

A keyword gap done by hand means three exports, a VLOOKUP and an afternoon of deciding which near-duplicates are the same topic. The agent does the reads, the join and the grouping in its own sandbox, and the script it wrote for the comparison is in the transcript, so anyone can see exactly which rule put a keyword on the list.

Each run is a separate session with its own files, so an agency can run one per client domain at the same time, up to the key's concurrent run limit, without the exports getting mixed. The session's transcript records every Semrush call the agent made, which also shows what the analysis spent in API units.

## Frequently asked questions

### How many Semrush API units does this use?

It depends on how many rows the agent reads per domain, and Semrush charges units per line returned. The prompt caps the rows per domain, which keeps the spend predictable. The transcript lists every report request, so you can see what was read.

### Can I compare against more than two competitors?

Yes. Name them in the prompt; gap.csv gets one position column per competitor. Each extra domain is another keyword read, so raise the competitor count with the row cap in mind.

### Does it use a database other than the US?

Semrush keeps a separate database per country. Change the database in the prompt, for example to the UK or Germany, and the agent reads every domain from that one. Comparing domains across different databases would be meaningless, so keep one per run.

### Will it write the content for the gaps?

Not in this run; it writes the plan. A follow-up message to the same session can ask for an outline or a draft for any topic in gap.md, and the agent still has the keyword data in its sandbox.