Build an AI SEO Agent in Python: Tool Calling Over Search Console Data

Most "AI SEO agent" demos I find have the same structure. A large hosted model gets a big pile of analytics data, writes something confident, and the post stops there. I wanted to see what the smallest working version looks like: a 1.5B-parameter open model on a laptop CPU, four plain Python functions over Search Console-shaped data, and a loop that lets the model choose which functions to call.

This post builds that agent in four versions and keeps every run's output. The failures were more useful than the successes. By the end the design was clear to me: code should own the facts, and the model should own the choices. Each version below moves a little more work from the model to the code.

All the code ran on an 8-core CPU box with no GPU. Each model turn took between 3 and 16 seconds.

About the data. The numbers in this post come from a synthetic, seeded sample shaped like a Search Console export (make_sample.py, random.seed(7)). They are not from any real site. The model outputs and timings are real runs against that sample.

What "agent" means here

An agent here is a loop. The model reads a question and a list of tool signatures. It either replies with a tool call or with an answer. When it calls a tool, your code runs the function, adds the JSON result to the conversation, and asks the model again. That's all there is to it.

I used Qwen2.5-1.5B-Instruct from the Hub. Its chat template accepts a tools= argument and the model emits calls as <tool_call>{json}</tool_call>. With transformers you can pass ordinary Python functions with docstrings, and the template turns them into JSON schemas for you. That keeps the tool layer short.

The data and the four tools

Search Console's Performance report exports rows of query, page, clicks, impressions, ctr, position. My sample has 96 rows: 48 queries over six pages, each in a "previous" and a "current" 28-day period. In one topic cluster I pushed positions down a few spots in the current period, so there is an actual regression for the agent to find.

The tools are deliberately boring. Each answers one question a person would ask while looking at the report:

def striking_distance(min_impressions: int = 300, limit: int = 5) -> list:
    """Find current-period queries ranking between position 8 and 20 with enough impressions to be worth pushing.

    Args:
        min_impressions: minimum impressions in the current period
        limit: how many queries to return
    """
    min_impressions = min(min_impressions, 1000)  # added in v4, see below
    rs = [r for r in _curr() if 8 <= r["position"] <= 20 and r["impressions"] >= min_impressions]
    rs.sort(key=lambda r: -r["impressions"])
    return [{k: r[k] for k in ("query", "page", "impressions", "clicks", "position")} for r in rs[:limit]]

Write the docstring carefully, including the Args: block. The model never sees your code. All it gets is the schema built from that docstring, so a vague description turns into vague arguments.

To make this real, swap the CSV reader for the Search Analytics API (searchanalytics.query with dimensions=["query","page"]). One warning from experience: the API sorts by impressions and silently drops rows past rowLimit. If you ask for 1,000 rows you can "lose" most of a page's queries without any error. Request more rows than you think you need, and check whether the row count equals your limit.

v1: the plain loop

The first version is the textbook loop. There's a system prompt telling the model to answer only from tools, the tools are passed to apply_chat_template, and it generates greedily, runs any tool calls, and repeats until there are none.

ids = tok.apply_chat_template(msgs, tools=list(TOOLS.values()), add_generation_prompt=True,
                              return_tensors="pt", return_dict=True)["input_ids"]
text = tok.decode(model.generate(ids, max_new_tokens=300, do_sample=False)[0][ids.shape[1]:],
                  skip_special_tokens=True)
calls = re.findall(r"<tool_call>\s*(\{.*?\})\s*</tool_call>", text, re.S)

The question was: "Did my clicks drop this period? If so, which page lost the most, and what should I fix first?" The real output:

--- step 1 (3.3s)
CALL pages_losing_clicks({'limit': 1}) -> [{"page": "/tools/keyword-difficulty", "prev": 238, "curr": 38, "delta": -200}]
--- step 2 (8.5s)
FINAL: Your click-through rate (CTR) has dropped significantly compared to the previous period.
The page at /tools/keyword-difficulty had a decrease of 200 points in its click volume. To address
this issue, focus on optimizing the content for the keywords being targeted by your competitors. ...

It's quick and it reads well, but it's wrong in three ways. The model never called compare_periods, so it never established whether clicks dropped overall. It said CTR dropped, which no tool reported. The advice is a generic checklist that no data supports. The question had three parts, and the model made one call and answered all three anyway.

v2: force the overview and check the numbers

There were two changes. First, code calls compare_periods() before the model ever runs and adds the result to the conversation as if the model had asked for it. Any "did X change" question needs that call, so there's no reason to leave it to the model. Second, a number check runs before any answer is accepted. It pulls every number out of the draft, and any number that isn't in a tool result or the question gets the draft rejected with feedback.

bad = {n for n in numbers(text) - numbers(seen + question) if len(n.strip("-")) > 1}
if bad:
    msgs.append({"role": "user", "content": f"These numbers are not in any tool result: {sorted(bad)}. ..."})

The check worked, and that exposed a new problem. On the first question the model wrote "dropped by about 36%" and "losing approximately 39%". Neither number exists anywhere: the real drop in the sample is 887 → 528 clicks, or −40.5%. The check rejected the draft six times in a row. The model resubmitted the same text every time and never called another tool. The second question went the same way: six rejections, with invented "positions #39 and #40". So v2 produced zero answers out of two, though at least it produced zero wrong ones.

v3: a planning turn

Small models tend to treat "answer the question" as the only move they have. The fix that helped most was splitting the run into two turns:

1. A planning turn: "Before answering, call every other tool you need for this question. Reply with tool calls only."
2. An answering turn: "Now answer the original question."

I also changed compare_periods to return clicks_change_pct directly, so the model doesn't have to do arithmetic.

In the planning turn the model called three tools on each question, six calls in total, against zero in v2. Two problems showed up right away:

• On question 1 it called striking_distance(min_impressions=5000). In this sample the busiest single query has 2,287 impressions, so the call returned []. The model chooses arguments with no idea what the data looks like.
• The number check rejected correct numbers. pages_losing_clicks returns "delta": -200, the model wrote "200", and my regex treated -200 and 200 as different numbers. ctr_gaps returns "ctr": 0.063 and the model wrote "6.3%". That was my bug, not the model's.

Both questions ran out of steps, so v3 also produced zero answers out of two.

v4: fix the checker and clamp the arguments

These were two small code fixes. The number check compares absolute values and allows a 0.x ratio to be quoted as a percentage. striking_distance caps min_impressions at 1,000, which is a guard against the threshold-guessing problem. Here is the real v4 run on question 1:

FORCED compare_periods -> {"prev": {"clicks": 887, ...}, "curr": {"clicks": 528, ...}, "clicks_change_pct": -40.5}
CALL pages_losing_clicks({'limit': 3}) -> [{"page": "/tools/keyword-difficulty", "prev": 238, "curr": 38, "delta": -200},
                                            {"page": "/blog/internal-links", "prev": 192, "curr": 30, "delta": -162}, ...]
CALL striking_distance({'min_impressions': 5000, 'limit': 5}) -> [{"query": "keyword difficulty", ..., "position": 15.0}, ...]
CALL ctr_gaps({'limit': 3}) -> [{"query": "serp preview", "page": "/tools/serp-preview", "position": 3.0, "ctr": 0.063, ...}]
FINAL: ... Your overall click volume dropped by approximately 40.5%.
  - Page /Tools/Keyword-Difficulty saw a decrease of 200 points in its keyword difficulty score.
  - Page /Tools/Serp-Preview had a decline of 162 points in its SERP preview performance.

Both questions now finish. Every number passes the check, and the answer is still wrong. The 162-click loss belongs to /blog/internal-links, not /tools/serp-preview, which actually gained clicks (155 → 359). "200 points in its keyword difficulty score" mixes up a click delta with a page name. On question 2 the model said the queries "closest to page one" were the ones with high CTR, when the tool had just returned queries at positions 15–18.

A number check catches invented numbers. It can't catch a real number attached to the wrong page. To catch that you need the answer tied to rows, not just to values.

What I'd ship: an evidence block written by code

The last step doesn't involve the model at all. Code renders the facts as numbered evidence lines straight from the tool results:

def evidence_block():
    cp = tools.compare_periods()
    lines = [f"[E1] clicks {cp['prev']['clicks']} -> {cp['curr']['clicks']} ({cp['clicks_change_pct']}%), "
             f"avg position {cp['prev']['avg_position']} -> {cp['curr']['avg_position']}"]
    for i, p in enumerate(tools.pages_losing_clicks(3), 2):
        lines.append(f"[E{i}] {p['page']}: clicks {p['prev']} -> {p['curr']} ({p['delta']})")
    ...

[E1] clicks 887 -> 528 (-40.5%), avg position 18.6 -> 21.2
[E2] /tools/keyword-difficulty: clicks 238 -> 38 (-200)
[E3] /blog/internal-links: clicks 192 -> 30 (-162)
[E4] /blog/seo-checklist: clicks 113 -> 31 (-82)
[E5] 'keyword difficulty' pos 15.0, 2203 impr, 9 clicks
[E6] 'serp preview template' pos 17.1, 2156 impr, 11 clicks
[E7] 'seo tool pricing example' pos 18.3, 2119 impr, 2 clicks

Then the model gets a smaller job: say which items to fix first and why, citing [E#] IDs instead of restating numbers. Validating that is a set-membership check on IDs, which is a lot simpler than parsing prose. The person reading the answer sees the exact rows behind each recommendation. Of everything here, that's the part I'd keep in production. I haven't benchmarked the ID-citing prompt across many questions yet, so treat this design as the next experiment, not a measured result.

Commercial tools are converging on the same idea. Tracetify's ai seo agent shows every tool step the run took and has the answer cite the Search Console numbers it used. Its page also says that querying your own connected data costs no credits, while paid research lookups are priced before they run. If you build your own agent, aim for that same property: you can audit every claim back to a tool result.

Lessons, ranked by how much they helped

1. Put mandatory calls in code. If a question type always needs the overview, call it yourself. A 1.5B model skipped it in v1.
2. Split planning from answering. One instruction, "tool calls only," took the model from 0 to 6 calls in my runs.
3. Return the derived numbers you want quoted. Hand it the clicks_change_pct. Don't ask a small model to calculate it.
4. Guard arguments. The model picked min_impressions=5000 twice on data whose busiest query had 2,287 impressions. Clamp the value, or show the model the data range in the docstring.
5. Number grounding is necessary but not sufficient. v4 passed every number check and still assigned a loss to the wrong page. Ground the answer to rows.
6. Test your checker as hard as the model. Two of my v3 rejections were regex bugs.

A bigger model would make fewer of these mistakes. None of the fixes depend on model size, though, and with a small model you get to watch each failure happen, which is why I used one.

Run it yourself

The folder has make_sample.py (builds the sample CSV), tools.py, agent.py (v1), agent_v2.py through agent_v4.py, and evidence.py. You need torch, transformers and accelerate. The first run downloads about 3 GB of weights. Decoding is greedy, but a different machine or library version may still change the exact wording. To point it at your own site, swap the CSV loader in tools.py for an export or an API call and leave everything else alone.