An agent that reads my own posts and verifies the numbers aren't made up

Google released version 2.14.0 of the Agent Development Kit on November 4, 2026, its Python SDK for building agents. The big shift in this release is that the execution sandboxes graduate out of experimental: an agent can now run Python code in a LocalEnvironment (your machine), in an E2BEnvironment (a remote micro-VM), in a DaytonaEnvironment (an ephemeral container), or in an AgentEngineEnvironment (Google Cloud’s managed runtime). Until now each of those four backends lived behind a subject to change in the docs.

An agent that can execute code is an agent that can, at last, audit something I wrote.

This post is the shortest proof of concept possible: a thirty-line agent that reads a blog .mdx, opens the raw results.json of the same experiment, and verifies that every figure in the text corresponds to an entry in the JSON.

Why this agent, and not another

Rule number one of this blog is: no number without having measured it. Every figure in the text must map to file:key of the raw data. I say it in the recipe and I repeat it in the build gates.

Until today, that rule was kept by eye. I wrote the number myself, I checked results.json had it myself, I signed at the bottom myself. The only wall between an honest 56.0 % and an invented 56.0 % was memory.

With the sandboxes out of experimental, it is trivial to put an agent in front doing the same thing I would do: look at the JSON.

The agent takes the path to the post and the path to results.json. It has three tools:

• read_post() returns the whole MDX.
• read_results_keys() returns the shape of the JSON, two levels deep, so it can plan its lookups without dragging down the whole tree.
• python_lookup(code) runs a snippet of Python in the local sandbox, with the variable d already loaded with the JSON. Whatever the code assigns to result comes back to the agent as its answer.

The agent’s prompt tells it to scan the post, pull every number with its unit and context, and for each one write a one-liner of the form result = d["agg"]["cola2_acc"]. Every time the sandbox returns a value, the agent compares, tolerates unit conversions (14.5 GB = 14,518 MB), and marks the figure as verified, not found, or context.

The run

I tried it against the post I published yesterday, Cola2 on 16 GB. Sixteen thousand characters of measured text and a results.json with sixty-five items plus its aggregate.

The agent ran in 76.8 seconds, made 11 tool calls, and emitted this list:

46 figures detected. All the ones the model considered numeric claims: it does not count years, software versions, or identifiers.

9 verified against results.json. The keys are the obvious ones: 56.0 (Cola2 accuracy), 22.0 (baseline accuracy), 28, 11, 50 (N of samples), 10.54 (mean latency), 72.86 (tokens per second), 14518 (VRAM MB, also quoted as 14.5 GB), and the margin 34 the agent worked out as 56 − 22.

1 figure not found. The load time 26.6 s appears in the text but the load_s field in results.json was left as null. It is an oversight of mine when recording the measurement, and the agent finds it without me asking.

36 figures classified as context. Release dates (November 4, October 30), model dimensions (4 billion parameters, 34 layers, 2,560 hidden, 346 billion training tokens), configurations (sample_seed=42, max_prompt_length=512, MAX_NEW_TOKENS=32), paper citations (60.10 average), derived calculations (3 tokens per second = 32/10.54), hypothetical estimates (“should be at 30-40 tokens per second”). None of those should map to the JSON, and the agent knows it.

Why a regex is not enough

I rewrote the agent as a fourteen-line regex: extract every number with an optional unit, flatten the JSON to {path: value}, cross-reference.

The regex detects 69 figures and claims it verifies 45. That sounds better than the agent, until you look at the matches: it “verified” the number 4 against n_error: 4, 2 against four different paths, 50 against the aggregate’s n. The regex does not distinguish between “the latency was 10.54 seconds” (measured) and “max_prompt_length=512” (configuration); both are numbers, and both appear in the JSON by structural coincidence, not because the figure in the text speaks to the field in the JSON.

The agent does one thing the regex cannot: it reads the context of the sentence. “The mean latency was 10.54 seconds” clearly points at agg.mean_latency_s; “I ran with --max-samples 100” talks about no measured field, it is a hyperparameter.

That is exactly the kind of distinction that would turn an eyeball audit into a build gate. The inflated regex gives false confidence; the agent, even if more expensive, gives actionable information.

What it costs

A single run against a 16 KB post:

• 76.8 s of wall time.
• 11 tool calls (1 read_post, 1 read_results_keys, 9 python_lookup).
• One model in/out I don’t measure in tokens because LiteLLM doesn’t expose them back to the agent in the SDK I tried.
• Zero GPU resources. It all runs on the Mac.
• The local sandbox is a subprocess.run with a timeout; for production, it would be worth switching to E2BEnvironment for isolation.

Compared to a build gate that runs in a second, it is an order of magnitude slow. But it is the only honest way to check that what I publish matches what I measured.

What’s missing

Give the agent a way to tell when null is an oversight and when it is by design. On my run it flagged load_s as “not found”. That is correct, it is an oversight, but it could not tell me so: it would have needed to read the measurement script to see that the field does get recorded there and I just didn’t capture it.

A second judge. An agent that audits itself leaves out the author’s bias but not the school of thought’s. The next useful step would be running the same verifier with Gemini 2.5 Flash in parallel, comparing the two lists of figures, and publishing only those both mark the same way.

Cost in tokens. The SDK doesn’t expose them per run; to use it as a gate you have to count outside.

Frequently asked questions

What is Google ADK?

Agent Development Kit. A Python SDK Google published for building agents with tools, memory, execution sandboxes, and evaluators. Version 2.14.0 shipped on November 4, 2026 and graduated `BaseEnvironment`, `LocalEnvironment`, `E2BEnvironment` and `DaytonaEnvironment`, the `EnvironmentToolset`, the multi-run evaluation runtime, and A2A out of experimental.

What does the VerificadorCifras agent do?

It reads an `.mdx` file from this blog and the `results.json` of its own experiment, extracts every number that appears in the text (percentages, latencies, VRAM, hits), and for each one writes a Python `one-liner` that looks that value up inside the JSON. It runs it in the local sandbox, looks at the result, and classifies each figure as *verified*, *not found*, or *context*. At the end it emits a verdict with the three counters.

What is it for?

To audit what I publish by myself. The blog's rule is that every figure in the text must map to `file:key` of the raw data. Until now that rule was kept by eye: the author writes it, the reviewer trusts it. With the agent in front, the policing happens before publishing, and 'I forgot to record `load_s`' shows up as a flaggable finding instead of a silent oversight.

How much did it detect and verify?

On the Cola2 post I published yesterday, it detected 46 figures, verified 9 against `results.json`, flagged 1 as not found (the load time I left unrecorded in the JSON), and classified 36 as context or configuration (dates, model dimensions, paper hyperparameters). It spent 11 tool calls and took 76.8 seconds running on Claude Sonnet 4.5 through LiteLLM.

Wouldn't a trivial regex do the same?

No. A regex that pulls every decimal number and looks it up in the JSON found 69 'figures' in the same post (many are `2`, `4`, `8`, `32`: secondary parameters that aren't claims) and marked 45 as 'verified' because the number `50` or the `4` appear in the JSON by sheer coincidence. Without the model's judgment on what is a *claim* versus a *parameter*, the regex lies with a lot of confidence.

What model is behind it?

`anthropic/claude-sonnet-4-5` served by the Anthropic API, plugged into ADK with the `LiteLlm` adapter. It was the cheapest one I had on hand for a test with few calls. ADK accepts Gemini, OpenAI, OpenRouter, Z.ai, MiniMax and anything LiteLLM exposes.

Sources

• SDK release: google/adk-python v2.14.0 (November 4, 2026).
• Sandbox docs: BaseEnvironment and derivatives.
• Adapter for non-Gemini models: LiteLLM.
• Target post: Cola2 on 16 GB.
• Raw agent data: lab/google-adk-verificador/ in the blog’s repository.

November 7, 2026 · Mac M2 · google-adk 2.14.0 · litellm with anthropic/claude-sonnet-4-5 · LocalEnvironment (subprocess) · 16 kB Cola2 post, 65-item results.json · single run, SDK default temperature.