[dgesteves]

The only benchmark the site and the docs report is bench/v2's 1,200 runs: 6 models, 3 providers, design-system errors per run from 0.85 to 0.00 as onsystem 0.5.0's own check counts them, no measurable change in tsc pass rate or cost, and @shadcn/lint findings per run from 4.97 to 2.64. The earlier, smaller benchmark in bench/agents was still rendered on https://onsystem.vercel.app/benchmark under "The earlier benchmark", with its clean-task counts, cost changes and run count, and its README still had the results tables. This removes them.

Files changed

• site/app/benchmark/page.tsx: removes the "The earlier benchmark" section (#v1) and its entry in "On this page".
• site/components/benchmark.tsx: deleted. It drew the earlier benchmark's model cards and per-task table, and nothing else used it.
• site/scripts/generate.mjs: no longer reads bench/agents/results or writes generated/bench.json, and no longer extracts the README's "What changes from v1" section (method.v1).
• site/lib/data.ts: removes the bench export, its types (Bench, BenchModel, RunSummary) and method.v1.
• bench/agents/README.md: the results tables, the per-task misses and the reading of them are gone. It now says it is an earlier, smaller benchmark replaced by bench/v2, and lists the files that stay and why.
• bench/v2/README.md: "What changes from v1" (the run count and the comparison table) is replaced by one sentence with no numbers. The two v1 mentions in sections the site renders (the none arm and Threats to validity) are dropped, and "v1's 40 transcripts" becomes "v1's transcripts".
• test/bench-v2-claims.test.ts: the results paragraph now ends at ## Method instead of the removed heading.
• test/readme.test.ts: drops the check on the removed table's Scoring column. The check on Threats to validity, which lists the same scorers, stays.
• site/test/benchmark.test.ts: new test that /benchmark has no #v1 section, no link to bench/agents, and no "earlier benchmark", "N / 10", "-11%", "40 runs" or ai-chatbot text.
• site/test/first-screen.test.ts: a comment said "1,240 runs" (both benchmarks). It now says 1,200.

generated/bench.json is gitignored. benchmark.json and ask-index.json were regenerated with pnpm --filter site generate, not edited by hand.

Kept on purpose

• bench/agents/run.mjs, tasks.json, results/ and output/. bench/v2 builds on them: test/bench-v2.test.ts checks that v2's prompt suffix matches run.mjs and that its chatbot prompts match tasks.json, and bench/v2/config/usage-model.json is measured from these runs. Nothing renders or quotes their numbers now.
• The cost-estimate paragraph in bench/v2/README.md (not rendered on the site) still says the estimator's token profiles come from v1's runs, and that the formula gives $0.17 per Haiku 4.5 run where v1 measured $0.13. That is the estimator's calibration, not a result about onsystem.
• docs/how-it-works.md and the icon-button-accessible-name rule doc mention vercel/ai-chatbot as a codebase check was run on. That is not the benchmark.

Overlap with open pull requests

• test/readme.test.ts is also changed by docs(readme): add a compatibility section, and say installed apps read a dist-only package's type declarations #203. This PR only touches the end of the bench/v2 Scoring test (around line 650), and docs(readme): add a compatibility section, and say installed apps read a dist-only package's type declarations #203 changes lines 73 to 91, so either can merge first. If docs(readme): add a compatibility section, and say installed apps read a dist-only package's type declarations #203 merges first, this one rebases cleanly.
• No other file here is in fix: per-user temp cache, quoted repository text, and linear color rule #199, feat(site): shareable playground links, no layout shift from the font swaps, a 44 px Ask target #200, feat(config): an app's config builds on its design-system package's, and so does the root's #202, docs: support tiers for every client setup, and a four-week integration freeze #204 or chore: release #229.

Checks run locally

• pnpm lint, pnpm format:check, pnpm typecheck, pnpm build, pnpm test (92 files passed, 1 skipped).
• pnpm --filter site generate, pnpm --filter site test (273 passed), pnpm --filter site typecheck, pnpm --filter site lint, pnpm --filter site build.

Grep of the built site

After pnpm --filter site build:

$ grep -c -i -E 'earlier benchmark|[0-9] ?/ ?10\b|[−-]11 ?%|\b40 runs|40-run|11% cheaper|4% false|ai-chatbot|bench/agents|What changes from v1|id="v1"' \
    site/.next/server/app/benchmark.html site/.next/server/app/benchmark.rsc site/.next/server/app/llms.txt.body
site/.next/server/app/benchmark.html:0
site/.next/server/app/benchmark.rsc:0
site/.next/server/app/llms.txt.body:0

$ grep -r -l -i -E 'earlier benchmark|What changes from v1|11% cheaper|4% false|bench/agents|clean components' site/.next/server
(no output)

$ grep -r -l -E '10 / 10|6 / 10|−11%' site/.next/server/app
(no output)

llms-full.txt has 3 matches for ai-chatbot, all from the how-it-works table and the rule doc above, none from the benchmark.

For comparison, the live /benchmark today has "The earlier benchmark", "10 / 10", "6 / 10" and "−11%".

Repository grep

$ git grep -n -i -E "earlier benchmark|10 ?/ ?10|\b6 ?/ ?10|6-8/10|[−-]11 ?%|11% cheaper|\b40 runs|40-run|4% false|What changes from v1" \
    -- ':!pnpm-lock.yaml' ':!bench/agents/output' ':!bench/agents/results' ':!bench/v2/results' ':!corpus'
site/test/benchmark.test.ts:561:      /earlier benchmark|\d ?\/ ?10\b|[−-]11 ?%|\b40 runs|ai-chatbot/i,

The only match is the new test's pattern.