[punitarani]

Requested by Punit · project thread

Opened by the nightly ModelBeats sync routine. Draft: Punit reviews and merges.

Before: the catalog was as of October 26, 2026. It was missing Google's Gemini 4 Carbon and Particle AI's Particle-1, both announced October 28.

After: the catalog is as of October 31, 2026, with 2 new models. Gemini 4 Carbon enters at #1 (3233.2), ahead of Kimi K3.1 Pro (3164.1). Particle-1 is unranked: it has only two scored benchmarks, below the coverage gate. Mistral Large 4 now reflects its October 29 open-weights release: license, released status, and the lower API price.

Window covered

2026-10-26 to 2026-10-31. GPT-6.1 Terra (Oct 26) already merged in #23. The Oct 30 run added the Mistral Large 4 update below, and the Oct 31 run swapped in Vals' run of the open-weights checkpoint. No other new LLM shipped on Oct 30 or 31.

New models

Model	Released	Open/closed	Price in/out ($/M)	Rows filed	Rank
Gemini 4 Carbon (Google)	2026-10-28	closed, gated rollout	none yet (see below)	Vals TB 4.0 63.13 (independent), AutomationBench 53.9, FrontierSWE 2 61.2, Chartography 72.5, SEC-bench Pro 88.2	#1 (3233.2)
Particle-1 (Particle AI)	2026-10-28	closed (API only)	0.25 / 1.00	GPQA 90.40, MMLU 86.08	unranked (1925.1)

Gemini 4 Carbon. Sources: Google's launch post, DeepMind's results table and methodology, plus Vals AI's model page and Terminal-Bench 4.0 board.

• Gated launch. Carbon is rolling out to selected users only, with no self-serve API. This follows the Argon and Gemini 3.8 Flash Cyber precedent: status released, apiAvailable: false, price null. The note records Google's $2/$10 introductory price and $4/$20 list price, so the price can be filed once self-serve opens.
• Terminal-Bench 4.0. Vals ran it on launch day (63.13, feat(data): add EQ-Bench 4 and Creative Writing v3 benchmarks (eqbench.com) #4 of 57) and credits no fallback work, so the independent row is filed.

Benchmarks deliberately not filed:

• Artificial Analysis numbers Google quotes. These are Terminal-Bench 4.0 65.2 and Terminal-Bench-Science 64.8. AA runs its own harness, while these corpus columns use the Vals boards. Vals has no Terminal-Bench-Science run for Carbon yet.
• Newer versions than the corpus columns. Agents' Last Exam v1.1 (77.4) and OSWorld-2.1 offline (81.5).
• Outside the column's bounds. CWE-bench 71.0, against normMax 60. Argon's 68.0 was left out for the same reason.
• Not comparable with the corpus columns. Vals Finance Agent v2.1 and HLAB-Vals-1.1.
• No corpus column. SWE-sweep, GraphWalks, RiemannBench, Vending-Bench 2, Blueprint-Bench 2, ExploitGym, and the Vals and AA indexes.

Particle-1. Sources: Particle's launch post and model docs.

• What it is. An RL post-train of DeepSeek V4.1 Flash (a rank-64 LoRA, trained with GSPO) that targets token efficiency.
• Specs. Context is 1M and input is text plus images. The price is the docs' $0.25/$1.00. The launch post's cost footnote assumes $0.30/$1.20 instead; the docs win.
• Not filed. The per-benchmark coding pass rates sit behind an unlabelled chart toggle. "Tau Airlines" doesn't say which tau-bench version it is.

Mistral Large 4 open weights (Oct 30 run)

Source: Mistral's open-weights post (Oct 29). The weights are an improved checkpoint with three more weeks of RL, published as mistralai/Mistral-Large-4-1T-A52B.

• Status goes from preview to released.
• License is now the Mistral Ecosystem Open-Weight License. It is a modified MIT license, and Model-as-a-Service operators need a separate commercial license.
• Price goes from the preview's $1.36/$4.18 to $0.70/$2.10 per Mtok ($0.07 cached input), with effectiveAt 2026-10-29. The pricing CSV row updates with it.
• New row: BrowseComp 84.5 (self-reported).
• Rank: it stays at #51, and its index goes from 2425.2 to 2447.4.
• Not filed:
  • Terminal-Bench 4.0 41% comes from Artificial Analysis's harness. The column uses Vals, and Carbon's AA figure was left out the same way.
  • FrontierCode 2.0 19.7% is a new version with no column. A new slug is Punit's call.
  • AA Coding Agent Index 48%, AA Cyber Index 51%, FinWorkBench (Finch) 73.3% and B3 attack resistance 97.1% have no columns.
• Kept as is: the preview-era rows (DeepSWE, SWE-Atlas-QnA, self-reported TB4 28.3, and the Harvey and Finance Agent Vals rows run on the preview API). Vals ran the OSS checkpoint only on HLAB-Vals-1.1 and Finance Agent v2.1, which have no corpus column. The note says which checkpoint they came from.
• Vals Terminal-Bench 4.0 (Oct 31 run): 27.78 → 26.26. Vals' Oct 29 board lists the open-weights checkpoint separately as Mistral Large 4 OSS, at 26.26 ±5.13 (#26 of 59). The preview-API run is still listed as Mistral Large 4 (27.78, feat(data): ingest Gemini 4 Carbon and Particle-1, Mistral Large 4 open weights (Oct 26–31, 2026) #25). The corpus allows one independent row per benchmark and this entry now describes the open-weights checkpoint, so the row uses the OSS run and its note keeps the preview figure. The index is unchanged at 2447.4.
• Parameter conflict: Hugging Face's launch blog says about 49B active parameters. The docs' 52B is kept, and the note records the conflict.

Checked, not ingested

• DeepSeek V4.1 Flash Cyber, unverified. Only LLM Gateway's timeline lists it, dated Oct 29. DeepSeek's change log, news pages and API docs still don't mention it as of Oct 31.
• GPT-6.1 Terra Pro, unverified. Only LM Market Cap lists it. OpenAI's API changelog and news page don't mention it.
• Rime Prism (Oct 28). A speech-to-speech voice-agent model, not an LLM.
• Also not LLMs: Nemotron 3.5 Transcribe (speech) and Genie 4 (world model).
• Nemotron 3.5 Super VL. It got a Vals TB4 row (0.00) on Oct 29, but it isn't in the corpus.
• Text Arena. The board is still the Oct 27 snapshot this PR's sibling fix(data): Kimi K3.1 Pro TB-Science (Vals) and Oct 27 Text Arena rows for Argon, Haiku 5.5, Granite 4.2 8B #24 already used. Carbon's row there (1528) is marked Preliminary, so it isn't filed.
• LFM3 (Liquid AI). Liquid AI isn't in organizations.json, so adding it is Punit's call.

Leaderboard moves

	Before	After
Gemini 4 Carbon	new	#1 (3233.2)
Kimi K3.1 Pro	#1 (3169.9)	#2 (3164.1)
Gemini 4 Argon	#2 (3126.1)	#3 (3099.0)
Muse Wonder Full	#3 (3100.5)	#4 (3081.3)
Claude Opus 5.5	#4 (3072.4)	#5 (3061.3)

• A new frontier model with wins over most of the field re-anchors the ratings. Everything below the top shifts down a few points, and models that lose to Carbon directly drop more: Argon -28.4, GPT-6.1 Terra -24.6, Muse Wonder Full -20.5.
• Most ranks below Carbon move down by one place.

Counts

597 → 599 models, 86 → 87 orgs, 6329 → 6337 results.

Re-pins

• Goldens: derive top 5 and feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1 slug, movers (Sarvam 105B +1618.4, then Mistral Large 4 +1239.6), Llama 3.1 405B, and snapshot asOfIso 2026-10-31.
• e2e:
  • Rankings, explorer, dashboard rail, quick-compare default and closed-frontier leader now show Gemini 4 Carbon (3233.2).
  • Movers is +1618.4, the compare legend is GPT-5.2 2370.5 and DeepSeek-V3.1 (Thinking) 1799.8, and the Opus 4.6 family delta is +241.9.
  • The scatter-tooltip test keeps Kimi K3.1 Pro because Carbon has no price and isn't plotted.
• README: counts updated.

Gates

• bun run generate-dataset && bun run derive && bun run validate-data is clean, and so is bun run ci (lint, typecheck, unit tests, build, budgets).
• Playwright: 115 of 115 pass on the Oct 31 head.

Merge order with #24

#24 (Kimi K3.1 Pro's Terminal-Bench Science correction) touches the same derived files and goldens. Whichever merges second needs a re-derive and re-pin. The next nightly run does that on whichever PR is still open.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CAthXVhtrPcUij6nbrXMv7