[punitarani]

Requested by Punit · project thread

Opened by the nightly ModelBeats sync routine. Draft: Punit reviews and merges.

Also carries #21. GitHub's API won't merge stacked PRs, so on Oct 27 #21 (Vals AI, ARC Prize and Arena refreshes, Gemini 4 Argon's price, Step 5 Preview's open weights, and its own code-review fixes) was folded into this branch and this PR merges both. #21's description has the details. On this combined head Kimi K3.1 Pro is #1 at 3169.9, then Gemini 4 Argon 3126.1, Muse Wonder Full 3100.5, Claude Opus 5.5 3072.4 and Muse Wonder 3017.3; the scores in the sections below are #20's alone.

Before: the catalog was as of October 10, 2026. It was missing Meta's first frontier models (released October 12), Meta's first open-weights Muse model, NVIDIA's Nemotron 3.5 Super VL and Bolt (all released October 15), DeepSeek's October 17 checkpoint of V4.1-Flash, MiniMax's M3.1 Flash (October 19), iFlytek's Spark X2.6 pair (October 20 and 22), Moonshot's Kimi K3.1 Pro and NVIDIA's Nemotron 3.5 Bolt E4B (both October 23), and OpenAI's GPT-6.1 Terra (October 26). MiniMax released M3.1 Flash's weights on October 23.

After: the catalog is as of October 26, 2026, with 12 new models and 1 new benchmark. Kimi K3.1 Pro enters at #1 (3240.4) on Moonshot's self-reported numbers, just ahead of Claude Sonnet 5.5 (3233.1) and Claude Opus 5.5 (3219.4). Muse Wonder Full enters at #4 (3164.1), GPT-6.1 Terra at #6 (3068.5) and Muse Wonder at #7 (3026.4). DeepSeek-V4.1-Flash-1017 enters at #16, MiniMax M3.1 Flash (now open weights) at #17, Muse Spark Open at #38, the two larger Nemotrons at #86 and #98, and Spark-X2.6-4B at #118. Nemotron 3.5 Bolt E4B (2 rows) and iFlytek's cloud Spark X2.6 (no scores) are unranked.

Model specs and scorecard rows come from each lab's own pages: dev.meta.ai, research.meta.ai and Meta's Hugging Face cards, NVIDIA's developer blog and Hugging Face cards, DeepSeek's API change log, pricing page and Hugging Face card, MiniMax's launch blog, docs, release notes and Hugging Face card, and SparkLLM's Hugging Face card, GitHub README and launch post, Moonshot's launch blog and API docs, and OpenAI's GPT-6.1 Terra launch post, API docs and system card addendum. Kimi K3.1 Pro's parameter count and price come from Artificial Analysis, as Kimi K3's did. Where Vals AI ran the same benchmark on launch day, the independent Vals number is filed instead.

Window covered

2026-10-10 to 2026-10-26. The Oct 11 and Oct 12 nightly runs did not complete, so the Oct 13 run covered three days. Later runs added new releases to this branch instead of opening a second PR: the Oct 15 releases on Oct 16, DeepSeek's Oct 17 checkpoint on Oct 18, MiniMax M3.1 Flash on Oct 20, and iFlytek's Spark X2.6 pair on Oct 22. Spark-X2.6-4B went up on Hugging Face on Oct 20 and the Oct 21 run missed it. The Oct 23 run found nothing new from the tracked labs and updated MiniMax M3.1 Flash's entry for its weights release. Kimi K3.1 Pro and Nemotron 3.5 Bolt E4B shipped later on Oct 23 and were added on Oct 24. The Oct 25 run failed at start and the Oct 26 run found nothing new. GPT-6.1 Terra shipped on Oct 26 and was added on Oct 27 in an on-demand run.

New models

Model	Released	Open/closed	Price in/out ($/M)	Headline score	Rank
Muse Wonder Full (Meta)	2026-10-12	closed (Muse Code subscription only)	no per-token price	DeepSWE 79.0, SWE-Marathon 57.5	#4 (3164.1)
Muse Wonder (Meta)	2026-10-12	closed	4 / 20	DeepSWE 78.8	#7 (3026.4)
Muse Spark Open (Meta, added Oct 16)	2026-10-15	open weights, Muse Model License	no first-party price	GPQA 89.0, Terminal-Bench 2 76.4	#38 (2555.4)
Nemotron 3.5 Super VL (NVIDIA, added Oct 16)	2026-10-15	open weights, OpenMDW 1.1	no first-party price	GPQA 89.52, SWE-bench Verified 71.10	#86 (2084.0)
Nemotron 3.5 Bolt 74B-A9B (NVIDIA, added Oct 16)	2026-10-15	open weights, OpenMDW 1.1	not on an API	GPQA 86.43, Terminal-Bench 2 50.98	#98 (1958.2)
DeepSeek-V4.1-Flash-1017 (DeepSeek, added Oct 18)	2026-10-17	open weights, MIT	0.30 / 1.20 (peak; unchanged)	DeepSWE 77.0, AutomationBench 60.5	#16 (2836.0)
MiniMax M3.1 Flash (MiniMax, added Oct 20; weights Oct 23)	2026-10-19	open weights, MiniMax Community License	0.15 / 0.60	DeepSWE 73.7, AutomationBench 62.8	#17 (2832.6)
Spark-X2.6-4B (iFlytek, added Oct 22)	2026-10-20	open weights, Apache-2.0	not on an API	SWE-bench Verified 62.8, SWE-bench Pro 54.3	#118 (1765.2)
Spark X2.6 (iFlytek, added Oct 22)	2026-10-22	closed	CNY launch price only, left empty	no published scores	unranked
Kimi K3.1 Pro (Moonshot, added Oct 24)	2026-10-23	closed until weights ship (promised by Nov 4)	2 / 10	Terminal-Bench 4.0 65.7, HLE (tools) 70.0, SWE-Marathon 51.7	#1 (3240.4)
Nemotron 3.5 Bolt E4B (NVIDIA, added Oct 24)	2026-10-23	open weights, OpenMDW 1.1	no first-party price (on build.nvidia.com)	BFCL v3 67.31, IFBench 64.00	unranked (2 rows)
GPT-6.1 Terra (OpenAI, added Oct 27)	2026-10-26	closed	1 / 5	Terminal-Bench 4.0 49.49 (Vals), HealthBench Professional 62.7	#6 (3068.5)

Corpus choices worth checking:

• Muse Wonder Full isn't in the Meta Model API. It's included only in the $50 and $200 Muse Code plans, so it follows the Gemini 4 Argon precedent: apiAvailable: false, price null.
• Independent over self-reported: Harvey's Legal Agent Benchmark, Vals Finance Agent v2, Terminal-Bench 4.0 and Terminal-Bench Science use Vals AI's launch-day runs (Wonder, Wonder Full). Meta's Finance Agent numbers (56.6 and 59.1) match Vals'.
• OSWorld 2: filed as Meta's OSWorld V2.1 partial-credit score (72.6 and 78.4), per the corpus convention. The binary scores (34.3 and 39.4) are in the row notes.
• Cyber rows for Wonder Full (CyberGym 88.4, ExploitBench 58.9) come from the model card's risk section.
• Undisclosed architecture: both Wonder models use archClass: dense with "architecture undisclosed".
• Muse Spark Open (HF card, blog): 816B total / 42B active MoE, 1M context, NVFP4 weights. The license requires an agreement for model-as-a-service providers above $20M revenue. The 10 rows are the card's xhigh numbers. Left out: the Artificial Analysis figures quoted on the card (Intelligence Index 39, GDPval-AA v2 1401, Omniscience), the HLE-Diamond subset, the vision and video suites with no corpus column, and the safety metrics.
• Nemotron 3.5 Super VL (HF card, NVIDIA developer blog, cited in the model file): 120B / 12B hybrid Mamba-Transformer MoE, 1M context. The 17 rows are NVIDIA's NeMo Gym evaluations from the BF16 card; HLE is the full set. Rows NVIDIA marks [offline] and its GDPval-AA v2 figure are not filed.
• Nemotron 3.5 Bolt 74B-A9B (HF card): compressed from Super VL and shipped with native 2-bit weights that fit in 24GB. The 8 rows are NVIDIA's numbers for that 24GB build, and it has no hosted API, so apiAvailable: false.
• DeepSeek-V4.1-Flash-1017 (HF card, change log, pricing): a post-training update on the same 552B MoE as the Sep 10 V4.1-Flash, filed as a separate entry the way the V4-Flash-0731 checkpoint was. The deepseek-flash API alias, the app and the legacy V4-Flash names now serve it at the same prices, so the Sep 10 entry is marked deprecated and keeps its Vals and Arena rows, which were run before Oct 17. The 5 rows are HLE (full set, no tools) 42.6, Terminal-Bench 4.0 42.3 (Claude Code harness), DeepSWE 77.0, NL2Repo 68.2 and AutomationBench 60.5. Left out: Agents' Last Exam (35.4 is a pass rate, the column holds the score component), Chartography (83.0 is with tools, the column is without), and ProgramBench, Apex Bench, BabyVision and ZeroBench, which have no corpus column.
• AutomationBench bound: normMax goes from 50 to 56 so DeepSeek's cited 60.5 passes the validator's bound check (the old ceiling allowed up to 55), and on Oct 23 from 56 to 58 so MiniMax's 62.8 passes too (56 allowed up to 61.6). Bounds feed only the category bars and radar, not the Elo ranking. The Oct 18 change moved no model by 5 points or more; I checked by regenerating without the new model.
• MiniMax M3.1 Flash (blog, HF card, license): filed Oct 20 as closed because the weights were only promised. They landed on Hugging Face on Oct 23, so on Oct 23 the entry moved to open-weights (the Step 5 Preview precedent). The MiniMax Community License is free for non-commercial use. Commercial use must show "Built with MiniMax M3.1-Flash" and send MiniMax a notice, and products above $20M yearly revenue need written authorization. The card also discloses the architecture: about 428B total / 23B active native multimodal MoE with MiniMax Sparse Attention throughout, 1M context. The card gives the launch table as text, so 7 rows are now filed from it: DeepSWE 73.7 and OSWorld 2.0 61.7 (partial credit, as before), CyberGym 82.2 (the blog's text rounded it to 82%), and new rows for Terminal-Bench 4.0 38.1 (Codex harness), NL2Repo 61.2, AutomationBench v1.0.6 62.8 and CharXiv RQ 82.5. Left out: HLE 43.4 (no tools, text-only subset; dropped in review, see below), Agents' Last Exam 29.6 (a pass rate), GDP.pdf AA 20.8 (taken from Artificial Analysis), and BankerToolBench, LOCA-Bench, PostTrain Bench, SWE Refactor Bench and BabyVision, which have no corpus column. The new rows dropped it from feat(compare): revamp compare page — unified spec sheet, 4 slots, verdict + field-context rail #15 to fix(data): Gemini 4 Argon GA price, Step 5 Preview open weights, Vals AI, Arena and ARC Prize results (Oct 14 to Oct 23, 2026) #21 on Oct 23 (#22 once Kimi K3.1 Pro joined), since it now loses head-to-head to DeepSeek-V4.1-Flash-1017 on Terminal-Bench 4.0, NL2Repo and DeepSWE; dropping the HLE row in review moved it to feat(data): ingest releases from Sep 25 to Oct 10, 2026 #17. Vals AI's independent Terminal-Bench 4.0 run (25.25) is filed in fix(data): Gemini 4 Argon GA price, Step 5 Preview open weights, Vals AI, Arena and ARC Prize results (Oct 14 to Oct 23, 2026) #21.
• Spark-X2.6-4B (HF card, GitHub, launch post): iFlytek's 4B on-device model from its SparkLLM (XHToken) team, with a 1M-token context and none/low/high thinking levels. The card's benchmark table is an image, so the 4 rows are the figures SparkLLM also gives as text in its launch post (thinking set to high): SWE-bench Verified 62.8, SWE-bench Pro 54.3, MCP Atlas 62.4 and BrowseComp 53.1. Left out: BFCL-V4 71.8 (the corpus has only bfcl and bfcl-v3, matching earlier ingests) and AA-LCR v1.1 67.3 (no column). Spark-X2.5-4B (Sep 1) predates this routine and isn't in the corpus.
• Kimi K3.1 Pro (blog, API guide, Artificial Analysis): Moonshot calls it a post-trained Kimi K3 (same 2.8T MoE, 1M context) with much more RL, served on the Kimi API at $2/$10. Full weights are promised by Nov 4, so it is closed until they ship (the MiniMax M3.1 Flash precedent); Moonshot also says it stripped cyber-related data before the open release. Terminal-Bench 4.0 65.7 and BenchCAD 92.7 (with tools) are stated in the blog text. The blog's 14-benchmark grid is an image only, so 6 more rows are filed where at least two independent text transcriptions of it agree exactly (the Kimi K3 precedent): SWE-Marathon 51.7, Terminal-Bench Science 38.1 and HLE with tools 70.0 (three transcriptions each), GPQA 94.2, BrowseComp 92.1 and OSWorld 2 79.6 (two each). All are Moonshot's own runs at max effort, mostly in the Claude Code harness. It lands at feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1 on self-reported numbers only; Artificial Analysis puts it feat(data): add EQ-Bench 4 and Creative Writing v3 benchmarks (eqbench.com) #4 on its index. Left out: OfficeQA Pro v2 (the corpus column holds v1 numbers), Frontier Finance, ProgramBench (binary variant), GDPval-AA v2.1, Agents' Last Exam, GDP.pdf, ChartoGraphy, ZeroBench, HLE without tools, Kimi's internal suites, and DeepSWE 75.5 and MMMU-Pro 84.1, which only one transcription carries. Kimi K3.1 Pro Highspeed isn't filed separately: it has no scores of its own, and Kimi's pricing table renders client-side, so its first-party price couldn't be read.
• Nemotron 3.5 Bolt E4B (HF card, NVIDIA blog): Gemma-4-E4B (8B total, 4B effective) post-trained by NVIDIA for local voice and device assistants, 64K context, shipped as a 4GB Q4_K_M build plus BF16 weights under OpenMDW-1.1. paramsB is 8 and activeParamsB is left empty, since "effective" parameters aren't MoE-active ones. Following the Bolt 74B entry, the 2 rows are the shipped Q4_K_M column: BFCL v3 67.31 and IFBench 64.00. Left out: Tau2-Bench (the card links the aggregate board; the corpus has per-domain columns) and 18 suites with no corpus column (MultiChallenge, RoleBench, EQ-Bench v3, RULER, LongBench v2 and others). With two rows it stays unranked.
• GPT-6.1 Terra (launch post, API docs, system card addendum, Vals): OpenAI's successor to GPT-5.6 Terra, $1/$5 per Mtok with cached input at $0.05, 1.05M context (922K max input). Terminal-Bench 4.0 uses Vals AI's launch-day run (49.49, tied feat(data): GPT-5.6 July 30 price cuts (Luna −80%, Terra −20%) #10 of 56, max effort) as the headline over OpenAI's 50.3. HealthBench Professional 62.7 (length-adjusted), ExploitBench 97.1 and SEC-Bench Pro 83.33 come from the system card. The launch post gives Agents' Last Exam, MMMU-Pro, OSWorld 2.1 and Terminal-Bench Science only as charts or relative claims, so those aren't filed. Its feat(data): GPT-5.6 Sol/Terra/Luna tier split + Frontier Elo v2 (D26) #6 rests on 4 benchmarks, 3 of them OpenAI's own, so read it loosely, as with Kimi K3.1 Pro.
• Spark X2.6 (IT Home): iFlytek's closed cloud model, launched Oct 22 at its 1024 Developer Festival. It's a 293B-A30B MoE with 1M context, trained on domestic Chinese compute. iFlytek published no benchmark table, so it has no scores. Launch pricing is a limited-time CNY rate (0.9 / 3.6 yuan per million tokens), so the price is left empty rather than converted.
• Sep 10 V4.1-Flash note: it said deepseek-v4-pro requests would be served by V4.1-Flash from Sep 14. DeepSeek reversed that, and its pricing page still lists DeepSeek-V4-Pro-0813 under that name, so the note now says so. DeepSeek's change log quotes NL2Repo 65.4 for the Sep 10 checkpoint while the original model card says 64.0. The corpus keeps 64.0, which cites the card.

New benchmark

Benchmark	Category	normMax	Rows
SWE-Marathon v1.1 (swe-marathon)	coding	60	21

• What it is: Abundant AI's 20-task, multi-hour software-engineering benchmark. Each task scores 1 only if every verifier test passes, averaged over 8 trials. Source: swemarathon.org.
• Independent rows (18): taken from the public v1.1 leaderboard. They cover Claude Opus 5, Opus 4.8, Fable 5.1 and Sonnet 5; GPT-6 Astra and GPT-5.6 Sol, Terra and Luna; Kimi K3; Grok 4.5, 4.6 and 4.7; GLM 5.3, 5.3 Flash and 5.2 Max; Muse Spark 1.3; Gemini 3.7 Flash; and DeepSeek V4 Pro.
• Self-reported rows (3): Muse Wonder's 53.1 and Wonder Full's 57.5 come from Meta's scorecard, and Kimi K3.1 Pro's 51.7 from Moonshot's blog. They aren't on the public leaderboard yet.
• Fable 5 left out: the leaderboard flags its 45.0 run as having used a fallback model, so it is noted on the model instead.
• Harness: each model runs in its lab's own agent product, so the rows compare model and harness pairs. This is noted in the benchmark description.
• Muse Spark 1.3: its leaderboard row is the xhigh run, while the corpus model describes max. This is noted on the row.

Code review fixes (Oct 27)

• Rows dropped and noted on the model: MiniMax M3.1 Flash's HLE 43.4 is the text-only subset (2,158 questions), and the HLE column is the full set. Claude Fable 5's SWE-Marathon 45.0 used a fallback model per the leaderboard.
• Nemotron 3.5 Bolt E4B: the card lists it on build.nvidia.com since Oct 23, so apiAvailable is now true (no price), and its Q4_K_M weights are 6.18 GB. Bolt 74B-A9B keeps a null Q4 size: it ships as a 2-bit 24GB build, which the Q4 field can't express, so hardware fit falls back to the parameter estimate.
• Notes added: Kimi K3.1 Pro's SWE-Marathon run doesn't state its trial count; Grok 4.7's SWE-Marathon is a high-effort run while its other figures are xHigh; Terra's ExploitBench is the same evaluation as Sol's (Terra's addendum doesn't give the effort); the AutomationBench description now says rows use either Zapier's private set or the 600 public v1.0.6 tasks.
• Lineage: the generator picked the nearest older family member, which gave five wrong "succeeds" links. A new optional corpus predecessor field overrides that: Kimi K3.1 Pro now succeeds Kimi K3 (was K3 Fast), Spark X2.6 succeeds Spark X1 (was Spark-X2.6-4B), and GPT-6.1 Terra, Muse Spark Open, Bolt E4B and Spark-X2.6-4B have no predecessor (they were linked to GPT-6.1 Sol, Muse Spark 1.3, Bolt 74B and Spark X1). validate-data still checks every link is same-family and older.

Counts (before → after)

• Models: 585 → 597
• Organizations: 86 → 86
• Benchmarks: 170 → 171
• Results: 6033 → 6138
• Pricing rows: 253 → 258
• Ranked models: 485 → 495

Leaderboard moves

Ranks are against main.

• Top 5 is now:
  1. Kimi K3.1 Pro 3240.4 (new)
  2. Claude Sonnet 5.5 3233.1 (was feat(data): add FrontierBench with its full 8-entry launch leaderboard #3)
  3. Claude Opus 5.5 3219.4 (was feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1)
  4. Muse Wonder Full 3164.1 (new)
  5. Gemini 4 Argon 3154.0 (was feat(data): ingest SealedBench sealed capability index #2)
• Why the top changed: the Muse Wonder rows land on the same agent columns where Argon's 9 self-reported rows sit (Vals Finance Agent, Harvey LAB, Terminal-Bench 4.0, DeepSWE). On DeepSWE, Wonder Full's 79.0 beats Argon's 77.9. Adding DeepSeek-V4.1-Flash-1017 on Oct 18 lowered most frontier scores in the joint fit; Opus 5.5 lost 34.2 points and Sonnet 5.5 lost 11.8, so Sonnet edged ahead. MiniMax M3.1 Flash, added Oct 20 with 3 self-reported rows, barely moved the top (Opus 5.5 3225.3 → 3227.5). Spark-X2.6-4B, added Oct 22, lowers every frontier score by about 3 points and changes no top-10 rank. MiniMax's 5 new card rows on Oct 23 lower Opus 5.5 by 16.5 and Argon by 20.9 and change no top-8 rank. On Oct 24 Kimi K3.1 Pro takes feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1: its self-reported HLE (tools) 70.0 and BrowseComp 92.1 top those columns, its SWE-Marathon 51.7 is third behind the two Wonder models, and it beats Sonnet 5.5 and Opus 5.5 head-to-head on GPQA. Muse Wonder drops to feat(data): GPT-5.6 Sol/Terra/Luna tier split + Frontier Elo v2 (D26) #6. On Oct 27 GPT-6.1 Terra enters at feat(data): GPT-5.6 Sol/Terra/Luna tier split + Frontier Elo v2 (D26) #6, pushing Muse Wonder to Add benchmarks to topbar search dropdown #7 and GPT-6 Astra to feat(dashboard): add Pareto frontier to quality-vs-price scatter #8; the frontier scores moved by under 11 points and the top 5 order held.
• Muse Spark 1.3 drops feat(data): add EQ-Bench 4 and Creative Writing v3 benchmarks (eqbench.com) #4 → feat(data): GPT-5.6 July 30 price cuts (Luna −80%, Terra −20%) #10: it now loses head-to-head to both Wonder models on DeepSWE, JobBench and OSWorld. Its new SWE-Marathon row (30.6, xhigh) also sits far below the frontier.
• Smaller moves: MiMo-V2.6-Pro feat(data): ingest Composite-Bench, the certified long-horizon browser-use benchmark #5 → feat(data): ingest 8 models released since the last catalog update #11, Claude Opus 5 feat(data): add Claude Opus 5 with its official launch benchmarks #9 → feat(data): ingest the 16 models released since the last catalog update #12, Claude Haiku 5.5 feat(data): GPT-5.6 July 30 price cuts (Luna −80%, Terra −20%) #10 → search: rank results by relevance (match → recency → Elo) #14, GPT-6 Sol feat(data): ingest the 16 models released since the last catalog update #12 → feat(data): ingest Muse Wonder, Kimi K3.1 Pro, GPT-6.1 Terra and 9 more releases (Oct 10 to Oct 26, 2026) #20. MiniMax M3.1 Flash moved feat(data): ingest Muse Wonder, Kimi K3.1 Pro, GPT-6.1 Terra and 9 more releases, plus Vals/ARC/Arena refresh (#20, #21) #23 → feat(data): ingest releases from Sep 25 to Oct 10, 2026 #17 when its text-only HLE row was dropped in review. Grok 4.7 rises on its SWE-Marathon 45.6.
• Movers: Sarvam +1620.9, Mistral Large 4 +1313.9, DeepSeek-V4-Flash-0731 +1031.7, SmolLM3 +1003.6, MiMo-V2.6-Pro +955.1.
• Top open model: still MiMo-V2.6-Pro (feat(data): ingest 8 models released since the last catalog update #11). DeepSeek-V4.1-Flash-1017 is next at fix(data): remove unconfirmed Sep 25 ingest entries and refresh stale numbers #16, then MiniMax M3.1 Flash at feat(data): ingest releases from Sep 25 to Oct 10, 2026 #17. Kimi K3.1 Pro would take this spot once its weights ship.

Checked but not ingested

• Meta scorecard rows left out:
  • Internal composites: Personal Superintelligence Index (59.1 / 54.3) and MemoryBench (85.5 / 85.6) are Meta-internal.
  • Harvey LAB-AA (13.1 / 10.0): this is Artificial Analysis's v1.1 regrading with hallucination checks, not the Vals-run harvey-legal-agent column. The Vals numbers are filed instead.
  • AutomationBench-AA (45.4 / 39.1): this is AA's variant, not the Zapier private-set automationbench column.
  • ProgramBench (59.0 / 50.0): Meta counts runs that pass at least 95% of tests, while Vals scores Wonder Full at 0.50%. No corpus column exists.
  • ExploitGym: Wonder Full captured 92 flags, but only 21 used the intended vulnerability. No corpus column exists.
  • Opus 5.5 SWE-Marathon 46.3: this appears only on Meta's table, not the public leaderboard.
• GLM-Legal (Z.ai, Oct 12): a legal fine-tune of GLM-5.3-Flash in beta by application. It isn't generally available, and the corpus doesn't track legal-domain fine-tunes.
• Ling 3.1 Flash (inclusionAI): only an OpenRouter listing so far, with no model card or release post. Unverified.
• Oct 14 to 16 releases left out:
  • Mistral Timestral: a time-series forecasting model, not an LLM.
  • IBM Granite Switch 4.1: a composition of adapters over Granite 4.1, not a new base model.
  • Liquid AI LFM2.5-24B-A2B and Paper Instruments Ultramarine 1 Flash: neither lab is in organizations.json. Adding them is Punit's call.
• Pending follow-ups:
  • Gemini 4 Argon's price and Step 5 Preview's open weights are both handled in fix(data): Gemini 4 Argon GA price, Step 5 Preview open weights, Vals AI, Arena and ARC Prize results (Oct 14 to Oct 23, 2026) #21.
  • Mistral Large 4 open weights are still due by the end of October.
  • Kimi K3.1 Pro's weights are due by Nov 4; the entry flips to open weights when they land.
• Oct 17 to 18, not ingested: Tencent's Hy Memory 2.0 (a 30B-A3B memory model for an agent-memory SDK, not a general LLM) and Reflection AI's Beam 1.0 (Reflection is not in organizations.json).
• Oct 19 to 20, not ingested: Inception Labs' Mercury 3 (Inception is not in organizations.json) and OpenAI's "aurora" model IDs, which OpenAI hasn't released.
• Oct 20 to 22, not ingested: inclusionAI's Ming 3.0 TTS and ASR models (speech, not LLMs), Cohere Rerank 5, NVIDIA's Nemotron 3.5 embed and rerank models, and Mistral's Voxtral 3 (speech). An LLM Gateway listing dates Qwen3.6 27B to Oct 21, but Qwen released it in April and it's already in the corpus. KRAFTON, H2O.ai and "recursive" (Flint) aren't in organizations.json.
• Oct 23, not ingested: Baseten's mammoth-1 and elephant-1 (Oct 21; Baseten isn't in organizations.json). No tracked lab shipped anything else on Oct 23 or 24.
• Oct 24 to 27, not ingested: Kimi K3.1 Pro Highspeed (a faster serving tier of K3.1 Pro with no separate benchmark table) and Qwen3 Rerank (a reranker, not an LLM). Reflection's Beam 1.0 weights shipped Oct 16 under Apache 2.0, but Reflection is still not in organizations.json; adding it is Punit's call.

Re-pins

• Goldens: derive.test.ts (top 5 with Kimi K3.1 Pro first, codex −110.7, Llama 3.1 405B 1121.0 / #218, movers, the feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1 slug) and snapshot.test.ts (asOfIso 2026-10-26).
• e2e literals:
  • The rail, rankings and explorer feat(data): add the DeepMind Gemini + Poolside Laguna releases since Kimi K3 #1, the quick-compare default and the closed-frontier leader are now Kimi K3.1 Pro (3240.4). Anthropic's top filtered row is still Claude Sonnet 5.5.
  • The compare legend reads GPT-5.2 2401.9 and DeepSeek-V3.1 (Thinking) 1815.3.
  • The Sarvam mover is +1620.9, and the Opus 4.6 succession delta is +228.5.
  • The SWE-bench Verified leaderboard counts 140 models (Spark-X2.6-4B joins), and the GPQA open-model scatter still has 191 points.
  • The open, under-15B, reasoning explorer facet now has 22 models led by Spark-X2.6-4B (was 20, led by Falcon-H1R 7B; Bolt E4B is the 22nd), and the fine-tune ≥1M context filter shows 3 models (was 2).
• README: counts updated (597 models, 171 benchmarks).

Gates

• bun run generate-dataset && bun run derive && bun run validate-data: pass.
• bun run ci (build-catalog, typecheck, lint, test, build, budgets): pass.
• Playwright: 115/115 pass locally. On Oct 24 only the explorer facet count failed after the new score literals were set (21 → 22), and it passed after re-pinning. On Oct 27 the five score literals that moved with GPT-6.1 Terra were re-pinned and the full suite passed; after the review fixes four literals moved again and were re-pinned, and the suite passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CAthXVhtrPcUij6nbrXMv7

[punitarani]

Merged to main through #23 (squash 11385c1), which carries this PR's exact head (ad2c21d, including #21). GitHub's API won't merge stacked PRs, so closing this one.

---

Generated by Claude Code