Skip to content

feat: register Claude Sonnet 5.5 in model/alias registry - #31

Merged
samuelboland merged 2 commits into
mainfrom
dux-register-claude-sonnet-5-5
Oct 5, 2026
Merged

samuelboland merged 2 commits into
mainfrom
dux-register-claude-sonnet-5-5

Conversation

@staging-supernova-dx-appf-io

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

What this does

Registers Claude Sonnet 5.5 (claude-sonnet-5-5) in RubyLLM's model and alias registries, following the same pattern already merged for Claude Opus 5.5.

Supernova (appfolio/supernova) reads effort support, pricing, and context-window data for LLMs out of ruby_llm's registry. Claude Sonnet 5.5 was missing, which meant supernova's Llm::ThinkingClamp.clamp would silently return 'none' for this model — no error, just silently dropped reasoning effort. This PR closes that gap by adding the registry data; a follow-up PR in appfolio/supernova will bump the pinned ruby_llm revision to the commit this PR merges as.

Changes:

  • lib/ruby_llm/models.json: 8 new entries for Sonnet 5.5 (one per existing Sonnet 5 counterpart) across anthropic, bedrock (anthropic., us., eu., global., au., jp. prefixes), and vertexai. Pricing matches the matching Sonnet 5 entry (eu. priced slightly higher), sourced from the AWS Marketplace rate card checked 2026-10-05. metadata.reasoning_options is effort-only (low/medium/high/xhigh/max) with no toggle option, since Sonnet 5.5 rejects thinking: {type: "disabled"} on Bedrock (verified live against Bedrock outside this PR's sandbox). Capabilities follow the Opus 5.5 pattern (drop structured_output for bedrock/vertexai per AWS's model card). knowledge_cutoff, metadata.knowledge, created_at, and metadata.last_updated set per AWS's Sonnet 5.5 launch (2026-09-28 / 2026-06). family, context_window, max_output_tokens, and modalities carried over unchanged from Sonnet 5. No existing entries were modified.
  • lib/ruby_llm/aliases.json: new claude-sonnet-5-5 alias block, matching the Sonnet 5 alias format (us.anthropic. bedrock id), not the Opus 5.5 alias format, since Sonnet 5.5 is Sonnet 5's successor.
  • spec/ruby_llm/models_sonnet_5_5_spec.rb: new spec asserting all 8 provider/id entries exist with the five effort values and no toggle option, correct metadata.cost, and that both aliases resolve correctly. A reviewer comment caught that the spec initially looked models up by id alone, silently skipping several bedrock/vertexai entries that share an id; this was fixed to match by [provider, id] pairs so all 8 entries are exercised.

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Performance improvement

Scope check

  • I read the Contributing Guide
  • This aligns with RubyLLM's focus on LLM communication
  • This isn't application-specific logic that belongs in user code
  • This benefits most users, not just my specific use case

Required for new features

N/A — this is registry data, following the precedent of the already-merged Opus 5.5 entries, not new feature code.

Quality check

  • I ran overcommit --install and all hooks pass
  • I tested my changes thoroughly
    • For provider changes: Re-recorded VCR cassettes with bundle exec rake vcr:record[provider_name]
    • All tests pass: bundle exec rspec
  • I updated documentation if needed
  • I didn't modify auto-generated files manually (models.json, aliases.json)

Note: models.json and aliases.json were edited manually, following the same approach used for the prior Opus 5.5 registration. Only targeted specs (models_sonnet_5_5_spec.rb, models_opus_5_5_spec.rb, models_json_validation_spec.rb, models_spec.rb, chat_model_aliases_spec.rb) were confirmed run and passing; the full bundle exec rspec suite was not explicitly reported as run. Rubocop was reported clean. The live Bedrock call confirming the thinking: disabled rejection was done outside this PR's sandbox against the appfolio-ci-dev-tooling account and is not reproduced as an automated test here.

AI-generated code

  • I used AI tools to help write this code
  • I have reviewed and understand all generated code (required if above is checked)

API changes

  • Breaking change
  • New public methods/classes
  • Changed method signatures
  • No API changes

Registers claude-sonnet-5-5 for anthropic, bedrock (base, us., eu., global.,
au., jp.) and vertexai, one entry per existing Sonnet 5 entry. Pricing is
unchanged from Sonnet 5 (AWS Marketplace rate card, checked 2026-10-05).
Reasoning is effort-only with no toggle option, since Sonnet 5.5 rejects
thinking: {type: "disabled"}; capabilities follow the Opus 5.5 entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-authored-by: Sam Boland <sam.boland@appfolio.com>
@staging-supernova-dx-appf-io

Copy link
Copy Markdown
Author

🤖 Supernova Code Review — View trace

🔄 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

@staging-supernova-dx-appf-io staging-supernova-dx-appf-io Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I compared the 8 new models.json entries, the alias, and the spec against the existing Sonnet 5 and Opus 5.5 entries, the Model::Info capability helpers, and the Bedrock Converse request path.

Checked and consistent:

  • Data and spec match: pricing, limits and reasoning values in the data match what the spec asserts.
  • Inherited patterns: the eu. 1.1x pricing and the alias format both follow existing precedent. The capabilities layout copies Opus 5.5: structured_output only on the anthropic entry.
  • Ordering: each entry sits in the correct sorted position.

Findings:

  • spec/ruby_llm/models_sonnet_5_5_spec.rb L14–17: the claim that the "no toggle" choice was "verified live on Bedrock" conflicts with the PR description, which says no live Bedrock check was done. Either cite the real source or reword it as an assumption.
  • nit, L29: the raw-id lookup only ever hits the anthropic claude-sonnet-5-5 entry, so the vertexai, bare bedrock, au. and jp. entries are never asserted.

🔄 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

Comment thread spec/ruby_llm/models_sonnet_5_5_spec.rb Outdated
Comment on lines +14 to +17
# Unlike Sonnet 5, these entries have no toggle reasoning option: Sonnet 5.5
# rejects thinking: {type: "disabled"} (verified live on Bedrock 2026-10-05)
# and only accepts adaptive thinking with output_config.effort, the same as
# Opus 5.5.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment says the toggle removal was "verified live on Bedrock 2026-10-05", but the PR description says no live Bedrock call was made because the sandbox had no Bedrock access.

Leaving out toggle is the one real behavior change here compared with Sonnet 5. Without it, consumers (e.g. Llm::ThinkingClamp) can't turn thinking off for any of the 8 entries. If the "rejects thinking: {type: "disabled"}" claim was actually carried over from Opus 5.5 and never tested against Sonnet 5.5, the registry could block a mode the model supports, and this comment would wrongly present that as verified for future maintainers.

Fix: either point to the real source (e.g. an AWS/Anthropic doc saying Sonnet 5.5 is adaptive-only), or reword this as an assumption carried over from Opus 5.5 until someone runs a live Converse check.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment is accurate. The live check was run outside the coder container, on the appfolio-ci-dev-tooling account on 2026-10-05: a Converse call to us.anthropic.claude-sonnet-5-5 with thinking: {type: "disabled"} was rejected, and the error said to use adaptive thinking with output_config.effort instead. The same call with adaptive thinking and effort max succeeded. Keep the comment, but say where the check ran ("verified live on the appfolio-ci-dev-tooling Bedrock account, 2026-10-05, outside this PR's container"), so it does not contradict the PR description. The second comment (match on [provider, id] so all 8 entries are covered) should be done.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Queued both fixes — updating the comment to cite the appfolio-ci-dev-tooling Bedrock account/date explicitly, and changing the spec lookup to match on [provider, id] so all 8 entries get tested.


🔍 View trace

# lookups renormalize the region prefix to the configured bedrock_region,
# which would mask the EU-specific entry's own pricing.
model = RubyLLM.models.all.find { |m| m.id == id }

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: all.find { |m| m.id == 'claude-sonnet-5-5' } only ever returns the anthropic entry, because the vertexai entry has the same id. So the vertexai entry, the bare anthropic.claude-sonnet-5-5, and the au./jp. entries are never tested. Matching on [provider, id] pairs would cover all 8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@samuelboland samuelboland left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registry data checked against the Sonnet 5 and Opus 5.5 entries; both review findings fixed.

@samuelboland
samuelboland merged commit 65c3dc0 into main Oct 5, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants