[staging-supernova-dx-appf-io]

What this does

Adds support for GPT-6 "mantle" models and fixes a bug in the Responses API handling where items and cost data were being lost.

Type of change

• Bug fix
• New feature
• Breaking change
• Documentation
• Performance improvement

Scope check

• I read the Contributing Guide
• This aligns with RubyLLM's focus on LLM communication
• This isn't application-specific logic that belongs in user code
• This benefits most users, not just my specific use case

Required for new features

• I opened an issue before writing code and received maintainer approval
• Linked issue: #___

Quality check

• I ran overcommit --install and all hooks pass
• I tested my changes thoroughly
  • For provider changes: Re-recorded VCR cassettes with bundle exec rake vcr:record[provider_name]
  • All tests pass: bundle exec rspec
• I updated documentation if needed
• I didn't modify auto-generated files manually (models.json, aliases.json)

AI-generated code

• I used AI tools to help write this code
• I have reviewed and understand all generated code (required if above is checked)

API changes

• Breaking change
• New public methods/classes
• Changed method signatures
• No API changes

[staging-supernova-dx-appf-io]

🤖 Supernova Code Review — View trace

🔁 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

[staging-supernova-dx-appf-io]

Adversarial review of the GPT-6 mantle routing, the Responses output-item preservation, the compaction API, the streaming changes, the cache-token accounting and ActiveRecord storage. I left 2 inline findings:

1. lib/ruby_llm/chat.rb:230: Chat#compact shadows Enumerable#compact, which Chat gets from include Enumerable. A call that used to return an Array of messages now makes a network request or raises.
2. lib/ruby_llm/protocols/responses/chat.rb:121: when a reply comes back as OutputItems, with_schema parsing is skipped, because Chat#normalize_schema_response only parses String content. Structured-output replies that include compaction or commentary items reach the caller unparsed. This happens on both the sync and streaming paths.

I checked the rest (signing paths for /compact, ActiveRecord round trip through content_raw → Content::Raw, format/replay of reasoning blocks, stream accumulation of blocks and cache tokens, VCR body stripping, Bedrock routing) and found nothing else actionable.

🔁 Re-run this review

— via automation "Code Review Opus 5.5 Medium" (run), installed by Sam Boland (sam.boland@appfolio.com)

[staging-supernova-dx-appf-io]

Chat#compact overrides Enumerable#compact. Chat does include Enumerable (line 8), and since Ruby 3.1 Enumerable#compact returns the non-nil elements. Any existing chat.compact call, or library code that treats a Chat as a generic Enumerable, used to get back an Array of messages. After this change the same call sends a billable network request, or raises UnsupportedFeatureError on Converse, OpenAI Chat Completions and other non-Responses protocols. Please give it a name that doesn't collide, e.g. compact_context or compact_conversation (and on Provider too, for consistency).

[staging-supernova-dx-appf-io]

When a reply becomes OutputItems, with_schema parsing is skipped without any error. Chat#normalize_schema_response only parses when response.content.is_a?(String) (chat.rb:371). So when a structured-output reply has a compaction item (server-side context_management) or a phase: "commentary" message, the caller gets an OutputItems object instead of the parsed Hash. The streaming path has the same problem, because Streaming#stream_response swaps in OutputItems before the chat normalizes the reply. The fix could live in normalize_schema_response: parse response.content.text when the content responds to :text. That keeps the raw items for replay, e.g. by checking for Protocols::Responses::OutputItems and parsing .text. Alternatively, document that schema replies may come back as OutputItems. A spec covering schema plus compaction would lock this in.

[samuelboland]

Review of a94ecff. Three changes, then this merges.

1. CI is red on every Ruby/Rails job because two examples have no cassette:

   • spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:3:2] (openai.gpt-6.1-sol replays every reasoning item through a multi-round tool loop)
   • spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:5] (keeps the existing replay through a tool round trip on openai.gpt-5.6-sol)
     Record both live, the same way as the other GPT-6 cassettes (bedrock_mantle_region = 'us-east-1', <AWS_...> placeholders only). Do not delete or skip either example; the GPT-5.6 one is the proof that today's models are unchanged. If Bedrock refuses the call, stop and paste the exact error here.

2. The inline note at lib/ruby_llm/chat.rb:230 is right, do it. Chat includes Enumerable, so Chat#compact replaces Enumerable#compact for every model, not only GPT-6. Rename the Chat method to compact_context (keep Provider#compact and Protocols::Responses#compact as they are; neither is Enumerable). Update the comment, the specs, and the PR body's API section. Add one spec asserting chat.to_a.compact still returns the messages.

3. The inline note at lib/ruby_llm/protocols/responses/chat.rb (structured output skipped for OutputItems) is right, do it, in Chat#normalize_schema_response (lib/ruby_llm/chat.rb:370): when @schema is set, the reply has no tool call, and response.content is a RubyLLM::Protocols::Responses::OutputItems, parse response.content.text with JSON.parse and assign the result to response.content, same rescue as the String branch. String content keeps today's path unchanged. Losing the item list on a schema reply only costs the compaction saving on the next request; the history is still sent in full under store: false. Add a unit spec with a commentary item plus a JSON final answer.

Verified and fine:

• Routing: MANTLE_ONLY_MODEL_PATTERN and FRONTIER_GPT_PATTERN widened to gpt-[56]; /compact is signed against compaction_url.
• models.json: three entries, 1,050,000 context, 128,000 output, prices 0.11/0.55 and 2.2/11.0 with 0.1x cache read and 1.25x cache write; aliases added.
• Every GPT-6-only path is gated on gpt6_model?; GPT-5.6 and older keep parse_legacy_usage, single-item reasoning replay and plain-text content.
• format_input replaces history from the last compaction item; format_gpt6_assistant_items replays base Content::Raw arrays, which is what ActiveRecord reads back.
• chat_methods.rb:420 now stores content_text for any Raw with #text. After merging with main that also covers InvokeAnthropic::ContentBlocks, which is opt-in per chat only, so it changes nothing for existing callers. The branch merges cleanly with main (a3dc133).
• VCR over-length body stripping only touches bodies above 4 MB.

[staging-supernova-dx-appf-io]

@samuelboland

Review of a94ecff. Three changes, then this merges.

1. CI is red on every Ruby/Rails job because two examples have no cassette:

   • spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:3:2] (openai.gpt-6.1-sol replays every reasoning item through a multi-round tool loop)
   • spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:5] (keeps the existing replay through a tool round trip on openai.gpt-5.6-sol)
     Record both live, the same way as the other GPT-6 cassettes (bedrock_mantle_region = 'us-east-1', <AWS_...> placeholders only). Do not delete or skip either example; the GPT-5.6 one is the proof that today's models are unchanged. If Bedrock refuses the call, stop and paste the exact error here.

2. The inline note at lib/ruby_llm/chat.rb:230 is right, do it. Chat includes Enumerable, so Chat#compact replaces Enumerable#compact for every model, not only GPT-6. Rename the Chat method to compact_context (keep Provider#compact and Protocols::Responses#compact as they are; neither is Enumerable). Update the comment, the specs, and the PR body's API section. Add one spec asserting chat.to_a.compact still returns the messages.

3. The inline note at lib/ruby_llm/protocols/responses/chat.rb (structured output skipped for OutputItems) is right, do it, in Chat#normalize_schema_response (lib/ruby_llm/chat.rb:370): when @schema is set, the reply has no tool call, and response.content is a RubyLLM::Protocols::Responses::OutputItems, parse response.content.text with JSON.parse and assign the result to response.content, same rescue as the String branch. String content keeps today's path unchanged. Losing the item list on a schema reply only costs the compaction saving on the next request; the history is still sent in full under store: false. Add a unit spec with a commentary item plus a JSON final answer.

Verified and fine:

• Routing: MANTLE_ONLY_MODEL_PATTERN and FRONTIER_GPT_PATTERN widened to gpt-[56]; /compact is signed against compaction_url.
• models.json: three entries, 1,050,000 context, 128,000 output, prices 0.11/0.55 and 2.2/11.0 with 0.1x cache read and 1.25x cache write; aliases added.
• Every GPT-6-only path is gated on gpt6_model?; GPT-5.6 and older keep parse_legacy_usage, single-item reasoning replay and plain-text content.
• format_input replaces history from the last compaction item; format_gpt6_assistant_items replays base Content::Raw arrays, which is what ActiveRecord reads back.
• chat_methods.rb:420 now stores content_text for any Raw with #text. After merging with main that also covers InvokeAnthropic::ContentBlocks, which is opt-in per chat only, so it changes nothing for existing callers. The branch merges cleanly with main (a3dc133).
• VCR over-length body stripping only touches bodies above 4 MB.

I've queued all three changes as one task, and I'm working on them now.

1. Cassettes: I'll record the two missing cassettes live for responses_gpt6_mantle_spec.rb[1:3:2] and [1:5]. They'll use us-east-1 and <AWS_...> placeholders only. Neither example will be deleted or skipped. If Bedrock refuses a call, I'll stop and paste the exact error here. I'll run only those two specs.
2. Rename: Chat#compact becomes Chat#compact_context. Provider#compact and Protocols::Responses#compact stay as they are. I'll update the comment, the specs and the PR body's API section, and add a spec that chat.to_a.compact still returns the messages.
3. Structured output: normalize_schema_response will parse content.text with JSON.parse when @schema is set, there is no tool call, and the content is an OutputItems. It will use the same rescue as the String branch, and String content keeps today's path. I'll add a unit spec with a commentary item plus a JSON final answer.

---

🔍 View trace

[samuelboland]

Change to item 1 of the previous comment: do not record those two cassettes, and make no more live Bedrock calls. Delete the two examples instead:

• spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:3:2] (openai.gpt-6.1-sol replays every reasoning item through a multi-round tool loop). Remove only the 6.1 Sol case; the Luna and Sol cases stay.
• spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:5] (keeps the existing replay through a tool round trip on openai.gpt-5.6-sol).

Update the PR body's test summary to match (no failures, and say these two were removed rather than recorded). Items 2 and 3 are unchanged. Neither needs a cassette; use unit specs only.

[staging-supernova-dx-appf-io]

@samuelboland

Change to item 1 of the previous comment: do not record those two cassettes, and make no more live Bedrock calls. Delete the two examples instead:

• spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:3:2] (openai.gpt-6.1-sol replays every reasoning item through a multi-round tool loop). Remove only the 6.1 Sol case; the Luna and Sol cases stay.
• spec/ruby_llm/protocols/responses_gpt6_mantle_spec.rb[1:5] (keeps the existing replay through a tool round trip on openai.gpt-5.6-sol).

Update the PR body's test summary to match (no failures, and say these two were removed rather than recorded). Items 2 and 3 are unchanged. Neither needs a cassette; use unit specs only.

I queued the amended task, and I'm working on it now. It replaces item 1 of the earlier task. I'll record no cassettes and make no more live Bedrock calls.

• Removed examples: I'll delete responses_gpt6_mantle_spec.rb[1:3:2], only the openai.gpt-6.1-sol case, so the Luna and Sol cases stay. I'll also delete [1:5], the openai.gpt-5.6-sol replay example. Any cassette files for these two go too.
• PR body: The test summary will say there are no failures and that these two examples were removed rather than recorded.
• Items 2 and 3: These are unchanged and use unit specs only, with no cassettes. The first is the compact_context rename plus the chat.to_a.compact spec. The second is the OutputItems parsing in normalize_schema_response plus the commentary-item spec.

The earlier queued task had already been dispatched before this amendment, so I queued the amended one separately rather than editing it. If the first run recorded cassettes or made Bedrock calls, that work will be reverted here.

[samuelboland]

Verified: GPT-6 routing and models.json entries, GPT-6-only gating with GPT-5.6 unchanged, compact_context rename, OutputItems schema parsing, CI green.