Repository navigation
Conversation
find_tools lists each tool with id "apps-apple-com/apps/app", provider "apps-apple-com" and capability "apps/app". The capabilities parameter said "Exact capability IDs", so agents holding an id passed it and got invalid_option; discovery only accepts the address, scoped by providers. Describe both parameters by the listing field they take, with an example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop "not its provider-prefixed `id`" from the capabilities description. Naming the `capability` field with an example is enough: on GPT-6 Luna and GPT-6.1 Sol both wordings got every first contract lookup right. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrite the firecrawl_find_tools description to say how to find a provider (urls / query / browse), read its contract with the listed tool's provider and capability fields (never the id), and run it with firecrawl_scrape. Keeps "Prefer normal firecrawl_search for a data task". 992 -> 821 characters. Define the description once in alexandria.ts; server.ts and the search surface carried identical copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ben2W
force-pushed
the
find-tools-description
branch
from
October 14, 2026 23:13
8118a1e to
e1f3017
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, stacked on #536. Review only the last commit (e1f3017); the first two are #536.
Rewrites the
firecrawl_find_toolsdescription to lead with the lookup sequence and the field rule:992 → 821 characters. It also defines the description once in
alexandria.ts;server.tsand the search surface had identical copies.Evidence
Same setup as #536: description-only overrides, replayed Firecrawl responses, 19 tasks. First contract lookup right, current text → this text with #536. These runs used #536's earlier wording, which also said "not its provider-prefixed
id"; #536 has since dropped that clause after it checked out the same on Luna and Sol.The models routed through OpenRouter ran 1–2 repeats. Claude Sonnet 5.5 and Kimi K3 rarely looked up a contract (they ran the provider directly), so they are left out. Judged accuracy on Luna and Sol: 0.99 → 0.99 (Luna), 0.99 → 0.97 (Sol, replay) and 0.96 → 0.99 (Sol, live); the Sol dips are a model declining to quote a transcript. The other models weren't judged.
What this adds over #536 alone: on Sol, 0.37 fewer tool calls per run in replay and 0.32 live, against 0.21 for the parameter text alone. On Luna, the parameter text alone did as well or slightly better (0.25 vs 0.16).
Why a draft
tests/mcp-description-budget.test.mjspins "Prefer normal firecrawl_search" as routing copy from your AX runs. The tested text ended "firecrawl_search already returns matching providers with web results; use this tool for …". This PR keeps your sentence instead, so this exact wording wasn't run.find_toolsrouting weren't covered by these tasks.pnpm test: 166/166 pass, with regenerated contract snapshots.🤖 Generated with Claude Code