[sroussey]

Summary

HFT tool calling parsed free-form output after the fact, so a small local model could emit a malformed or off-schema call. When a turn must end in one call, decode is now constrained with @huggingface/transformers-structured-output (already a dependency, previously used only by HFT_StructuredGeneration).

• When it applies: toolChoice is a named tool or "required", the family is qwen, qwen2, qwen3 or hermes (JSON inside <tool_call>), and no tool results are in the history (the call's own or its checkpoint prefix's).
• How: <tool_call>\n is appended to the prompt and a StructuredOutputProcessor constrains the body to {"name": <const>, "arguments": <tool inputSchema>}. With several tools ("required") the name is an enum with per-tool if/then branches, so a wrong name is refused token by token. Tool schemas are embedded for the engine (x-* annotations dropped, local $refs re-rooted), and envelopes are cached by content so the engine's mask cache is reused. The closing tag is written back before the family parser runs, and the body is not streamed as assistant text since it is the call.
• Checkpoints: history and tools are read from the checkpoint prefix as well as the call, and decoded calls are validated against both tool lists.
• Fallback: if the grammar attempt throws mid-generation, the turn decodes again without it; usage carries across the retry so totals only grow.
• Left unconstrained: "auto" (a grammar can't choose between answering and calling), other families, Qwen 3.5 (XML calls), a prompt ending inside an open <think>, and any tool schema the engine rejects.

Also: the HF Inference conformance test pinned Llama 3.3 70B to a router provider that no longer serves it; it now uses Llama 3.1 8B on deepinfra (live, with tool support on the HF router).

New logic lives in HFT_ConstrainedToolCall.ts, wired into HFT_ToolCalling.ts.

Test plan

• HFT_ToolCalling.test.ts drives the real grammar through a one-byte-per-id vocabulary (name refusal, missing arguments, checkpoint prefix, fallback, usage across the retry); HFT provider suite passes
• bun run build:types, oxlint and oxfmt clean
• CI green on ba533e6
• Not run against a real Qwen model (fake tokenizer only): worth confirming EOS follows the closing brace
• Constrained turn not exercised against a KV session

🤖 Generated with Claude Code

https://claude.ai/code/session_019frF5cRGhNB8N8QKDYY7dG

[mergestorm-vortex]

Review progress ██████████ 3/3 files

Approve — reviewed files look good at 1992dbd.

✅ All clear — nothing to fix.

---

[sroussey]

test-vitest-ai-provider-api failed on 1992dbd, and I don't think it's this PR's.

The failing calls are HFI_ToolCalling_Stream in providers/huggingface-inference, which hit the live Hugging Face Inference API and get We have not been able to find inference provider information for model meta-llama/Llama-3.3-70B-Instruct. This PR only touches providers/huggingface-transformers (local ONNX), and that provider's own job is a separate check. The error comes from the remote provider lookup for that model, so it isn't something this diff can cause.

No fix exists yet that I can port. Build, lint and typecheck are green on this head. I'll re-run the failed job once, when the rest of the run finishes.

---

Generated by Claude Code

[mergestorm-vortex]

Review progress ██████████ 4/4 files

Comment — found 1 issue(s) at 082c46d.

Actionable comment posted: 1

🤖 Prompt for AI agents

Pull request workglow-dev/libs#1031: https://github.com/workglow-dev/libs/pull/1031 (head branch `claude/hft-constrained-tool-calling`)

Verify each finding against current code. Fix only still-valid concrete bugs, skip the
rest with a brief reason, keep changes minimal, and validate. Skip Decision required,
policy forks, and "consider X" alternatives. Do not add new features, refactors, or
architecture beyond the fix; prefer the smallest diff.

Findings to address:
1. In `@packages/test/src/test/ai-provider-api/HFI_Generic.integration.test.ts` (line 25, Important):
   `TOOL_CALLING_ROUTABLE` is hard-coded to `false`, so `RUN` is always false even when `HF_TOKEN` is set and `runAiProviderConformance` skips its entire suite. That also disables the embedding finite-vector check for the separate `EMBED_MODEL_ID` and registry/capability checks, although those paths do not use the unroutable tool-call model. Preserve the checks that remain runnable while skipping only the failing tool-call coverage.

---

[sroussey]

test-vitest-ai-provider-api is still red on 082c46d, but for a different reason than before. With the HF Inference suite skipped, the failure is now Gemini_ImageGenerate_Stream returning HTTP 402 RESOURCE_EXHAUSTED: Your prepayment credits are depleted from the Google AI Studio account the CI key belongs to.

That is a billing state on the live Gemini API, not something this PR's diff (providers/huggingface-transformers plus the HF Inference test skip) can cause, and there is nothing to port. It needs the Gemini key topped up or that test skipped separately. I'm not re-running it, since the error is deterministic until credits are restored.

---

Generated by Claude Code

[mergestorm-vortex]

Review progress ██████████ 4/4 files

Approve — reviewed files look good at 6cf4074.

✅ All clear — nothing to fix.

---

[sroussey]

Update on test-vitest-ai-provider-api at 6cf4074: the HF Inference conformance suite no longer fails with Llama 3.1 8B pinned to deepinfra. The remaining 3 failing files, all live-provider calls, are unrelated to this PR:

• Gemini image generation and Gemini text embedding: HTTP 402 RESOURCE_EXHAUSTED (depleted prepayment credits), as in my earlier comment.
• Xai_Generic (StructuredGenerationTask): The model is currently at capacity due to high demand, a retryable xAI server error.

None of these touch providers/huggingface-transformers. The Gemini ones need the key's credits restored. The xAI one is transient capacity and may pass on a re-run.

---

Generated by Claude Code

[mergestorm-vortex]

Review progress ██████████ 4/4 files

Request changes — found 1 issue(s) at 1b61bd3.

Actionable comment posted: 1

🤖 Prompt for AI agents

Pull request workglow-dev/libs#1031: https://github.com/workglow-dev/libs/pull/1031 (head branch `claude/hft-constrained-tool-calling`)

Verify each finding against current code. Fix only still-valid concrete bugs, skip the
rest with a brief reason, keep changes minimal, and validate. Skip Decision required,
policy forks, and "consider X" alternatives. Do not add new features, refactors, or
architecture beyond the fix; prefer the smallest diff.

Findings to address:
1. In `@providers/huggingface-transformers/src/ai/common/HFT_ConstrainedToolCall.ts` (line 226, Blocker):
   `candidateTools` can select tools from `prefix.tools` when the checkpoint consumer's `input.tools` is empty, but `HFT_ToolCalling` later validates parsed calls using only `input.tools` (`filterValidToolCalls` at line 690). A checkpoint turn relying on its stored tools can therefore decode a valid constrained call and then silently drop it, so no `toolCalls` event reaches the caller. Use the same effective tool list for post-parse validation.

---

[mergestorm-vortex]

Review progress ██████████ 4/4 files

Request changes — found 2 issue(s) at d08deac.

Actionable comments posted: 2

🤖 Prompt for AI agents

Pull request workglow-dev/libs#1031: https://github.com/workglow-dev/libs/pull/1031 (head branch `claude/hft-constrained-tool-calling`)

Verify each finding against current code. Fix only still-valid concrete bugs, skip the
rest with a brief reason, keep changes minimal, and validate. Skip Decision required,
policy forks, and "consider X" alternatives. Do not add new features, refactors, or
architecture beyond the fix; prefer the smallest diff.

Findings to address:
1. In `@providers/huggingface-transformers/src/ai/common/HFT_ToolCalling.ts` (line 573, Important):
   `createStreamingTextStreamer` creates a fresh usage reporter for each `decode()` attempt. If constrained generation emits usage and then throws, the fallback reporter starts again at zero (including emitting the same prompt count with output `0`); usage snapshots replace one another in `StreamEventAccumulator`, so the final reported usage omits the failed attempt's tokens and can move backwards. Preserve cumulative counters across both attempts, including the repeated prompt work.

Additional findings outside changed lines:
1. In `@providers/huggingface-transformers/src/ai/common/HFT_ConstrainedToolCall.ts` (line 226, Blocker):
   Validate calls against checkpoint tools too
   `candidateTools` can select tools from `prefix.tools` when the checkpoint consumer's `input.tools` is empty, but `HFT_ToolCalling` later validates parsed calls using only `input.tools` (`filterValidToolCalls` at line 690). A checkpoint turn relying on its stored tools can therefore decode a valid constrained call and then silently drop it, so no `toolCalls` event reaches the caller. Use the same effective tool list for post-parse validation.

---

Still open from earlier reviews (1) — the lines they point at have not changed:

• 🔴 Blocker · Validate calls against checkpoint tools too · providers/huggingface-transformers/src/ai/common/HFT_ConstrainedToolCall.ts line 226 · from 1b61bd3
candidateTools can select tools from prefix.tools when the checkpoint consumer's input.tools is empty, but HFT_ToolCalling later validates parsed calls using only input.tools (filterValidToolCalls at line 690). A checkpoint turn relying on its stored tools can therefore decode a valid constrained call and then silently drop it, so no toolCalls event reaches the caller. Use the same effective tool list for post-parse validation.

---

[mergestorm-vortex]

Review progress ██████████ 4/4 files

Request changes — found 1 issue(s) outside changed lines at ba533e6.

Actionable comment posted: 1

🤖 Prompt for AI agents

Pull request workglow-dev/libs#1031: https://github.com/workglow-dev/libs/pull/1031 (head branch `claude/hft-constrained-tool-calling`)

Verify each finding against current code. Fix only still-valid concrete bugs, skip the
rest with a brief reason, keep changes minimal, and validate. Skip Decision required,
policy forks, and "consider X" alternatives. Do not add new features, refactors, or
architecture beyond the fix; prefer the smallest diff.

Findings to address:
1. In `@providers/huggingface-transformers/src/ai/common/HFT_ConstrainedToolCall.ts` (line 226, Blocker):
   Validate calls against checkpoint tools too
   `candidateTools` can select tools from `prefix.tools` when the checkpoint consumer's `input.tools` is empty, but `HFT_ToolCalling` later validates parsed calls using only `input.tools` (`filterValidToolCalls` at line 690). A checkpoint turn relying on its stored tools can therefore decode a valid constrained call and then silently drop it, so no `toolCalls` event reaches the caller. Use the same effective tool list for post-parse validation.

---

Still open from earlier reviews (1) — the lines they point at have not changed:

• 🔴 Blocker · Validate calls against checkpoint tools too · providers/huggingface-transformers/src/ai/common/HFT_ConstrainedToolCall.ts line 226 · from 1b61bd3
candidateTools can select tools from prefix.tools when the checkpoint consumer's input.tools is empty, but HFT_ToolCalling later validates parsed calls using only input.tools (filterValidToolCalls at line 690). A checkpoint turn relying on its stored tools can therefore decode a valid constrained call and then silently drop it, so no toolCalls event reaches the caller. Use the same effective tool list for post-parse validation.

---