Skip to content

Commit c9e4b8a

Browse files
committed
fix: tool-call pipeline, crash recovery, and Responses API tool shape
Problem A (service stops working): - /v1/responses catch block called res.json() unconditionally; after SSE headers were sent this threw ERR_HTTP_HEADERS_SENT which escaped the async handler as an unhandled rejection, killing the Node process under the default --unhandled-rejections=throw. Added res.headersSent guard and added a process-level unhandledRejection handler in index.js as a safety net so a single bad request can never bring the whole proxy down. Problem B (tool calls never fired from pi): - buildExternalToolRegistry only read tool.function.name; the Responses API sends a flat shape {type,name,parameters} without a .function wrapper, so every pi tool was silently dropped. Added normalizeToolDefinition() that accepts both the Chat Completions nested shape and the flat Responses shape. Also fixed router.js tool_choice to accept flat {type,name} alongside {type,function:{name}}. Problem C (polling truncation): - pollForAssistantResponse returned as soon as any content or reasoning appeared, even mid-generation (done=false). Reasoning models emit the reasoning part first and the text part only after; returning early produced reasoning-only responses. Now waits for done===true and keeps the last partial snapshot only as a timeout fallback. Problem D (streaming first-delta timeout too short): - DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS was hardcoded to 4000ms. DeepSeek takes ~10-30s before its first token. The event stream consistently timed out and fell back to polling, which then hit the truncation bug above. Changed to 30000ms (configurable via OPENCODE2API_EVENT_FIRST_DELTA_TIMEOUT_MS). Problem E (nested function_calls tag not parsed): - Models echo the <function_calls> example from the contract reminder and wrap the actual JSON in a second outer tag, producing: <function_calls>\n<function_calls>\n{...}\n</function_calls> rawCallsFromJsonText called JSON.parse(body) which failed because the body starts with the stray tag, and returned []. Added findFirstJsonValue as a fallback to scan past the prefix and extract the embedded JSON. Problem F (upstream error detection): - collectFromEvents waited out the full first-delta window (30s) before discovering a MessageAbortedError via polling. Now resolves immediately when a message.updated event carries info.error. Additional fixes: - XML child element arguments (Cline/Roo style): <read><path>a.txt</path></read> was matched but children were discarded, producing empty arguments. Added argsFromXmlChildren() that maps child tag names to declared schema properties. - Contract reminder injected as the last prompt part (just before generation), not only in the 16KB+ system prompt where it gets buried. Empirically raised tool call compliance from 50% to 100% in baseline tests. - finish==='tool' treated as intermediate (not done) in pollForAssistantResponse. - All changes covered by 128 passing tests (+23 new in this commit).
1 parent 67b1e2f commit c9e4b8a

7 files changed

Lines changed: 665 additions & 36 deletions

File tree

‎index.js‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,17 @@ console.log(` - Cleanup Interval: ${finalConfig.CLEANUP_INTERVAL_MS}ms`);
147147
console.log(` - Cleanup Max Age: ${finalConfig.CLEANUP_MAX_AGE_MS}ms`);
148148
console.log(` - Debug: ${finalConfig.DEBUG ? 'Yes' : 'No'}`);
149149

150+
// A rejected promise inside a request handler must not take the whole proxy down.
151+
// Node's default --unhandled-rejections=throw turns one bad request into a process
152+
// exit, which reads to users as "the service stopped working after a while".
153+
process.on('unhandledRejection', (reason) => {
154+
const detail = reason instanceof Error
155+
? `${reason.name}: ${reason.message}`
156+
: typeof reason === 'string' ? reason : JSON.stringify(reason);
157+
console.error('[Proxy] Unhandled rejection (request dropped, server continues):', detail);
158+
if (reason instanceof Error && reason.stack) console.error(reason.stack);
159+
});
160+
150161
// Start the proxy
151162
try {
152163
const proxy = startProxy(finalConfig);

‎src/proxy.js‎

Lines changed: 88 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,10 @@ const STARTING_WAIT_ITERATIONS = 120;
193193
const STARTING_WAIT_INTERVAL_MS = 1000;
194194
const DEFAULT_REQUEST_TIMEOUT_MS = 300000;
195195
const DEFAULT_POLL_INTERVAL_MS = 500;
196-
const DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS = 4000;
196+
// Reasoning models can take well over 10s before emitting their first token.
197+
// A short window here makes the event stream give up and fall back to polling on
198+
// every request, which loses true streaming. Configurable for slow backends.
199+
const DEFAULT_EVENT_FIRST_DELTA_TIMEOUT_MS = Number(process.env.OPENCODE2API_EVENT_FIRST_DELTA_TIMEOUT_MS) || 30000;
197200
const DEFAULT_EVENT_IDLE_TIMEOUT_MS = Number(process.env.OPENCODE2API_EVENT_IDLE_TIMEOUT_MS) || 8000;
198201

199202
const OPENCODE_BASENAME = 'opencode';
@@ -1078,15 +1081,22 @@ export function createApp(config) {
10781081
}
10791082

10801083
function extractFromParts(parts) {
1081-
if (!Array.isArray(parts)) return { content: '', reasoning: '' };
1084+
if (!Array.isArray(parts)) return { content: '', reasoning: '', toolParts: [] };
10821085
const content = parts.filter(p => p.type === 'text').map(p => p.text).join('');
10831086
const reasoning = parts.filter(p => p.type === 'reasoning').map(p => p.text).join('');
1084-
return { content, reasoning };
1087+
const toolParts = parts.filter(p => p.type === 'tool');
1088+
return { content, reasoning, toolParts };
10851089
}
10861090

10871091
async function pollForAssistantResponse(sessionId, timeoutMs, intervalMs = DEFAULT_POLL_INTERVAL_MS) {
10881092
const pollStart = Date.now();
10891093
const startedAt = Date.now();
1094+
// Best-effort snapshot of the most recent in-flight assistant message. Polling
1095+
// observes partial messages: a reasoning model emits its reasoning part first and
1096+
// the text part only afterwards, so returning on the first non-empty snapshot
1097+
// truncates the answer to the reasoning alone. Keep the partial around purely as
1098+
// a timeout fallback and otherwise wait for the message to actually finish.
1099+
let lastPartial = null;
10901100
while (Date.now() - startedAt < timeoutMs) {
10911101
const messagesRes = await client.session.messages({ path: { id: sessionId } });
10921102
const messages = messagesRes?.data || messagesRes || [];
@@ -1095,10 +1105,25 @@ export function createApp(config) {
10951105
const entry = messages[i];
10961106
const info = entry?.info;
10971107
if (info?.role !== 'assistant') continue;
1098-
const { content, reasoning } = extractFromParts(entry?.parts || []);
1108+
const { content, reasoning, toolParts } = extractFromParts(entry?.parts || []);
10991109
const error = info?.error || null;
1100-
const done = Boolean(info.finish || info.time?.completed || error);
1101-
if (done || content || reasoning) {
1110+
// finish === 'tool' marks an intermediate turn that pauses for a tool
1111+
// result; the assistant is not done producing output yet.
1112+
const finished = info.finish && info.finish !== 'tool';
1113+
const done = Boolean(finished || info.time?.completed || error);
1114+
if (toolParts.length > 0) {
1115+
logDebug('Polling found tool parts', {
1116+
sessionId,
1117+
count: toolParts.length,
1118+
parts: toolParts.map(p => ({
1119+
id: p.id,
1120+
tool: p.tool,
1121+
status: p.state?.status,
1122+
input: p.state?.input
1123+
}))
1124+
});
1125+
}
1126+
if (done) {
11021127
if (error) {
11031128
console.error('[Proxy] OpenCode assistant error:', error);
11041129
}
@@ -1112,10 +1137,23 @@ export function createApp(config) {
11121137
});
11131138
return { content, reasoning, error };
11141139
}
1140+
if (content || reasoning) {
1141+
lastPartial = { content, reasoning, error: null };
1142+
}
1143+
break;
11151144
}
11161145
}
11171146
await sleep(intervalMs);
11181147
}
1148+
if (lastPartial) {
1149+
logDebug('Polling timeout with partial response', {
1150+
sessionId,
1151+
ms: Date.now() - pollStart,
1152+
contentLen: lastPartial.content.length,
1153+
reasoningLen: lastPartial.reasoning.length
1154+
});
1155+
return lastPartial;
1156+
}
11191157
logDebug('Polling timeout', { sessionId, ms: Date.now() - pollStart });
11201158
throw new Error(`Request timeout after ${timeoutMs}ms`);
11211159
}
@@ -1236,6 +1274,22 @@ export function createApp(config) {
12361274
event.properties.info.sessionID === sessionId) {
12371275
const info = event.properties.info;
12381276
const finish = info.finish;
1277+
// An aborted or failed message never produces another delta. Without
1278+
// this, the collector waits out the whole first-delta window before
1279+
// polling rediscovers the same error.
1280+
if (info.error && !finished) {
1281+
finished = true;
1282+
clearTimeout(timeoutId);
1283+
if (firstDeltaTimer) clearTimeout(firstDeltaTimer);
1284+
if (idleTimer) clearTimeout(idleTimer);
1285+
logDebug('SSE upstream message error', {
1286+
sessionId,
1287+
ms: Date.now() - startedAt,
1288+
error: info.error.name || 'UnknownError'
1289+
});
1290+
resolve({ content, reasoning, error: info.error });
1291+
break;
1292+
}
12391293
// Reconcile active tool calls from the full message snapshot so we
12401294
// detect pending tools even when only message.updated fires.
12411295
if (Array.isArray(info.parts)) {
@@ -1523,7 +1577,13 @@ export function createApp(config) {
15231577
body: {
15241578
model: { providerID: pID, modelID: mID },
15251579
system: systemWithGuard,
1526-
parts: parts,
1580+
// Append a short contract reminder as the last part so the model
1581+
// sees it immediately before generating. With the contract only in
1582+
// the 16KB+ system prompt it gets buried; position matters a lot for
1583+
// compliance. deepseek-v4-flash-free: 50% → 100% call rate.
1584+
parts: externalToolContext.reminder
1585+
? [...parts, { type: 'text', text: externalToolContext.reminder }]
1586+
: parts,
15271587
...(requestParams.max_tokens && { max_tokens: requestParams.max_tokens }),
15281588
...(requestParams.temperature !== undefined && { temperature: requestParams.temperature }),
15291589
...(requestParams.top_p !== undefined && { top_p: requestParams.top_p }),
@@ -2239,7 +2299,9 @@ export function createApp(config) {
22392299
body: {
22402300
model: { providerID: pID, modelID: mID },
22412301
...(systemWithGuard ? { system: systemWithGuard } : {}),
2242-
parts,
2302+
parts: externalToolContext.reminder
2303+
? [...parts, { type: 'text', text: externalToolContext.reminder }]
2304+
: parts,
22432305
...(max_output_tokens && { max_tokens: max_output_tokens }),
22442306
...(temperature !== undefined && { temperature }),
22452307
...(top_p !== undefined && { top_p })
@@ -2679,9 +2741,25 @@ export function createApp(config) {
26792741

26802742
return res.json(response);
26812743
} catch (error) {
2682-
console.error('[Proxy] Responses API Error:', error.message);
2744+
console.error('[Proxy] Responses API Error:', error?.message || error?.data?.message || error?.name || error);
26832745
const transformed = transformUpstreamError(error);
2684-
res.status(transformed.statusCode).json(transformed.error);
2746+
// Once the SSE headers are out, res.json() throws ERR_HTTP_HEADERS_SENT. That throw
2747+
// escapes this async handler as an unhandled rejection, which terminates the whole
2748+
// process under Node's default --unhandled-rejections=throw. Report the failure on
2749+
// the already-open stream instead.
2750+
if (res.headersSent) {
2751+
try {
2752+
res.write(`data: ${JSON.stringify({
2753+
type: 'response.failed',
2754+
response: { error: transformed.error.error || transformed.error }
2755+
})}\n\n`);
2756+
res.write('data: [DONE]\n\n');
2757+
} catch (writeError) {
2758+
logDebug('Failed to report error on open response stream', { error: writeError.message });
2759+
}
2760+
return res.end();
2761+
}
2762+
return res.status(transformed.statusCode).json(transformed.error);
26852763
}
26862764
});
26872765

‎src/tool-runtime/parser.js‎

Lines changed: 90 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,18 @@ function registryNames(registry) {
7070
return [...names].sort((a, b) => b.length - a.length);
7171
}
7272

73+
/** Map every accepted tool name (namespaced and original) to its declared JSON schema. */
74+
function registrySchemas(registry) {
75+
const schemas = new Map();
76+
if (!Array.isArray(registry)) return schemas;
77+
registry.forEach((tool) => {
78+
if (!tool?.parameters) return;
79+
if (tool.namespacedName) schemas.set(tool.namespacedName, tool.parameters);
80+
if (tool.originalName) schemas.set(tool.originalName, tool.parameters);
81+
});
82+
return schemas;
83+
}
84+
7385
/**
7486
* DSML parameter bodies usually sit on their own line. Drop one leading and one trailing
7587
* newline so `<parameter>\nvalue\n</parameter>` yields "value", while interior whitespace
@@ -133,7 +145,16 @@ function rawCallsFromJsonText(text) {
133145
try {
134146
return rawCallsFromJsonPayload(JSON.parse(trimmed));
135147
} catch {
136-
return [];
148+
// The body may be prefixed by stray markup — a nested <function_calls> tag copied
149+
// from the contract reminder, or a tool wrapper tag emitted by the model before
150+
// the JSON payload. Scan for the first balanced JSON value and try again.
151+
const found = findFirstJsonValue(trimmed);
152+
if (!found) return [];
153+
try {
154+
return rawCallsFromJsonPayload(JSON.parse(found.json));
155+
} catch {
156+
return [];
157+
}
137158
}
138159
}
139160

@@ -270,6 +291,66 @@ function argsFromAttrs(attrs) {
270291
return null;
271292
}
272293

294+
/**
295+
* Coerce an XML child element's text using the declared JSON-schema type. A schema type of
296+
* `string` is honoured verbatim so a path like `123.txt` or a body of JSON-looking text is
297+
* not silently turned into a number or an object.
298+
*/
299+
function coerceSchemaValue(value, schema) {
300+
const type = Array.isArray(schema?.type) ? schema.type[0] : schema?.type;
301+
if (type === 'string') return value;
302+
const trimmed = value.trim();
303+
if (!trimmed) return value;
304+
if (type === 'number' || type === 'integer') {
305+
const num = Number(trimmed);
306+
return Number.isFinite(num) ? num : value;
307+
}
308+
if (type === 'boolean') {
309+
if (/^true$/i.test(trimmed)) return true;
310+
if (/^false$/i.test(trimmed)) return false;
311+
return value;
312+
}
313+
if (type === 'object' || type === 'array') {
314+
try {
315+
return JSON.parse(trimmed);
316+
} catch {
317+
return value;
318+
}
319+
}
320+
// Untyped: fall back to the permissive decoding used elsewhere.
321+
return coerceParamValue(value, '');
322+
}
323+
324+
/**
325+
* Arguments carried as XML child elements rather than JSON:
326+
*
327+
* <read>
328+
* <path>a.txt</path>
329+
* <offset>10</offset>
330+
* </read>
331+
*
332+
* Widely used by Cline/Roo-style harnesses, so many models emit it from training even when
333+
* asked for JSON. Only child names declared in the tool's own schema are accepted, which
334+
* keeps prose containing angle brackets from being mistaken for arguments.
335+
*/
336+
function argsFromXmlChildren(body, parameters) {
337+
const properties = parameters && typeof parameters === 'object' ? parameters.properties : null;
338+
if (!properties || typeof properties !== 'object') return null;
339+
const allowed = Object.keys(properties);
340+
if (!allowed.length) return null;
341+
342+
const args = {};
343+
let matched = 0;
344+
for (const key of allowed) {
345+
const re = new RegExp(`<${escapeRegExp(key)}\\s*>([\\s\\S]*?)</${escapeRegExp(key)}\\s*>`, 'i');
346+
const found = body.match(re);
347+
if (!found) continue;
348+
matched += 1;
349+
args[key] = coerceSchemaValue(trimParamValue(found[1]), properties[key]);
350+
}
351+
return matched > 0 ? args : null;
352+
}
353+
273354
/**
274355
* End index of a tag's attribute region, skipping over quoted strings and balanced JSON
275356
* so that a `>` inside an attribute value (`arguments={"command":"ls > out"}`) does not
@@ -306,7 +387,7 @@ function findTagEnd(text, from) {
306387
* Registry-gated tag formats: `<tool .../>`, `<tool ...>body</tool>`, and an unclosed
307388
* `<tool ...>` followed by a JSON body.
308389
*/
309-
function extractTagNamed(text, names) {
390+
function extractTagNamed(text, names, schemas = new Map()) {
310391
const calls = [];
311392
const spans = [];
312393
if (!names.length) return { calls, spans };
@@ -366,7 +447,11 @@ function extractTagNamed(text, names) {
366447
}
367448
}
368449
} else {
369-
calls.push({ name, arguments: {} });
450+
// No JSON body. Arguments may still be present as XML child elements named
451+
// after the schema's properties; dropping them here produced tool calls with
452+
// empty arguments, which fail validation for any tool with required fields.
453+
const xmlArgs = argsFromXmlChildren(body, schemas.get(name));
454+
calls.push({ name, arguments: xmlArgs || {} });
370455
}
371456
spans.push([match.index, consumedEnd]);
372457
}
@@ -399,12 +484,13 @@ function collectAll(text, registry) {
399484
const source = typeof text === 'string' ? text : '';
400485
if (!source) return { calls: [], spans: [] };
401486
const names = registryNames(registry);
487+
const schemas = registrySchemas(registry);
402488

403489
const results = [
404490
extractCanonical(source),
405491
extractDsml(source),
406492
extractJsonWrapper(source),
407-
extractTagNamed(source, names),
493+
extractTagNamed(source, names, schemas),
408494
extractBareJson(source, names)
409495
];
410496

0 commit comments

Comments
 (0)