Skip to content

Add guarded Responses streaming with explicit usage and cleanup state - #106

Draft
ijaesin wants to merge 10 commits into
feat/json-whitespace-guardfrom
feat/guarded-responses-stream
Draft

ijaesin wants to merge 10 commits into
feat/json-whitespace-guardfrom
feat/guarded-responses-stream

Conversation

@ijaesin

@ijaesin ijaesin commented Oct 25, 2026 •

Copy link
Copy Markdown

Add a separately versioned native Responses tool transport that feeds function arguments through the JSON whitespace guard before assembly, closes the HTTP response on failure, and distinguishes observed usage from final usage received. It borrows an AsyncOpenAI client, disables retries by preflight, preserves external cancellation, and requires a matching completed response plus strict validated tool output before success.

The transport preserves source/question text and normalized strict schema, bounds creation/consumption to60s, acquired HTTP-body cooperative cleanup separately to1s, and arguments/events to80,000 UTF-8bytes/10,000 decoded events, and validates stream/item identities, sequence, completion snapshots and usage. It does not mint existing generation receipts, validate source/access/meaning or change the existing SDK/app/worker/main paths. Local HTTP close is not proof of provider compute cancellation or billing savings.

Validation uses the actual installed SDK with synthetic MockTransport/SSE:119focusedtests, scopedRuff/format/strictmypyBOTHfiles pass; initial cleanup/overflow findings fixed and scoped re-review C0I0M0. Whole-branch finding corrected; scoped final re-review C0I0M0 with no unresolved findings. Fresh full checks pass: 9435 tests (8615 Python + 820 JS), lint/format/types/build/OpenAPI; failures/skips/warnings0. A completed prior full run exposed stale class references after a frozen reload test; ordered regression300 and focused119 now pass, with narrow fixture review C0I0M0. Post-fix recordedv1/v2 arguments through8syntheticnativeHTTPrequests preserve incomplete/finalusage versus whitespace/unknownfinalusage, close all8HTTPresponses and keep borrowedclients open. Synthetic chunk partitions do not reconstruct real SSE timing. No provider calls, retries, repair or paid old-runner execution this task. Existing470 execution files and historical experiment evidence are frozen.

Decisions and costs:

  • Separate native transport preserves legacy contracts but leaves app behavior unchanged until later generator/worker integration and acceptance.

  • Close without draining; retain observed usage and final-observation flag. Final provider charges/compute cancellation can remain unknown and savings are unproven.

  • Explicit60s operation/80k UTF-8/10k event limits may reject valid long runs and do not bound decoder memory for one native event.

  • Cancellation during cleanup remains a CancelledError subclass while retaining an earlier primary reason; callers that inspect only reason can miss cancellation and must handle exception class first.

  • Cooperative1s close can reject a slow valid cleanup and cannot forcibly stop cancellation-resistant arbitrary closers; pre-acquisition borrowedSDK internals are opaque, so no hard wall-clock SLA.

  • Rejecting JSON numbers that overflow finite Python floats prevents inf success but may require another representation for otherwise representable very large numbers.

Draft stacked on PR105; no main merge or application activation. Planning Site v128 published successfully (source9fe955e); previousv127 and7exact canonical document copies(sourcebe4c102) preserved. All3owned QA containers/anonymous volumes and current SDD workspace cleaned;25590closed, otherresources untouched. Worktree/branch remain. Repository progress records the post-publication receipt. Initial wholeQA was stopped (not counted as passing) for independent-review cleanup/finite-number fixes; a fresh run follows final code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant