Add guarded Responses streaming with explicit usage and cleanup state

README.md

@@ -1,3 +1,5 @@
+현재 구현: [중단 가능한 Responses 스트림](docs/changes/guarded-responses-stream.md). 신규119/새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 모의 HTTP/SSE에서 공백 정체 후 닫힘·관측/최종 사용량·외부 취소를 검증했습니다. 이번 제공자 호출0이며 실제 제공자 중단/과금·기존 생성 SDK/앱 연결은 후속입니다.
+
 현재 구현: [JSON 연속 공백 감지기](docs/changes/json-whitespace-guard.md). 신규155/새전체9,316개(Python8,496+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 기록된 v2 인자를 같은 위치에서 감지했지만 실제 요청 취소·앱 연결·비용 절감은 후속입니다. [compact 지침 진단](docs/research/question-evidence-stall-investigation.md)은1회/60.106초 타임아웃·응답/사용량 미관측·재호출0으로 효과 미판정입니다.
 
 최신 실제 관측: [질문별 근거 생성 v2](docs/research/question-evidence-generation-v2-acceptance.md). 출력4,000상한에서도 미완료·SDK 거부였고, 응답 끝 연속 공백6,619bytes를 확인했습니다. 재호출 없이 사용량/실패를 보존하며 출력 정체 제어를 먼저 검토합니다. 의미·앱 인수는 남습니다.

docs/README.md

@@ -1,3 +1,5 @@
+현재 구현: [중단 가능한 Responses 스트림](changes/guarded-responses-stream.md). 신규119/새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 모의 HTTP/SSE에서 공백 정체 후 닫힘·관측/최종 사용량·외부 취소를 검증했습니다. 이번 제공자 호출0이며 실제 제공자 중단/과금·기존 생성 SDK/앱 연결은 후속입니다.
+
 현재 구현: [JSON 연속 공백 감지기](changes/json-whitespace-guard.md). 신규155/새전체9,316개(Python8,496+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 기록된 v2 인자를 같은 위치에서 감지했지만 실제 요청 취소·앱 연결·비용 절감은 후속입니다. [compact 지침 진단](research/question-evidence-stall-investigation.md)은1회/60.106초 타임아웃·응답/사용량 미관측·재호출0으로 효과 미판정입니다.
 
 최신 실제 관측: [질문별 근거 생성 v2](research/question-evidence-generation-v2-acceptance.md). 출력4,000상한에서도 미완료·SDK 거부였고, 응답 끝 연속 공백6,619bytes를 확인했습니다. 재호출 없이 사용량/실패를 보존하며 출력 정체 제어를 먼저 검토합니다. 의미·앱 인수는 남습니다.

docs/changes/guarded-responses-stream.md

@@ -0,0 +1,56 @@
+# 중단 가능한 Responses 도구 스트림
+
+2026-10-25 한국 시간. PR105의 순수 공백 감지기를 실제 OpenAI Python SDK 스트림 소비 경로에 연결하는 별도 재사용 transport다. OpenSpec root:null이므로 init 없이 SDD/TDD를 적용한다. 기존 앱/worker/생성 SDK와 그 영수증은 변경하지 않는다. 합성 HTTP/SSE 검증과 실제 제공자의 생성·과금·취소 보장은 구분한다.
+
+## Global Constraints
+
+Implementer owns ONLY new packages/agents/src/issuedigest_agents/guarded_responses_stream.py, new packages/agents/tests/test_guarded_responses_stream.py and Task1 checkbox. All470 existing non-Markdown execution files and prior experiment namespaces are frozen. Do not change exports/dependencies/schema/API/UI/server/DB/legacy SDK/runtime/model defaults/main. No private source data, provider calls, network, resource allocation, wholeQA, old runners or subagents. Controller owns fullQA/recorded-argument synthetic HTTP replay/docs/privacy/PR/Sites/cleanup. Borrow required caller-owned AsyncOpenAI; never create/close its client or change its settings. No retry/repair/fallback/truncation/background/retrieve/server-cancel requests. Stream close is local HTTP response close, not proof of provider compute cancellation/billing savings.
+
+## Spec
+
+New module public constants: GUARDED_RESPONSES_PROFILE="guarded-responses-tool-v1", STREAM_TOTAL_TOKEN_LIMIT=20000, STREAM_OUTPUT_TOKEN_LIMIT=4000, STREAM_DEADLINE_SECONDS=60, STREAM_CLOSE_DEADLINE_SECONDS=1, STREAM_ARGUMENT_BYTE_LIMIT=80000, STREAM_EVENT_LIMIT=10000. Public async function run_guarded_responses_tool(*, client:AsyncOpenAI, model:str, prompt:str, instructions:str, tool:ToolOutput[BaseModel])->GuardedResponsesToolResult.
+
+Result is frozen dataclass profile exact constant, output actual requested BaseModel type, usage:KnownUsage, usage_complete:Literal[True]. This is a transport result, NOT a stored generation receipt, source/access validation or semantic approval. Caller must run original source/question/canonical validation after using this transport; existing v2 receipts are not minted.
+
+Error GuardedResponsesError(ValueError) has safe fixed message "Guarded Responses request failed", known_usage:KnownUsage|None, may_have_started:bool, usage_complete:bool, stream_closed:bool, reason:Literal["input","whitespace","protocol","transport","timeout","usage","output","limit","cancelled"]. GuardedResponsesCancelled(asyncio.CancelledError) carries the same observations and fixed message so external cancellation remains cancellation. Raise from None, no raw event, input, partial output, original cause, raw provider IDs or error messages in exception text/extra fields. Properties only; don't claim adversarial deep immutability or traceback-frame scrubbing. Reason records primary failure even if closing fails; stream_closed only true when response is actually closed, never assumed. Unknown final usage must not be represented as zero/free.
+
+Preflight before dispatch: reject invalid/open-client state, client.max_retries other than exact integer0 (including bool), non-exact-str or blank model/prompt/instructions, invalid tool name (ASCII [A-Za-z0-9_-]{1,64}), nonexact ToolOutput instance, strict not True, output not a BaseModel class or config extra not forbid. Build schema with GenerateToolJsonSchema and OpenAIJsonSchemaTransformer(strict=True) matching pinned PydanticAI; reject errors. Use existing _request_envelope(prompt,[tool],instructions,4000)<=20000; preserve full strings and reserve actual normalized schema metadata. Nonblank means strip only for checking, never rewrite inputs. No arbitrary extra request kwargs. No client is created or closed. Invalid preflight is reason input, known_usage None/may_have_started False/usage_complete False/stream_closed False.
+
+One await client.responses.create with stream=True, store=False, background=False, parallel_tool_calls=False, model exact, instructions exact, input=prompt exact, reasoning={"effort":"low"}, service_tier="default", max_output_tokens=4000, timeout=60, tools=[normalized strict function with exact name/description/schema], tool_choice={"type":"function","name":tool.name}. Set may_have_started=True immediately before this await (uncertain dispatch is conservative). Wrap creation+consumption in asyncio.timeout(60), a new total operation deadline distinct from historical transport timeouts. Exactly one native HTTP request; SDK retries preflight-disabled. No SDK run loops or second calls. The caller continues to own its AsyncOpenAI/httpx lifetime.
+
+Use raw typed SSE events from the native SDK, before any cumulative argument assembly. Track strictly increasing exact nonnegative int sequence_number, one response.created (matching nonempty response ID) before other events, response.in_progress consistent ID/status if present. Only reasoning and exactly one expected function_call output item permitted. Track unique nonnegative output indexes/item IDs (bool invalid). Function identity includes name/id/call_id/output_index, cannot change; arguments initial string at output_item.added may be nonempty and must be guarded. Reasoning IDs/indexes cannot collide. Reasoning delta/summary events are ignored only for reasoning-related event type prefixes; they still count toward sequence/event limits and cannot introduce function args. Unknown/text/refusal/error/builtin tool events fail closed. Do not keep reasoning text.
+
+For response.function_call_arguments.delta: existing live function item/index must match; exact-str delta feeds JsonWhitespaceGuard before appending/using any of that chunk; a rejected chunk appends no prefix. Maintain one guard for that argument stream, cumulative accepted UTF-8 bytes<=80000. Empty deltas preserve lexical state. At threshold failure stop iterating immediately, close the HTTP response, do not drain future usage/events and return no partial result. Never strip/repair args. Guard any full argument snapshots before inspecting/using them (fresh guard for snapshot), then require exact equality to assembled delta text so done/terminal-only replacement cannot bypass guard. A function arguments.done and output_item.done must each match name/identity/arguments where supplied, occur once, and precede successful completion. A final function item has status completed; reasoning items can be final completed or status omitted as allowed by SDK. Reject extra/duplicate/out-of-order calls/items/deltas, contradictory snapshots or function completion before args.done.
+
+Terminal response.completed/incomplete/failed: response ID must match created ID. First validate/capture valid native usage even if later output/status checks fail. Native usage exact nonnegative integer input_tokens/output_tokens/total_tokens with total=input+output, requests=1, monotonically nondecreasing against any valid earlier observations. Preserve last valid observed counters on invalid/decreasing/missing usage. Observed counters above output4000/total20000 must be retained on reason usage failure, never clipped. usage_complete becomes True ONLY when a matching terminal event has valid usage. It means final usage observed, NOT generation complete. Earlier response.created/in_progress valid usage can be retained but usage_complete False. Missing/invalid terminal usage is failure; no successful free/default usage. Terminal incomplete/failed (and contradictory event/status) never succeeds even with parseable JSON. Completed success requires status completed/no error/incomplete_details, exactly one expected completed function plus permitted reasoning only, full identity/arguments agreement with accumulated stream/done snapshots, and all item completions. EOF without terminal is failure/unknown final usage. A terminal response defines the end: close without draining further events.
+
+On completed success only: parse exact assembled JSON without duplicate keys/nonfinite constants or numbers that overflow to nonfinite Python floats (recursively across lists/objects), require root object, validate requested output strictly, then existing _canonical_models/_actual_models. Invalid complete output retains terminal usage and fails reason output. No partial JSON parse or result before terminal. On protocol/whitespace/usage/limits/transport/deadline/external cancellation close this response in finally, preserving primary error and known usage. Bound cooperative close of the ACQUIRED HTTP response body with asyncio.timeout(STREAM_CLOSE_DEADLINE_SECONDS=1), including native SDK generator-finally close as well as our final cleanup; a narrowly owned AsyncByteStream proxy may wrap that response body before iteration without mutating borrowed client/settings. Use shared confirmed-close state/idempotence, never report closure merely from native is_closed. Pre-acquisition borrowed SDK internals are opaque/cooperative; this is not a hard end-to-end SLA. Bound cooperative finally-close separately with the same close policy; expiry must retain any earlier primary failure, stream_closed False unless closure confirmed, and no false cancellation classification from the cleanup timer. If there was no earlier error use reason timeout. Never leave an unowned background cleanup task; arbitrary cancellation-resistant custom closers cannot be forcibly stopped by Python, and this is not a hard wall-clock SLA. Creation/consumption may take60s plus up to1s cooperative cleanup and scheduling. External CancelledError propagates as GuardedResponsesCancelled, never ordinary success/ValueError. Closing failures must not leak provider messages; prevent a false stream_closed claim. Max10000 decoded events, max80000 accepted argument UTF-8 bytes and60-second operation deadline are explicit safety bounds; they do not cap decoder memory for a single native event or prove server-side stop time/cost. No overall app budget/reservation mutation.
+
+## Task 1: Guarded native stream transport
+
+- [x] Implement only new module/tests+checkbox with RED→GREEN TDD.
+
+Read test-driven-development. Use real AsyncOpenAI with max_retries=0 and httpx.MockTransport + close-observing AsyncByteStream supplying synthetic SSE. No provider/network/private fixtures. Spy wire body to assert all exact options/full prompt/instructions/strict schema/tool choice/one HTTP call; preflight rejects make0HTTPcalls. Successful complete typed output and usage; whole/1char/mixed/empty argument chunks, quoted long whitespace/Unicode unchanged. 256 vs257 outside whitespace; reject added/done/terminal snapshots with excess whitespace/mismatches; no suffix events consumed after guard failure; HTTP stream closed and borrowed client stays open. Valid observed nonterminal usage survives failure but is incomplete; no usage None neverzero; final usage preserved on invalidJSON/incomplete/overbudget; invalid/negative/bool/missing/inconsistent/decreasing usage cases. EOF, unknown event, reordered/wrong IDs/indexes/sequence/call name, second tools/text/refusal/errors, duplicates, missing done markers; schema output strictness/extra/wrongtype/duplicateJSONkeys/nonfinite; event/UTF8 bounds. Timeout/transport close errors and external cancellation tests use deterministic events or patched deadline (no60s waits); cancelled remains BaseException cancellation and carries usage. Sanitized fixed errors/cause, primary error/closed observations; no raw source values in repr/str. Strict focused pytest/scoped Ruff+format/mypy/diff only, no fullQA. Commit only owned3paths, write exact RED/GREEN/static/selfreview evidence in task report. No copied scanner oracle or tests that only mirror implementation.
+
+## Controller gates
+
+- [x] Independent task/wholebranch review C0I0M0·새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·실패/skip/warning0.
+- [x] Old recorded v1/v2 arguments through new mocked native HTTP/SSE path, no provider; preserve470/priorproof/unknown paid usage.
+- [x] Docs/privacy/links·Draft PR106 basePR105·Sitesv128/previousv127·7exact changed documents·ownQA3/volume3/SDD cleanup; no app/worker/main activation.
+
+## 판단과 비용
+
+1. 별도 native streaming transport를 추가하고 기존 SDK/앱은 유지한다. 기존 영수증을 위조하지 않는 대신 앱의 실제 응답은 아직 바뀌지 않으며 후속 생성 SDK·worker 연결 인수가 필요하다.
+2. 공백 감지 시 drain 없이 HTTP 응답을 닫고 관측 사용량/최종 여부를 나눈다. 알려진 비용을 보존하지만 제공자의 실제 중단·최종 과금은 미확인이며 비용 절감을 보장하지 않는다.
+3. 새 경로에60초 전체 deadline·8만 argument UTF-8bytes·1만 event 제한을 명시한다. 무한 소비를 제한하지만 정상적인 긴 실행을 차단할 수 있고 단일 native event의 decoder 메모리는 별도다.
+
+4. 정리 중 외부 취소도 CancelledError 하위 클래스로 전파하되 먼저 발생한 실패 reason을 보존한다. reason만 보는 호출자는 취소를 놓칠 수 있으므로 예외 종류를 먼저 처리해야 한다.
+
+5. 정리에 별도1초 cooperative deadline을 둔다. 정상적인 느린 close도 미확인으로 실패할 수 있으며 Python 취소를 무시하는 임의 closer까지 강제 종료하지는 못한다.
+6. JSON 숫자의 Python float overflow도 거부한다. inf 성공을 막는 대신 큰 수를 정확히 표현할 수 있는 다른 숫자 모델에도 문자열 등 별도 표현이 필요할 수 있다.
+
+공식 API 이벤트 기준: [Responses streaming consumers](https://developers.openai.com/api/docs/guides/migrate-to-responses#7-update-streaming-consumers). 실제 배포된 SDK는 로컬 잠금 버전 코드로 확인하며 현재 제공자 동작·요금은 별도 인수한다.
+
+## 검증 결과
+
+실행1d1d0c7·합성119/새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·실패/skip/warning0, 태스크/전체 검토 C0I0M0다. [검증 기록](../research/guarded-responses-stream-validation.md)과 [Draft PR106](https://github.com/DigestWorks/IssueDigest/pull/106)을 따른다. 실제 제공자 호출0·기존470실행/과거 증거 보존이며 기존 SDK/worker 연결은 후속이다.

docs/planning/implementation-plan.md

@@ -2,6 +2,10 @@
 
 2026-10-20 KST 기준. IssueDigest는 질문에서 시작해 이슈의 맥락·쟁점·변화를 이해하는 B2C 브리핑 서비스다. [D016·D017·D019·D020·D021·D022](decisions.md)과 [제품 경험 인터뷰](../research/product-experience-interview.md)가 제품 방향의 원본이다. 이 문서는 현재 구현과 다음 구현을 구분한다. 두 사람의 공동 자료 검증을 최종 제품 구조로 확대하지 않는다.
 
+## M3 현재 구현 · 스트림 transport와 생성 연결
+
+[새 native 스트림](../changes/guarded-responses-stream.md)에 공백 감지기를 연결해 신규119/새전체9,435개(Python8,615+JS820)·독립 검토 C0I0M0를 통과했다. 모의 HTTP8회로 기존 미완료 인자의 닫힘/사용량을 재생했고 제공자 호출0이다. 관측 사용량과 최종 사용량 수신 여부, 불확실한 송신/응답 닫힘·외부 취소를 구분한다. 신규 경로만60초 operation deadline·acquired body 별도1초 cooperative 정리·8만 argument UTF-8bytes·1만 events로 제한하며 기존 SDK/영수증/앱은 유지한다. 합성 HTTP 검증과 실제 제공자의 중단·과금/생성 품질은 구분한다.
+
 ## M3 현재 구현 · 순수 공백 감지와 다음 통합
 
 [독립 JSON 공백 감지기](../changes/json-whitespace-guard.md)는 신규155/새전체9,316개(Python8,496+JS820)·정적 검사/빌드/OpenAPI·독립 검토 C0I0M0를 통과했다. 기록된 v2 인자는 네 분할에서 같은 위치에 감지했으며 v1 미완료는 그대로 통과했다. 문자열 밖 ASCII 연속 공백256까지만 허용하는 고정 휴리스틱이며 JSON 검증기·실제 스트림 취소가 아니다. 기존468실행 파일과 유료 경로는 유지한다. [compact 지침 1회 진단](../research/question-evidence-stall-investigation.md)은60.106초 타임아웃으로 HTTP 응답/사용량이 없어 효과 미판정·비용 미관측이다. 이후 통합에서는 취소 후 최종 사용량이 없는 상태를 별도로 처리해야 한다.