diff --git a/README.md b/README.md index 7a3b669..f9f3166 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +현재 구현: [질문별 근거 스트림 연결](docs/changes/guarded-question-evidence.md). 신규33/관련287·새전체9,468개(Python8,648+JS820)·정적 검사/빌드/OpenAPI·독립 검토 C0I0M0를 통과했다. 기존472실행 파일과 직전69개 증거를 보존했으며 제공자 호출0·앱/worker/main 활성화0이다. 실제 모델 의미와 versioned worker/앱 인수는 남는다. + 현재 구현: [중단 가능한 Responses 스트림](docs/changes/guarded-responses-stream.md). 신규119/새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 모의 HTTP/SSE에서 공백 정체 후 닫힘·관측/최종 사용량·외부 취소를 검증했습니다. 이번 제공자 호출0이며 실제 제공자 중단/과금·기존 생성 SDK/앱 연결은 후속입니다. 현재 구현: [JSON 연속 공백 감지기](docs/changes/json-whitespace-guard.md). 신규155/새전체9,316개(Python8,496+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 기록된 v2 인자를 같은 위치에서 감지했지만 실제 요청 취소·앱 연결·비용 절감은 후속입니다. [compact 지침 진단](docs/research/question-evidence-stall-investigation.md)은1회/60.106초 타임아웃·응답/사용량 미관측·재호출0으로 효과 미판정입니다. diff --git a/docs/README.md b/docs/README.md index 000f9a7..14bf041 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,3 +1,5 @@ +현재 구현: [질문별 근거 스트림 연결](changes/guarded-question-evidence.md). 신규33/관련287·새전체9,468개(Python8,648+JS820)·정적 검사/빌드/OpenAPI·독립 검토 C0I0M0를 통과했다. 기존472실행 파일과 직전69개 증거를 보존했으며 제공자 호출0·앱/worker/main 활성화0이다. 실제 모델 의미·합산 예산/권한/복구·앱 인수는 후속이다. Sites v129에 요약을 게시했고 이전v128을 보존했다. + 현재 구현: [중단 가능한 Responses 스트림](changes/guarded-responses-stream.md). 신규119/새전체9,435개(Python8,615+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 모의 HTTP/SSE에서 공백 정체 후 닫힘·관측/최종 사용량·외부 취소를 검증했습니다. 이번 제공자 호출0이며 실제 제공자 중단/과금·기존 생성 SDK/앱 연결은 후속입니다. 현재 구현: [JSON 연속 공백 감지기](changes/json-whitespace-guard.md). 신규155/새전체9,316개(Python8,496+JS820)·정적 검사/빌드/OpenAPI·독립 검토를 통과했습니다. 기록된 v2 인자를 같은 위치에서 감지했지만 실제 요청 취소·앱 연결·비용 절감은 후속입니다. [compact 지침 진단](research/question-evidence-stall-investigation.md)은1회/60.106초 타임아웃·응답/사용량 미관측·재호출0으로 효과 미판정입니다. diff --git a/docs/changes/guarded-question-evidence.md b/docs/changes/guarded-question-evidence.md new file mode 100644 index 0000000..33a12ba --- /dev/null +++ b/docs/changes/guarded-question-evidence.md @@ -0,0 +1,36 @@ +# 질문별 근거 생성의 중단 가능한 스트림 연결 + +2026-10-25. [기존 transport](guarded-responses-stream.md)를 실제 source/질문별 근거 생성 SDK에 연결한다. 기존 v1/v2 SDK와 앱/worker의 기본 경로는 유지한다. 합성 연결 검증과 실제 모델 의미·앱 인수를 구분한다. + +## Global Constraints + +- Only new `packages/agents/src/issuedigest_agents/guarded_question_evidence.py` and its new test file may change execution code. Existing execution files, dependencies, exports, provider instructions/prompt bytes, old paid results and namespaces remain byte-identical. Documentation may change. No actual provider requests, database/server writes, app/worker activation or main merge. +- New async public `run_guarded_question_evidence(*, client: AsyncOpenAI, model: str, source: Source, question: str) -> GuardedQuestionEvidenceGeneration`. Borrow caller client; use existing `question_evidence_generation_v2._preflight` and exact `QUESTION_EVIDENCE_V2_INSTRUCTIONS`, plus one call to `run_guarded_responses_tool`. No retry/repair/fallback, no source shortening, no client creation/close, no sync loop bridge. +- New strict receipt `GuardedQuestionEvidenceGeneration`: profile Literal `guarded-question-evidence-v1`, transport_profile Literal `guarded-responses-tool-v1`, input_format Literal `question-evidence-source-v2`, status Literal `unreviewed`, evidence QuestionEvidencePropositions, prompt_sha256 Sha256, instructions_sha256 Sha256, usage KnownUsage. Receipt requires exact single request, output<=4000 and input+output<=20000. Different profile deliberately does not impersonate historical v2 receipts. It is structural consistency, not semantic/public/access approval. +- New `validate_guarded_question_evidence(source, question, result)` reconstructs and validates fresh evidence with existing proposition validator, original actual-type/raw-required guards, exact current prompt/instructions hashes and budget. Reject unchecked model_construct/coercion, extra/missing/type-corrupt fields, source metadata/text changes, question/hash changes and profile substitutions. Return validated receipt without changing input/source. +- Before dispatch check strict original source, exact question and v2 preflight; retain full source hash. On completed transport require exact expected output class, expected transport profile, usage_complete exactly True, actual valid KnownUsage, budgets, and immutable-source hash before/after evidence build plus saved validation. Build with existing build_question_evidence_propositions. Do not manufacture receipt from partial/incomplete/invalid output. Source mutation during await fails with observed final usage retained. +- Wrapper failures: `GuardedQuestionEvidenceError` subclass of `GuardedResponsesError`, fixed sanitized text inherited, same observations. Local/preflight invalid input reason input / known_usage None / may_have_started False / usage_complete False / stream_closed False. Native GuardedResponsesError copied to wrapper preserving all observations and reason, from None. Native GuardedResponsesCancelled re-raised unchanged as cancellation, never ordinary error. Post-transport structural/source failure reason output, known terminal usage retained, may_have_started True, usage_complete True, stream_closed True (transport success establishes closed body). Do not copy raw exception text/inputs/partial outputs/provider IDs to error fields. Unknown/unexpected ordinary failure conservatively has may_have_started True after dispatch but no invented usage/close confirmation. No exception handler catches BaseException broadly. +- Transport's 60s/1s/80k/10k and exact no-retry client requirements remain unchanged. Closing HTTP is not proof of provider compute cancellation/final cost. No claim of app quality or provider semantics from synthetic output. + +## Task 1: Connect source-specific generation to native stream + +- [x] Implement the two new execution files only plus this checkbox with TDD. + +Read test-driven-development skill. RED tests before implementation, then GREEN and scoped lint/format/mypy. New synthetic tests must exercise real AsyncOpenAI with httpx2.MockTransport and close-observing SSE, exact question/full quotes/instructions/strict tool name/schema/wire options; one complete valid source-attributed proposition receipt + JSON save/revalidate roundtrip; valid empty insufficient response allowed but not semantic success. Assert native incomplete/invalid JSON/whitespace/transport/cancel observations survive wrapper; no partial receipt, no retries/client close; source quote/title/metadata mutation during await rejects retaining final usage; invalid source/question/budget/client/model preflight sends zero requests; tampered saved receipts/raw-required/constructed wrong types rejected. Use existing fixture module restoration pattern so imports do not poison frozen reload tests. Do not duplicate transport's entire event matrix: focus on integration boundaries and assertion of no app activation. Use deterministic events/deadline overrides, no private data/network/provider calls. Commit owned files; full report records RED/GREEN/static commands/results and self-review. Controller runs full QA once and independent task/final review. No subagent spawning by implementer. + +## Controller gates + +- [x] Independent task and whole-branch review C0I0M0, full repository checks. +- [x] Existing472 execution files/private69 prior-evidence integrity, documentation and privacy checks. +- [x] Draft PR107, Sitesv129/previousv128/6exact document snapshots, own QA1/volume1 cleanup. App/worker/provider acceptance remains separate. + +## 판단과 영향 + +1. 원문 검증과 native transport를 새 SDK로 연결하고 기존 v2는 유지한다. 기존 저장 결과·재생을 보존하지만 앱에 적용하려면 versioned worker·예산/권한/복구 인수가 추가로 필요하다. +2. 기존 정확한 지침·질문을 유지해 전달 경로의 효과만 분리한다. 이전의 출력 정체가 여전히 발생할 수 있으며 이번 합성 성공을 실제 모델 품질로 확대하지 않는다. + +## 검증 결과 + +신규33/관련287·새전체9,468개(Python8,648+JS820)·정적 검사/빌드/OpenAPI·독립 검토 C0I0M0를 통과했다. 기존472실행 파일과 직전69개 증거를 보존했으며 제공자 호출0·앱/worker/main 활성화0이다. 원문/title/bill metadata 변경, 불완전/공백/취소, 영수증 변조와 모든 required 필드 누락을 합성으로 확인했다. 초기 RED24개 부재 실패→GREEN24, 추가 한도 초과 사용량 보존 결함1개를 재현·수정한 뒤 최종33개/관련287개를 통과했다. 전체 검사는 실패/skip/warning0이며 실제 provider semantic·worker/app readiness를 인증하지 않는다. + +최종 인계: [Draft PR107](https://github.com/DigestWorks/IssueDigest/pull/107)·Sites v129 게시 성공(이전v128/변경6문서 정확 사본 보존). 사이트 문서 기준은dfd3c93, Site source ebe02c1이며 이후 인계 영수증은 저장소를 따른다. 자체 QA1개/전용 volume1개를 정리했다. 실제 모델 호출0·앱/worker/main 활성화0이며 다음은 실제 모델 의미·versioned worker/합산 예산/권한/복구 인수다. diff --git a/docs/planning/implementation-plan.md b/docs/planning/implementation-plan.md index 4e58c83..5b18f4c 100644 --- a/docs/planning/implementation-plan.md +++ b/docs/planning/implementation-plan.md @@ -1,3 +1,7 @@ +## M3 진행 · 원문 검증과 중단 가능한 생성 연결 + +[별도 async SDK](../changes/guarded-question-evidence.md)로 정확한 source·질문과 native 스트림을 연결하고, 완료 응답만 현재 원문·인용·저장 영수증 검증으로 넘긴다. 기존 v2와 앱 실행은 유지한다. 합성 연결33/관련287·새전체9,468개(Python8,648+JS820)·독립 검토를 통과했으며 실제 모델 의미·versioned worker의 합산 예산/권한/내구 복구·개발 반영은 후속 인수다. + # IssueDigest 구현 계획 2026-10-20 KST 기준. IssueDigest는 질문에서 시작해 이슈의 맥락·쟁점·변화를 이해하는 B2C 브리핑 서비스다. [D016·D017·D019·D020·D021·D022](decisions.md)과 [제품 경험 인터뷰](../research/product-experience-interview.md)가 제품 방향의 원본이다. 이 문서는 현재 구현과 다음 구현을 구분한다. 두 사람의 공동 자료 검증을 최종 제품 구조로 확대하지 않는다. diff --git a/docs/planning/progress.md b/docs/planning/progress.md index 4899a35..29af2cc 100644 --- a/docs/planning/progress.md +++ b/docs/planning/progress.md @@ -1,3 +1,7 @@ +## 현재 구현 · 질문별 근거 생성 스트림 연결 · 2026-10-25 + +[새 SDK 계약](../changes/guarded-question-evidence.md). 신규33/관련287·새전체9,468개(Python8,648+JS820)·정적 검사/빌드/OpenAPI·독립 검토 C0I0M0를 통과했다. 기존472실행 파일과 직전69개 증거를 보존했으며 제공자 호출0·앱/worker/main 활성화0이다. 원문/질문/인용·별도 저장 결과를 재검증하고 실패에도 관측된 비용과 취소를 유지한다. 합성 검증을 실제 모델 의미/제공자 과금·앱 인수로 세지 않는다. 실제 versioned worker/합산 예산/권한/복구 연결이 다음 단계이며 Sites 게시 결과는 아래 최종 인계를 따른다. + ## 최종 인계 · Responses 스트림·사용량 관측 · 2026-10-25 한국 시간 [Draft PR106](https://github.com/DigestWorks/IssueDigest/pull/106)은 PR105 기반·미병합이다. 런타임ab7649a/테스트 격리1d1d0c7·신규119/전체9,435QA(Python8,615+JS820), 정적 검사/빌드/OpenAPI·실패/skip/warning0이다. 초기 태스크의 close/숫자 overflow와 전체 검토의 최초 오류 보존을 수정·재검토했고, 전체 실행에서 드러난 새 테스트 모듈 격리는 순서300개·좁은 검토로 보완했다. 최종 검토 C0I0M0·미해결0, 중단/실패 전체 실행은 성공으로 세지 않고 별도 보존한다. @@ -12,6 +16,8 @@ Sitesv128 public/native succeeded(2026-10-25T10:44:16.362351+00:00), source `9fe 기록된 v1/v2 인자 각각 네 분할을 새 transport로 재생했다. v2는 공백 감지 후 terminal까지 읽지 않고 HTTP를 닫아 known_usage None/usage_complete False를 유지한다. v1 미완료는 관측된 최종 사용량을 보존하고 실패한다. 모의 HTTP8회/제공자 호출0이며 실제 과금 절감·계산 취소·의미 성공의 증거가 아니다. 생성 SDK/worker 연결·실제 완료/의미·앱 인수는 후속이고 앱/worker/main 활성화0이다. 자체QA3개/전용 익명볼륨3개를 정리하고25590포트 닫힘을 확인했다. Sites/현재SDD 정리를 마무리한다. +최종 인계: [Draft PR107](https://github.com/DigestWorks/IssueDigest/pull/107)·Sites v129 게시 성공(이전v128/변경6문서 정확 사본 보존). 사이트 문서 기준은dfd3c93, Site source ebe02c1이며 이후 인계 영수증은 저장소를 따른다. 자체 QA1개/전용 volume1개를 정리했다. 실제 모델 호출0·앱/worker/main 활성화0이며 다음은 실제 모델 의미·versioned worker/합산 예산/권한/복구 인수다. + ## 진행 · 중단 가능한 Responses 스트림 · 2026-10-25 한국 시간 [고정 계약](../changes/guarded-responses-stream.md)으로 실제 native SDK의 스트리밍 소비·공백 감지 후 HTTP 닫힘·관측/최종 사용량·취소 전파를 별도 구현한다. 기존470실행/이전 증거를 고정하고 합성 TDD·독립 검토·새 전체QA·기록된 인자 재생을 진행한다. 이번 제공자 호출0이며 기존 사용량 미관측을0/무료로 바꾸지 않는다. 앱/worker/main 활성화와 실제 제공자 중단·과금/의미 인수는 별도다. diff --git a/docs/planning/tasks.md b/docs/planning/tasks.md index 5fe16a8..28fc5c5 100644 --- a/docs/planning/tasks.md +++ b/docs/planning/tasks.md @@ -1,3 +1,11 @@ +## 진행 · 질문별 근거 생성 스트림 연결 · 2026-10-25 + +- [x] [정확한 입력·원문 재검증·관측 비용 계약](../changes/guarded-question-evidence.md) 고정. +- [x] 새 async 생성 SDK/영수증·실제 native MockTransport 연결 TDD33/관련287. +- [x] 독립 태스크/전체 검토 C0I0M0·전체9,468개(Python8,648+JS820)·기존472실행/69직전증거 보존. +- [x] 문서·Draft PR107·Sitesv129/이전v128/변경6문서 정확 사본·자체QA/전용volume 정리. +- [ ] 실제 제공자 완료/의미·versioned worker/예산/권한/복구·앱 인수. + ## 진행 · 중단 가능한 Responses 스트림 · 2026-10-25 한국 시간 - [x] [native transport 계약](../changes/guarded-responses-stream.md)·관측/최종 사용량·취소/응답 닫힘·명시적 한도 고정. diff --git a/packages/agents/src/issuedigest_agents/guarded_question_evidence.py b/packages/agents/src/issuedigest_agents/guarded_question_evidence.py new file mode 100644 index 0000000..123a3a4 --- /dev/null +++ b/packages/agents/src/issuedigest_agents/guarded_question_evidence.py @@ -0,0 +1,165 @@ +"""Borrowed native transport for unreviewed source-specific evidence receipts. + +Structural consistency does not establish meaning, access approval, or provider cancellation. +""" + +from hashlib import sha256 +from typing import Any, Literal, Self + +from issuedigest_domain.sources import Sha256 +from openai import AsyncOpenAI +from pydantic import model_validator + +from .guarded_responses_stream import ( + GUARDED_RESPONSES_PROFILE, + GuardedResponsesError, + GuardedResponsesToolResult, + run_guarded_responses_tool, +) +from .numeric_evidence import KnownUsage, Source, _StrictModel +from .numeric_selection_evaluation import _read +from .question_evidence_generation_v2 import QUESTION_EVIDENCE_V2_INSTRUCTIONS, _preflight +from .question_evidence_propositions import ( + QuestionEvidenceInput, + QuestionEvidencePropositions, + _guard_actual, + _original, + _source, + build_question_evidence_propositions, + validate_question_evidence_propositions, +) + + +class GuardedQuestionEvidenceError(GuardedResponsesError): + """Sanitized failure preserving native observations without partial evidence.""" + + +class GuardedQuestionEvidenceGeneration(_StrictModel): + profile: Literal["guarded-question-evidence-v1"] + transport_profile: Literal["guarded-responses-tool-v1"] + input_format: Literal["question-evidence-source-v2"] + status: Literal["unreviewed"] + evidence: QuestionEvidencePropositions + prompt_sha256: Sha256 + instructions_sha256: Sha256 + usage: KnownUsage + + @model_validator(mode="after") + def bounded_usage(self) -> Self: + _budget(self.usage) + return self + + +def _budget(usage: KnownUsage) -> None: + _guard_actual(usage, KnownUsage) + KnownUsage.model_validate(usage.__dict__, strict=True) + if ( + usage.requests != 1 + or usage.output_tokens > 4000 + or usage.input_tokens + usage.output_tokens > 20000 + ): + raise ValueError("Invalid question evidence usage") + + +def _hash(value: str) -> str: + return sha256(value.encode("utf-8")).hexdigest() + + +def validate_guarded_question_evidence( + source: Source, + question: str, + result: GuardedQuestionEvidenceGeneration | dict[str, Any], +) -> GuardedQuestionEvidenceGeneration: + """Rebuild fresh evidence and receipt from original actual types and current source.""" + try: + _guard_actual(source, Source) + _original(result, GuardedQuestionEvidenceGeneration) + prompt, _ = _preflight(source, question) + checked = _read(GuardedQuestionEvidenceGeneration, result) + evidence = validate_question_evidence_propositions(source, question, checked.evidence) + _budget(checked.usage) + if ( + evidence != checked.evidence + or checked.prompt_sha256 != _hash(prompt) + or checked.instructions_sha256 != _hash(QUESTION_EVIDENCE_V2_INSTRUCTIONS) + ): + raise ValueError("Changed question evidence receipt") + return checked + except Exception: + raise GuardedQuestionEvidenceError(reason="input") from None + + +async def run_guarded_question_evidence( + *, client: AsyncOpenAI, model: str, source: Source, question: str +) -> GuardedQuestionEvidenceGeneration: + """Make one native request; caller owns the client and all application acceptance.""" + try: + _guard_actual(source, Source) + prompt, tools = _preflight(source, question) + source_hash, _ = _source(source) + except Exception: + raise GuardedQuestionEvidenceError(reason="input") from None + terminal = False + known = None + try: + result = await run_guarded_responses_tool( + client=client, + model=model, + prompt=prompt, + instructions=QUESTION_EVIDENCE_V2_INSTRUCTIONS, + tool=tools[0], + ) + if ( + type(result) is not GuardedResponsesToolResult + or type(result.profile) is not str + or result.profile != GUARDED_RESPONSES_PROFILE + or result.usage_complete is not True + ): + raise ValueError("Invalid native completion") + _guard_actual(result.usage, KnownUsage) + KnownUsage.model_validate(result.usage.__dict__, strict=True) + known, terminal = result.usage, True + _budget(result.usage) + if type(result.output) is not QuestionEvidenceInput or _source(source)[0] != source_hash: + raise ValueError("Changed source or output") + evidence = build_question_evidence_propositions(source, question, result.output) + if evidence.source_sha256 != source_hash or _source(source)[0] != source_hash: + raise ValueError("Changed source during evidence construction") + receipt = GuardedQuestionEvidenceGeneration( + profile="guarded-question-evidence-v1", + transport_profile="guarded-responses-tool-v1", + input_format="question-evidence-source-v2", + status="unreviewed", + evidence=evidence, + prompt_sha256=_hash(prompt), + instructions_sha256=_hash(QUESTION_EVIDENCE_V2_INSTRUCTIONS), + usage=known, + ) + checked = validate_guarded_question_evidence(source, question, receipt) + if _source(source)[0] != source_hash: + raise ValueError("Changed source during receipt validation") + return checked + except GuardedResponsesError as error: + if not terminal: + raise GuardedQuestionEvidenceError( + reason=error.reason, + known_usage=error.known_usage, + may_have_started=error.may_have_started, + usage_complete=error.usage_complete, + stream_closed=error.stream_closed, + ) from None + raise GuardedQuestionEvidenceError( + reason="output", + known_usage=known, + may_have_started=True, + usage_complete=True, + stream_closed=True, + ) from None + except Exception: + raise GuardedQuestionEvidenceError( + reason="output", + known_usage=known, + may_have_started=True, + usage_complete=terminal, + stream_closed=terminal, + ) from None diff --git a/packages/agents/tests/test_guarded_question_evidence.py b/packages/agents/tests/test_guarded_question_evidence.py new file mode 100644 index 0000000..7195a00 --- /dev/null +++ b/packages/agents/tests/test_guarded_question_evidence.py @@ -0,0 +1,335 @@ +"""Synthetic source-specific native SDK integration; no app or provider activation.""" + +import asyncio +import importlib +import importlib.util +import json +from copy import deepcopy +from hashlib import sha256 + +import httpx2 as httpx +import pytest +from openai import AsyncOpenAI +from pydantic_ai.profiles.openai import OpenAIJsonSchemaTransformer +from pydantic_ai.tools import GenerateToolJsonSchema +from test_guarded_responses_stream import Body, events +from test_guarded_responses_stream import ( + restore_introduced_agent_modules as restore_introduced_agent_modules, +) +from test_numeric_evidence import document +from test_question_evidence_propositions import QUESTION, TEXT, packet + +EMPTY = dict( + propositions=[], coverage=dict(linked_proposition_ids=[], unresolved_reason="insufficient") +) +USAGE = dict(input_tokens=10, output_tokens=5, requests=1) + + +def api(): + name = "issuedigest_agents.guarded_question_evidence" + assert importlib.util.find_spec(name) is not None, "Missing guarded question SDK" + return importlib.import_module(name) + + +def rows(data): + result = events(json.dumps(data)) + for row in result: + for item in [row.get("item"), *row.get("response", {}).get("output", [])]: + if item is not None: + item["name"] = "fill_question_evidence_v2" + return result + + +async def exercise(src, data=None, *, stream=None, mutate=None, cancel=False, options=None): + mod = api() + body = stream or Body(rows(packet(src) if data is None else data)) + requests = [] + + def serve(request): + requests.append(request) + if mutate: + mutate(src) + return httpx.Response(200, headers={"content-type": "text/event-stream"}, stream=body) + + http = httpx.AsyncClient(transport=httpx.MockTransport(serve)) + client = AsyncOpenAI(api_key="synthetic", http_client=http, max_retries=0) + kwargs = dict(client=client, model=" model ", source=src, question=QUESTION) + kwargs.update(options or {}) + try: + try: + if cancel: + task = asyncio.create_task(mod.run_guarded_question_evidence(**kwargs)) + await body.blocked.wait() + task.cancel() + got = await task + else: + got = await mod.run_guarded_question_evidence(**kwargs) + except (Exception, asyncio.CancelledError) as error: + got = error + assert not client.is_closed() and not http.is_closed + return got, body, requests + finally: + await client.close() + + +def run(src, data=None, **kwargs): + return asyncio.run(exercise(src, data, **kwargs)) + + +def failure(got, reason, *, known=None, started=True, complete=False, closed=True): + assert isinstance(got, api().GuardedQuestionEvidenceError) + assert str(got) == "Guarded Responses request failed" + assert got.reason == reason + assert got.known_usage.model_dump() == known if known else got.known_usage is None + assert (got.may_have_started, got.usage_complete, got.stream_closed) == ( + started, + complete, + closed, + ) + assert got.__cause__ is None and got.__suppress_context__ + + +def test_complete_receipt_exact_wire_and_fresh_json_roundtrip(): + mod, src = api(), document(TEXT, " e\u0301\nwhole extra quote ") + got, body, requests = run(src) + assert type(got) is mod.GuardedQuestionEvidenceGeneration + assert got.profile == "guarded-question-evidence-v1" + assert got.transport_profile == "guarded-responses-tool-v1" + assert got.status == got.evidence.status == "unreviewed" + assert got.evidence.model_dump(mode="json")["propositions"] == packet(src)["propositions"] + assert got.usage.model_dump() == USAGE + assert body.closed and len(requests) == 1 + wire = json.loads(requests[0].content) + v2 = importlib.import_module("issuedigest_agents.question_evidence_generation_v2") + prompt, tools = v2._preflight(src, QUESTION) + schema = tools[0].output.model_json_schema(schema_generator=GenerateToolJsonSchema) + assert wire == dict( + model=" model ", + input=prompt, + instructions=v2.QUESTION_EVIDENCE_V2_INSTRUCTIONS, + stream=True, + store=False, + background=False, + parallel_tool_calls=False, + reasoning={"effort": "low"}, + service_tier="default", + max_output_tokens=4000, + tool_choice={"type": "function", "name": "fill_question_evidence_v2"}, + tools=[ + dict( + type="function", + name="fill_question_evidence_v2", + strict=True, + parameters=OpenAIJsonSchemaTransformer(schema, strict=True).walk(), + ) + ], + ) + assert json.loads(prompt)["question"] == QUESTION + assert [q["text"] for q in json.loads(prompt)["quotes"]] == [ + TEXT, + " e\u0301\nwhole extra quote ", + ] + assert got.prompt_sha256 == sha256(prompt.encode()).hexdigest() + for saved in (got, json.loads(got.model_dump_json())): + checked = mod.validate_guarded_question_evidence(src, QUESTION, saved) + assert checked == got and checked is not got and checked.evidence is not got.evidence + + +def test_empty_insufficient_is_structural_only(): + got, _, _ = run(document(TEXT), EMPTY) + assert got.evidence.propositions == () + assert got.evidence.coverage.unresolved_reason == "insufficient" + assert got.status == "unreviewed" + + +@pytest.mark.parametrize("kind", ["incomplete", "json", "whitespace", "transport", "cancel"]) +def test_native_failures_retain_observations(kind): + src = document(TEXT) + stream = rows(EMPTY) + known, complete, reason = None, False, kind + if kind == "incomplete": + stream[-1]["type"] = "response.incomplete" + stream[-1]["response"]["status"] = "incomplete" + known, complete, reason = USAGE, True, "protocol" + elif kind == "json": + stream = rows(EMPTY) + for row in stream: + if "delta" in row: + row["delta"] = "{" + if "arguments" in row: + row["arguments"] = "{" + for item in [row.get("item"), *row.get("response", {}).get("output", [])]: + if item and item["status"] == "completed": + item["arguments"] = "{" + known, complete, reason = USAGE, True, "output" + elif kind == "whitespace": + stream[2]["delta"] = " " * 300 + elif kind in ("transport", "cancel"): + stream = stream[:2] + body = Body(stream, read_error=kind == "transport", wait=kind == "cancel") + got, body, requests = run(src, stream=body, cancel=kind == "cancel") + assert body.closed and len(requests) == 1 + if kind == "cancel": + native = importlib.import_module("issuedigest_agents.guarded_responses_stream") + assert type(got) is native.GuardedResponsesCancelled + assert got.reason == "cancelled" and got.stream_closed + else: + failure(got, reason, known=known, complete=complete) + + +@pytest.mark.parametrize("field", ["quote", "title", "bill_number"]) +def test_source_mutation_during_await_rejects_with_terminal_usage(field): + src = document(TEXT) + + def mutate(source): + target = source[1].citations[0].selector if field == "quote" else source[2] + object.__setattr__(target, field, "changed source") + + got, _, requests = run(src, mutate=mutate) + failure(got, "output", known=USAGE, complete=True) + assert len(requests) == 1 + + +@pytest.mark.parametrize("kind", ["source", "question", "budget", "client", "model"]) +def test_invalid_preflight_sends_zero_requests(kind): + src, options = document(TEXT), {} + if kind == "source": + src = list(src) + elif kind == "question": + options["question"] = True + elif kind == "budget": + src = document("x" * 22000) + elif kind == "client": + options["client"] = object() + else: + options["model"] = " " + got, _, requests = run(src, EMPTY, options=options) + failure(got, "input", started=False, closed=False) + assert requests == [] + + +@pytest.mark.parametrize( + "kind", + [ + "profile", + "transport", + "hash", + "missing", + "extra", + "usage", + "constructed", + "question", + "source", + ], +) +def test_saved_tampering_and_original_types_reject(kind): + mod, src = api(), document(TEXT) + got, _, _ = run(src) + raw = deepcopy(got.model_dump(mode="json")) + question = QUESTION + if kind == "profile": + raw["profile"] = "question-evidence-generation-v2" + elif kind == "transport": + raw["transport_profile"] = "other" + elif kind == "hash": + raw["prompt_sha256"] = "0" * 64 + elif kind == "missing": + del raw["evidence"]["coverage"]["unresolved_reason"] + elif kind == "extra": + raw["extra"] = "secret" + elif kind == "usage": + raw["usage"]["requests"] = True + elif kind == "constructed": + raw = got.model_copy(update={"usage": got.usage.model_dump()}) + elif kind == "question": + question += " " + else: + object.__setattr__(src[2], "title", "changed metadata") + with pytest.raises(mod.GuardedQuestionEvidenceError): + mod.validate_guarded_question_evidence(src, question, raw) + + +@pytest.mark.parametrize( + "kind", ["output", "profile", "complete", "usage_type", "usage_budget", "unknown"] +) +def test_unchecked_native_boundary_never_manufactures_receipt(monkeypatch, kind): + mod, src = api(), document(TEXT) + native = importlib.import_module("issuedigest_agents.guarded_responses_stream") + contracts = importlib.import_module("issuedigest_agents.numeric_evidence") + usage = contracts.KnownUsage(**USAGE) + output = mod.QuestionEvidenceInput.model_validate_json(json.dumps(packet(src)), strict=True) + if kind == "output": + output = output.model_dump() + if kind == "usage_type": + usage = usage.model_copy(update={"requests": True}) + if kind == "usage_budget": + usage = usage.model_copy(update={"output_tokens": 4001}) + result = native.GuardedResponsesToolResult( + "wrong" if kind == "profile" else native.GUARDED_RESPONSES_PROFILE, + output, + usage, + 1 if kind == "complete" else True, + ) + + async def forged(**kwargs): + if kind == "unknown": + raise RuntimeError("sensitive raw detail") + return result + + monkeypatch.setattr(mod, "run_guarded_responses_tool", forged) + got, _, requests = run(src) + known = usage.model_dump() if kind in ("output", "usage_budget") else None + failure( + got, + "output", + known=known, + complete=kind in ("output", "usage_budget"), + closed=kind in ("output", "usage_budget"), + ) + assert requests == [] + + +def test_cancellation_is_reraised_by_identity_and_no_app_imports(monkeypatch): + import sys + + mod, src = api(), document(TEXT) + native = importlib.import_module("issuedigest_agents.guarded_responses_stream") + original = native.GuardedResponsesCancelled(reason="cancelled", may_have_started=True) + before = set(sys.modules) + + async def cancelled(**kwargs): + raise original + + monkeypatch.setattr(mod, "run_guarded_responses_tool", cancelled) + got, _, _ = run(src) + assert got is original + introduced = set(sys.modules) - before + assert not any( + name.startswith(("issuedigest_api", "issuedigest_worker", "dbos")) for name in introduced + ) + + +@pytest.mark.parametrize("actual", [False, True]) +def test_every_required_saved_field_is_required_even_null_empty(actual): + from pydantic import BaseModel + + mod, src = api(), document(TEXT) + got, _, _ = run(src) + + def paths(value, prefix=()): + if isinstance(value, dict): + for key, child in value.items(): + yield (*prefix, key) + yield from paths(child, (*prefix, key)) + elif isinstance(value, list): + for index, child in enumerate(value): + yield from paths(child, (*prefix, index)) + + for path in paths(got.model_dump(mode="json")): + changed = deepcopy(got) if actual else got.model_dump(mode="json") + parent = changed + for key in path[:-1]: + parent = getattr(parent, key) if isinstance(parent, BaseModel) else parent[key] + del (parent.__dict__ if isinstance(parent, BaseModel) else parent)[path[-1]] + with pytest.raises(mod.GuardedQuestionEvidenceError): + mod.validate_guarded_question_evidence(src, QUESTION, changed)