Skip to content

test: adopt shared LMI lifecycle suite - #931

Open
zhongkechen wants to merge 54 commits into
mainfrom
feat/python-lmi-shared-suite
Open

zhongkechen wants to merge 54 commits into
mainfrom
feat/python-lmi-shared-suite

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

This adapts the Python SDK to the seven lifecycle requirements and reusable harness in conformance #186, pinned to 41878367192be996cb3414939e15270febedb210.

Design note for the shared producer refactor

Agreed before this refactor: centralize the reusable fixture producer in the conformance stack, remove the unreleased no-coverage fixture mode, and move the deadline/recovery victim execution-status policy into the shared normalizer. Independent Codex and Claude design analyses completed; the maintainer selected the two behavior changes where they differed.

  1. Problem. SDK fixtures must publish invocation-owned evidence and obey case controls exactly as the shared driver reads them. An evidence publication failure must preserve the actual SDK result or exception. Python still owns this producer protocol and reimplements one shared scenario policy.
  2. Cause. The producer was embedded in the SDK fixture. Publication failures are guarded at only some call sites, and the handler classifies completion reporting together with SDK execution. The old fixture mode and unused gate callbacks widen a contract the shared suite no longer uses.
  3. Options. Keep the code local and patch each call site; move only stateless transport helpers; or extract one SDK-independent ledger/control leaf with centralized publication isolation and a minimal pinned staging helper. Moving real SDK hooks, fault loops, complete builds or bootstrap trust into the shared layer would add unnecessary coupling. A background publisher would add timing and retirement work.
  4. Chosen. The narrow leaf in conformance ci(sdk): fix branch parsing for integ framework #189, composed behind the Python Trace facade, with canonical imports and verified staged-byte provenance. Ordinary measurement publication errors become observation errors and source gaps; controls and real SDK errors remain strict. Python retains real public SDK calls, hook mapping, gate timing, probes, replay and build commands. The shared normalizer owns the existing RUNNING/SUCCEEDED/FAILED/TIMED_OUT/STOPPED allowance only for deadline/recovery victims; other roles and scenarios retain SDK expected-outcome callbacks.
  5. Removed. Duplicate Python ledger/control/publication logic, no-coverage mode and its test-only branches, unused on_hold/on_release gate callbacks, the copied coverage-profile list and the Python-specific copy of the shared victim status allowance. Gate still checkpoints None. Native identity and actual SDK boundary timestamps keep their meaning.

The Lambda ZIP will contain the core SDK, its resolved dependencies, fixture/hook modules and only the allowlisted shared runtime leaf plus minimal namespace/legal files. It will exclude the conformance runner, catalogs, YAML/report stack and optional OTel dependencies. The independent pre-install exact-Git-pin guard remains in the caller.

Merge order: #187 → #188 → #189 → #190 → #931. Update every caller pin to the same remotely available final shared revision. Implementation and its validation are in progress; the validation below describes the existing pinned revision until it is replaced with the final results.

Change

.github/workflows/lmi-e2e-tests.yml calls the shared lmi-lifecycle.yml at that exact SHA. The shared workflow owns harness/cloud jobs, AWS credentials and account checks, deployment/test budgets, collection, cleanup, artifacts and summary. .github/scripts/lmi-ci.sh supplies Python setup/build/test commands. GitHub Actions' fixed cloud-job concurrency group remains the sole cross-run guard, covering deployment through cleanup; no S3 shared lock is used.

Reusable AWS transport, deployment, scenario orchestration, evidence normalization, reporting, cleanup and their regression tests moved to #186. Python retains its real public SDK fixtures, lifecycle observation hooks, core wheel/ZIP build, bootstrap and thin runtime bindings. The Lambda bundle contains only the checked-out core SDK, its dependencies and fixture/observation modules. Production SDK behavior is unchanged.

All seven cases are enabled in their default order:

Requirement Scenario Native target Roles
lmi-lifecycle-1 native-overlap c2 victim, peer
lmi-lifecycle-2 normal-return c1 victim
lmi-lifecycle-3 deadline-during-step c1 victim
lmi-lifecycle-4 peer-recovery c2 victim, peer, recovery
lmi-lifecycle-5 suspension-retry c1 victim, resume
lmi-lifecycle-6 warm-reuse c1 victim, warm
lmi-lifecycle-7 normal-failure c1 victim

The adapter reads the shared matrix directly. Identity is run/case based, without cohort_id or cohortId. Probes after both wrapper return and raise preserve actual SDK admission behavior. Case 4 checkpoints the victim's original request ID, admits the peer halfway through the victim timeout, verifies the remaining recovery window and excludes expired originals or service retries from healthy-peer evidence. Retries do not reinject the victim fault. Completion timestamps are captured once in milliseconds and reused without conversion drift.

Decisions not covered by the design note

One line each, for maintainer acceptance (rule 7):

  • Deadline exit waits 200 ms past the runtime deadline. One 200 ms control-poll interval, so the platform's timeout is recorded before the cooperative exit (the shared validator requires timeout ≤ task exit), well inside the 5 s cleanupMs budget.
  • Held fault I/O escapes after 150 s. A bound so a lost control release cannot hold a step forever; the escape is reported as a harness failure, never a pass.
  • Warm probe keeps its invocation alive 500 ms. Gives the original invocation's late callback, woken by an in-process event, time to record late_activity while the warm invocation is still running, which the validator requires.
  • Admission probes are joined for the declared admissionMs (default 5 s). Each probe record is a separate S3 publication and the runtime freezes the environment when the handler returns; the earlier 0.5 s join could truncate the ledger.
  • A failed evidence publication is reported as an observation_error, never as an SDK outcome. The backend's checkpoint or state result, or its error, is returned unchanged.
  • The replay e2e test uses the local runner's real clock (skip_time=False). The default virtual clock skips the fixture's durable wait and dates service events ahead of the wall-clock retirement they are compared against.

Validation

At 4187836, with this branch's head:

  • lmi_tests unit and local e2e: 70 passed, including real public-SDK replay, normal return/failure, warm reuse and recovery instrumentation.
  • New regressions fail on the previous code: six evidence-sink tests for ObservedClient, and each condition of the deadline cooperative exit was checked by removing it from the fixture.
  • hatch run lmi:lint and the lint-commits script over the PR range: passed locally. Pinned checkout verified with verify_checkout.
  • Shared chore(sdk): update supported python version #186 workspace at 4187836: 2,005 tests passed, ruff and mypy clean.
  • Reported at an earlier pin and not re-run at 4187836: scoped mypy, Bash syntax and stage tests, and the Python 3.14 arm64 Lambda bundle build and inspection.

Unverified (rule 6): whole-suite conformance against real LMI. The latest completed cloud run, at the earlier pin 793ecdc, run 43956550294, passed case 1; cases 2 and 4 failed on incomplete evidence, and case 2's unretired owner quarantined the c1 function, so cases 3, 5, 6 and 7 did not run cleanly. That run predates the evidence-publication isolation fix. Existing late-admission behavior remains visible as failure: an asynchronous checkpoint can be accepted after normal completion and a late step can remain pending.

See run instructions and suite ownership. Merge #186 before #931; retain the exact shared revision pin.

Agreed redesign after the shared-producer review

Before further code, independent Codex and Claude designs revisited the new findings under CONTRIBUTING rules 1–9. The maintainer approved placement being owned by the shared original-request environment checks, and retention of the necessary global Hatch skip-install flag with its workspace effects documented.

The fixture placement-admission checkpoint and unused raw placement phases will be removed. This changes an unreleased fixture operation sequence, so a newly built ZIP/configuration identity, new run/execution namespace and independently completed retirement isolate older checkpoint histories. Misplaced healthy peer/recovery work remains case-owned and retired; the victim fault path and public SDK behavior stay unchanged.

Other corrections narrow interruption handling, retain the actual execution ErrorObject in local evidence, report absent durable context as an observation error rather than fake admission probes, remove inactive observer APIs and stale cleanup/invocation branches, and scope PR triggers to relevant main-branch inputs. Real public hook, operation/task, map/parallel/replay coverage remains.

Actual CI-pinned Hatch1.16.5 testing showed removing global skip-install causes root installation to fail with two discovered top-level packages, packages and lmi_tests. The flag will remain; its effect on the default and eight inherited environments is an explicit accepted workspace decision. Detached LMI behavior is independently verified. IO_CONFIG, timing, toolchain and trusted checkout override choices will be documented with tests and unverified native assumptions.

Implementation and final rebased/shared-pin validation are in progress. The older validation section is historical until replaced with exact final-head results; a provisional shared pin will not be published before that commit is remotely available.

Approved evidence protocol design

The source collector currently compares controller, fixture and service timestamps as if they came from one clock. A contiguous ledger can also lose its last event, and an interruption before the preparation checkpoint commits can let a retry inject a fault intended for the original invocation. These can change a conformance verdict or incorrectly return a target to the next case.

The proposed raw protocol revision keeps the existing runner entry points. Each new run pins its SDK artifact and collector revision. Historical evidence stays with its original reader. The shared driver records clock domains, capture order, causal receipts, outstanding work and a positive end-of-stream record. Same-domain timing and bounded cross-domain intervals can establish the manifest budgets, including equality. Ambiguous timing is incomplete. A recorded late transport handoff or fresh post-return body is a behavioral failure. A backend rejection preserves the fact that a handoff occurred. At retirement the driver needs evidence covering every owner through the cutoff. A successful operation, native timeout or generic throttle alone cannot establish that coverage. A target with independently confirmed physical retirement can be reused even when the evidence cannot establish a behavioral budget.

Python will exercise the public SDK, bind early observations after the SDK resolves paginated input, and compare delivered results and caught failures with service history by operation ID. The regression cases include a stored failure returned as a value, an extra nested body with unchanged output, loss of a terminal publication, and a retry on either side of the preparation checkpoint. In a local counterexample the current fixture raised StopIteration with the root on page two and made zero page requests. Actual cloud acceptance of the proposed revision is unverified.

Original request ownership. The maintainer selected driver-owned admission. Before releasing preparation, the shared driver will persist an immutable case-owned record that identifies the launch ARN and selected native request, then verify the stored record. Only the selected request may enter victim work or start its probes. If a retry resumes before the preparation checkpoint commits, it reads the same selection. Ambiguous identity yields incomplete evidence. A producer that has finished can publish a final sequence after accounting for its work. A peer that is still held can acknowledge the observation cutoff. If a hard kill leaves user work unaccounted for, the target stays fenced pending independently recorded physical retirement.

This is the agreed design under contribution rule 8. The shared implementation belongs in conformance #189 and the Python producer belongs here. The mandatory post-closure probes and the existing validator verdict still apply.

zhongkechen and others added 23 commits September 23, 2026 00:24
Add opt-in cloud deployment and lifecycle coverage for Python LMI.
Assert invocation deadlines, early branch cleanup, replay, and worker
recovery; keep issue 741 regressions red until the SDK fix.
The root only coordinates workspace packages. Disable its implicit
editable install so the LMI test directory does not trigger setuptools'
multiple top-level package discovery error when starting examples.

Exercise examples environment initialization in the unprivileged LMI
harness job and verify that it imports the checked-out SDK.
Run the complete cloud matrix on every trusted PR update, including
drafts, and every main push without labels or path filters. Preserve
existing cloud credential restrictions for forks and Dependabot.

Use run-owned resources without a cross-run concurrency group so newer
commits cannot replace another commit's pending cloud job.
Initialize explicit hold/release objects before invoking fixtures and
report missing or forbidden controls as infrastructure failures. Signal
the winner only after the losing step has entered held I/O, and reject
cleanup assertions when that loser exited before the winner was ready.

Poll events by case with bounded parallel reads, correlate checkpoint
holds to their request, and check stale effects before waiting for a
retry on a potentially pinned worker. Cover the observed false positives
and full fixture handshakes with regression tests.
Keep LMI on Python 3.14 with environment concurrency one and two.
Exercise root finally before PENDING, bounded nested progress, concurrent
shared-handler invocations, in-flight return/failure cleanup, and late
operations from abandoned children using Python public APIs.

Document the Java/Python mapping and preserve the newly exposed late-step
scope defect as an explicit issue 741 regression.
Collect each case's own executions during teardown and leave full-run
history/log collection to the final workflow step. Pace history reads
and bound throttle-only retries, including reads through the public
cloud runner, without retrying invocation or callback writes.

Cover pagination, throttling exhaustion, permission errors, and scoped
versus full-run collection with focused tests.
Checkpoint a victim's placement decision before starting fault work.
If a runtime environment rotates between anchor and victim admission,
replace the pair with a bounded retry. Never retry an admitted lifecycle
assertion. Keep the admission decision stable during replay.
Hold a repository-wide concurrency slot through deployment, testing,
and cleanup without increasing shared provider capacity. Retain up to
100 pending cloud jobs with queue: max instead of evicting older jobs.

Keep the harness parallel and document the queue and repository scope.
Deploy one function per native concurrency setting and reuse it for
ordinary and deadline scenarios. Run all 28 cases in one cloud job
without changing provider capacity or losing regression assertions.

Release controls, stop remaining executions, and observe wrapper exit
before reusing each function. Extend fault observation for the shared
60-second timeout and retain a bounded emergency release.
Create the persistent stack once and update the same two functions
after previous durable executions finish. Preserve failed stacks,
code artifacts, and infrastructure after every test run.

Isolate controls and events by run ID, expire only run data, and scope
cleanup to the current run without deleting functions or buckets.
Remove the infrastructure janitor and verify create/update behavior.
Keep strict cloud regressions manual while the harness runs on PRs.
Pin artifact uploads to the immutable revision used on current main.
Invalidate old run metadata before build and deployment preflight,
preserving the build manifest and preventing prior-run cleanup.
@zhongkechen
zhongkechen deleted the feat/python-lmi-shared-suite branch November 12, 2026 22:21
@zhongkechen
zhongkechen restored the feat/python-lmi-shared-suite branch November 12, 2026 22:27
@zhongkechen zhongkechen reopened this Nov 12, 2026
Comment thread lmi_tests/fixture.py Resolved
Comment thread lmi_tests/fixture.py Resolved
Comment thread .github/workflows/lmi-e2e-tests.yml Resolved
@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime November 16, 2026 23:24 — with GitHub Actions Active
Comment thread lmi_tests/fixture.py
Comment on lines +807 to +812
trace.emit(
"WRAPPER_RETURN",
status=result["Status"],
resources=trace.sdk_resources,
time=trace.sdk_completed_at,
)

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Evidence publication now records a source failure without replacing the SDK result or exception. The regression exercises both outcomes.

contents: read

jobs:
lifecycle:

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer agreement is still pending. The design and its limitations are documented for review.

Comment thread lmi_tests/fixture.py Outdated
Comment on lines +265 to +266
if on_release is not None:
on_release(released)

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Removed the unused gate callbacks. The fixture observes the control state directly.

Comment thread lmi_tests/shared_evidence.py Outdated
Comment on lines +20 to +23
elif fixture == "deadline" and scenario in {
"deadline-during-step",
"peer-recovery",
}:

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Added parameterized coverage for deadline-during-step and peer-recovery, including cases where the native timeout and logical execution disposition differ.

@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime November 17, 2026 22:38 — with GitHub Actions Active
Comment thread lmi_tests/fixture.py
Comment on lines +807 to +812
trace.emit(
"WRAPPER_RETURN",
status=result["Status"],
resources=trace.sdk_resources,
time=trace.sdk_completed_at,
)

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Evidence publication now records a source failure without replacing the SDK result or exception. The regression exercises both outcomes.

Comment thread lmi_tests/shared_evidence.py Outdated
Comment on lines +20 to +23
elif fixture == "deadline" and scenario in {
"deadline-during-step",
"peer-recovery",
}:

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Added parameterized coverage for deadline-during-step and peer-recovery, including cases where the native timeout and logical execution disposition differ.

Comment thread lmi_tests/fixture.py Outdated
Comment on lines +265 to +266
if on_release is not None:
on_release(released)

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in 268ade0. Removed the unused gate callbacks. The fixture observes the control state directly.

@github-actions

This comment has been minimized.

A request seal follows both the fixture wrapper and registered work.
Register SDK transport calls until their exit records are published.
Matches against the local shared driver confirm that a completed call
can be sealed and that written late records invalidate the old seal.

The first review found that writing diagnostic evidence could replace
an SDK response. Later reviews reproduced the same failure during
bootstrap, suspension, stored-error replay, and gate exit. One
publication boundary now retains the gap and leaves the SDK result
intact. Control reads and writes remain part of fixture behavior
and still surface their own errors. Allocate sequence numbers at
publication; retain observed times separately. Removing detached
capture objects and unused scope hooks avoids unsupported paths.

Let SDK initialization own pagination. The prefetch implementation
used to throw a nonretryable initial-fetch error before the
SDK could return its FAILED result. Delayed polls retain actual
transport times.  The fixture asserts delivered retry results,
batches, and callback results.

The staggered-overlap test exposed that a first held read could
predate the other worker. Reuse the driver's existing normal gate
release after both live heartbeats. Preserve BLOCKED from the first
read. The driver owns peer placement; an off-target peer uses the
ordinary control instead of extra placement checkpoints. A preview
uses its own new directory so retained run files stay intact.

Local event renumbering rebuilt records and dropped completion
identity and timestamps. The regression failed before the fix because
InvocationCompletedDetails disappeared when the event ID changed.
Terminal StepFailed history used to expose a zero delay. The earlier
test failed on an unexpected NextAttemptDelaySeconds key. The
immediate retry test at b9a7a387 failed on an omitted zero for RETRY
without options. Preserve explicit nonterminal delays, including
the existing SUCCEED compatibility test.

Updated expectations cover SDK-owned bootstrap, exact disposition
of both disposal probes, late raw publication uniqueness, and
publication failure through the actual trace. The old split-capture
and unused scope tests exercised no production path and were replaced
by ordered publication and matched-task observations.

An independent publication channel or stopping a real late SDK call
would change the protocol or the measured behavior. If every write
after an already valid seal is lost, its consumer can still see
the old seal. Tests explicitly show that blind spot. Native process
termination, recovery identity before the preparation acknowledgment,
and the reusable-workflow pin remain separate rollout concerns.

The deployed fixture now requires a case coverage profile and owned
controls. The driver judges peer and recovery placement from actual
observations, with no duplicate fixture decisions. The callback
result is fetched and checked even with Python optimization
enabled. The registered warm worker uses the same nonthrowing
publication boundary.

Native overlap still uses each worker's actual normal release-read
time. The gate contains no precise shared release witness. A delayed
read can follow the other worker's native completion. Backdating
it would create evidence the fixture did not observe. The driver
and fixture will need a shared proof to close that limitation.

During cleanup, resolve the requested run from a recovery journal
before validating its identity. A prior-run manifest previously
caused the CLI to reject the current recovery before reconciliation.
Let the current runner resolve a saved case-output directory when
the operator omits --output. Preserve explicit manifest and directory
options and the published runner's established fallback.

The local callback transport now reads actual SDK service history.
Before this adaptation, the strengthened callback verifier stopped
at the test transport's missing history method. The test uses fresh
native request IDs, keeps service time comparable to wrapper time,
and checks the completed requests against recorded wrapper entries.

Rebuild and collection now stop when an unresolved deployment journal
prevents proving the current verified target. Previously rebuilding
could archive the journal out of the recovery path, and collection
could reach AWS using the prior run's manifest. The cleanup CLI
blocks conflicting journal identity before reconciliation. Completed
artifacts downloaded from CI resolve case manifests relative to their
local manifest; relocated absolute case paths require explicit output
instead of guessing. Unresolved journals retain their original path.

The runtime verifies one canonical run ID before creating its
storage client. Remove pass-through observer calls and the unused
fixture-owned ready-control handshake. The deployed scenarios use
driver releases.

Verify separate cleanup receipts for two deployment states. Recovery
before archival writes a run-keyed receipt while preserving the prior
active report. An active run instead writes to its saved case output.
The old test expected the active destination even before archival.

Direct deploy shares the unresolved-journal guard with build. Collect
requires the exact function-name map that the shared deployer writes
after verifying every target. Cleanup resolves symlinked manifests
before locating adjacent outputs and leaves journal reconciliation
identity to the shared implementation. An older pin can still clean
up completed downloads beside their manifest, and manual recovery
documents the run selector.

The fixture verifies the exact case hash in the control prefix before
opening storage. Matching aliases from a different deployment still
fail identity validation. Broken diagnostic stdout cannot change
a returned SDK result or replace the original SDK exception.

Console diagnostics are best effort independently of S3. After a
permanent pre-seal write failure, avoid another upload retry budget
for each record. The first failure marks the source incomplete and
leaves no valid seal. After sealing, keep attempting late writes
so recovered storage can invalidate the previous seal.

Resolve run and cleanup defaults from the selected manifest, but
find a pending recovery journal adjacent to the supplied path if
its active manifest still points elsewhere. Reject invalid deploy
arguments before the pinned deployer can erase run artifacts.
Preserve the requested run when rendering summaries.

Use the installed runner's case-key function in fixture tests.
Assert release classification on the published record and keep
root-task observation owned by the root scope.

Bound cumulative synchronous publication work before sealing even if
several slow writes eventually succeed. Preserve late invalidation
attempts after a seal. Retain capacity throughout the old pin's
callback rehearsal, including earlier retry suspensions. Distinct
parallel branch outputs detect incorrect reuse of the left result.
Build failure summaries retain diagnostics when no manifest exists.

Count retry and read-back overhead toward the evidence budget, while
preserving healthy writes through a deadline-length heartbeat stream.
Local terminal failure history clears retry deadlines from prior
attempts. Bundle entries give regular files 0644 and executable
files 0755 for Lambda extraction. No published pin or wire changes.

Verify retry-to-failure and retry-to-success history reconstruction.
Both regressions fail against the prior history model because the
terminal step retains its first retry deadline. Audit finished bundle
permissions rather than stopping before packaging. Keep warm reuse
isolated from other test case IDs. Reject duplicated logical fixture
bodies independently of their operation IDs. Partial missing bodies
alone do not establish a duplicate.

The pre-seal budget counts failed and ambiguous write attempts,
while healthy first-attempt writes and post-seal invalidation remain
eligible. Slow successful writes can still consume native time.

Record replay duplicates as existing side-effect assertion counts.
Keep collection watermarks intact so independent retirement can
succeed. Count repeats per map input and scope this oracle to replay.
Remove the unused gate return object and preserve checkpointed None.

Persist the Python replay assertion rows with full evidence
snapshots.  A duplicate observed during partial collection keeps its
actual body count, even when the shared builder has no sideEffects
rows yet.  An observed input outside the fixture map also expects
zero runs.  These verdicts remain independent from source coverage
and retirement.

Provide logical replay expectations before the shared builder saves.
The paired runner applies them to complete and partial evidence
and reports late behavior independently of physical retirement. The
published dependency lacks this policy and retains its per-ID checks.

Verify the imported conformance module comes from the exact pinned
source tree, rather than accepting an older wheel in a nested venv.
Require replayed map results to retain their integer value types.
The shared builder gives the fixture policy validated body
identities.

Validate conformance checkout provenance before deployment touches
persistent resources. Check SDK provenance inside the LMI Hatch
environment before running its tests. Verify that the shared
cleaner stops an execution recorded in a relocated case tree without
rewriting the saved case manifest.

Run the harness provenance assertion inside the test, including a
wrong SDK import that must stop before pytest. Exercise relocated
case-journal cleanup with each supported runner.
@zhongkechen
zhongkechen deployed to ai-pr-review-runtime November 20, 2026 02:08 — with GitHub Actions Active
Comment thread lmi_tests/fixture.py
Comment on lines +588 to +590
def prepare(_):
trace.gate(preparation_gate, preparation=True)
return trace.identity["request"]

This comment was marked as outdated.

Comment on lines +79 to +84
parameters = signature(shared_build_evidence).parameters
options: dict = {}
if "expected_replay_effects" in parameters:
options["expected_replay_effects"] = _EXPECTED_REPLAY_EFFECTS
if "replay_effect_policy" in parameters:
options["replay_effect_policy"] = replay_effects

This comment was marked as outdated.

Comment on lines +1342 to +1345
delay = None
if (
self.operation_update.operation_type == OperationType.STEP
and self.operation_update.step_options
and self.operation_update.action != OperationAction.FAIL

This comment was marked as outdated.

@github-actions

This comment has been minimized.

@zhongkechen
zhongkechen deployed to ai-pr-review-runtime November 20, 2026 02:15 — with GitHub Actions Active
Comment thread lmi_tests/fixture.py
Comment on lines +588 to +590
def prepare(_):
trace.gate(preparation_gate, preparation=True)
return trace.identity["request"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_jxp2j535qepr5q6lj6ckg4revg

[P1] Preserve recovery ownership before preparation commits

If the original invocation is interrupted after preparation is released but before its SUCCEED checkpoint persists, the default at-least-once semantics rerun prepare() on the next invocation. This returns the retry's request ID, admitting that retry into the victim fault and admission probes. The supplied regression interrupts only after preparation commits. Read and verify the immutable driver-owned request selection before admitting preparation, checkpoint that selected identity, and add a regression covering interruption before the preparation checkpoint.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

Comment on lines +79 to +84
parameters = signature(shared_build_evidence).parameters
options: dict = {}
if "expected_replay_effects" in parameters:
options["expected_replay_effects"] = _EXPECTED_REPLAY_EFFECTS
if "replay_effect_policy" in parameters:
options["replay_effect_policy"] = replay_effects

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_tad5hccecyxnngwp33vgugt66z

[P2] Return to one supported evidence contract

Rule 2, Simplify before you add, remains unmet: this negotiates multiple reader APIs even though verify_checkout() permits exactly one revision. Cleanup and summary contain similar compatibility paths without explaining why one fixed interface cannot work. The committed pin also bypasses the new replay policy, and consumer tests conditionally skip unsupported checks. Return to the design phase (rule 8, step 6) and agree on one validated producer/reader contract; binding directly to that contract is the simpler direction.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

Comment on lines +1342 to +1345
delay = None
if (
self.operation_update.operation_type == OperationType.STEP
and self.operation_update.step_options
and self.operation_update.action != OperationAction.FAIL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_a5yghnygi3ci3lqbcdpztaehck

[P2] Include the testing-model fixes in the root-cause account

Rule 5, A bug fix starts with a failing test, requires the description to name each bug's root cause. The description explains fixture and evidence-protocol problems but omits these shipped testing-library fixes: terminal events retaining retry metadata and event-ID reassignment dropping invocation completion details. These independently affect public history results. Return to the design phase (rule 8, step 6) and include their causes and reproducing tests in the agreed scope.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

Three findings: replay ownership (lmi_tests/fixture.py:589), unnecessary compatibility paths (lmi_tests/shared_evidence.py:79), and undocumented testing-model fixes (packages/aws-durable-execution-sdk-python-testing/src/aws_durable_execution_sdk_python_testing/model.py:1342). Tests were not executed; real-LMI acceptance is unverified.

Reviewed commit 5fc72e865278207c5c2153ab91ad29ebe2cc2634. Workflow run

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — 5fc72e86 Deployed Nov 20, 2026 by zhongkechen via ai-pr-review / Codex review / Generate Codex review #3291
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant