Repository navigation
test(lmi): reproduce gaps in held-fault recovery evidence - #199
zhongkechen wants to merge 1 commit into
Conversation
584254d to
43abab3
Compare
|
/ai review |
| if self.scenario == "peer-recovery": | ||
| # Delayed history publication does not extend the ledger watermark. | ||
| self.cloud.recovery_completion(self.items["recovery"]) |
There was a problem hiding this comment.
Codex AI review · Finding arf_v1_55rdb2wjuqyryg4geaulodlnc6
[P2] Wait for native recovery completion before releasing the peer
This wait happens after _deadline() has already released and finished the healthy peer. Cloud.finish(recovery) waits for logical execution success and wrapper return, but does not require the original request's native completion. If that completion occurs after the released peer finishes, the new validator rejects the run even though the harness caused the capacity gap. Move the completion wait into _deadline(), before its final peer-liveness check and peer release. Add a regression where native recovery completion lags logical success and wrapper return.
To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.
| remaining = iter(complete) | ||
| return all( | ||
| any( | ||
| all( | ||
| key not in fragment or same_json(fragment[key], receipt.get(key)) |
There was a problem hiding this comment.
Codex AI review · Finding arf_v1_e22ajchen6d2cppwuohokuomqw
[P2] Match completed receipts independently of array order
The shared iterator makes receipt matching order-dependent. For partial receipts [A, B] from different requests and completed receipts [B, A] with matching identities, searching for A consumes B, which is then incorrectly reported missing. Cross-request array order is not an evidence identity, so this rejects valid cleanup and quarantines the profile solely because a collector reordered records. Use one-to-one identity matching independent of array position, preserving multiplicity, and add a regression with reordered completed receipts.
To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.
Codex AI reviewTwo correctness findings affect recovery sequencing and cleanup receipt matching. Review was static; tests and live LMI validation were not run. Reviewed commit |
The
lmi-lifecycle-4validator accepts two runs that do not establish recovery while the victim’s external fault is held:Both cases pass observation and final validation today. The controls also retain the Python fixture’s own deadline observation: it cannot substitute for the platform timeout or the victim task’s exit. No SDK cancellation signal is assumed.
The fix also needs to reject an empty fault inventory. On #193, clearing both
faultsCreatedandretirement.faultslets lifecycle 4 pass observation and final validation. The draft does not yet include this regression. Add coverage in both phases and require a fault record tied to the victim’s gate that stays held until the selected recovery request succeeds. A nonempty inventory alone cannot establish that link. Review commentRecovery policy
For profiles that run lifecycle 4, require the manifest to declare the expected
recovery_policy(reuse-workerorreplace-worker) and have the CLI reject evidence that reports a different policy. In the lifecycle-4 evidence schema branch, requireprofile.recoveryPolicyto match the semantic validator. Add regression tests for a missing manifest policy, a policy mismatch, and missing evidence policy. Manifest review comment · Schema review commentThis draft follows up on point 3 in #193. It changes only the evidence tests; the fix will come later. Stack: #193 → #195 → #189 → #190 → this draft.
To reproduce both gaps, run from the repository root:
That run produces four failures and seven passing controls. With the four regressions marked as strict expected failures, the full workspace suite had 2,108 passes. Type, format, lint, YAML, and distribution checks passed. These are offline validator tests; no live LMI invocation was run.