Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
384529a
test: add LMI cloud lifecycle regressions
zhongkechen Sep 23, 2026
f93a3c8
fix: skip installing the workspace root
zhongkechen Sep 23, 2026
d8005d2
ci: run LMI tests on PRs and main
zhongkechen Sep 23, 2026
70e24c6
fix: repair LMI control barriers and evidence
zhongkechen Sep 23, 2026
9150c87
test: add Java lifecycle guards on Python 3.14
zhongkechen Sep 23, 2026
3ebcf7d
test: pace LMI execution history reads
zhongkechen Sep 23, 2026
ef13146
test: bound LMI peer placement before admission
zhongkechen Sep 23, 2026
f385a1d
ci: serialize LMI cloud jobs across runs
zhongkechen Sep 23, 2026
31d9a30
test: share LMI functions in one deployment
zhongkechen Sep 23, 2026
7bd33b1
test: retain LMI functions across runs
zhongkechen Sep 23, 2026
9e0564f
Merge main into Python LMI cloud tests
zhongkechen Oct 16, 2026
ab723ef
fix: isolate LMI runs and gate cloud regressions
zhongkechen Oct 16, 2026
1fd8900
chore: merge latest upstream main
Oct 16, 2026
045ed56
Merge branch 'main' into test/python-lmi-e2e
zhongkechen Oct 19, 2026
558892b
Merge commit 'refs/codex/lmi-observability-20261023/main' into codex/…
zhongkechen Oct 23, 2026
d2c5262
test: add LMI observability coverage
zhongkechen Oct 23, 2026
00949ab
Merge branch 'main' into test/python-lmi-e2e
zhongkechen Oct 23, 2026
de4588d
ci: refresh credentials for LMI test phases
zhongkechen Oct 23, 2026
b691585
test: preserve the observed SDK return boundary
zhongkechen Oct 23, 2026
3aeb6a5
Merge branch 'main' into test/python-lmi-e2e
zhongkechen Oct 23, 2026
d31f074
Merge branch 'main' into test/python-lmi-e2e
zhongkechen Oct 26, 2026
2179e01
Merge branch 'main' into test/python-lmi-e2e
zhongkechen Oct 28, 2026
aaaf2b6
docs(lmi): describe shared conformance adapter design
zhongkechen Nov 12, 2026
b2412d5
Merge branch 'main' into feat/python-lmi-shared-suite
zhongkechen Nov 12, 2026
8d1308c
test: adapt Python LMI fixtures to shared conformance
zhongkechen Nov 12, 2026
c6c9db0
ci: run LMI PR tests on CodeBuild
zhongkechen Nov 12, 2026
d835bcc
fix(lmi): consume corrected conformance evidence
zhongkechen Nov 13, 2026
33fc193
test(lmi): own Python-specific regression contracts
zhongkechen Nov 13, 2026
fda7daa
test: adopt shared LMI lifecycle suite
zhongkechen Nov 13, 2026
ecb0cb3
fix: reuse LMI completion timestamp
zhongkechen Nov 13, 2026
2c8162b
fix: handle unacquired LMI run leases
zhongkechen Nov 13, 2026
0996b2e
refactor: use Actions for LMI run concurrency
zhongkechen Nov 13, 2026
ac35090
fix: preserve the original recovery peer
zhongkechen Nov 13, 2026
e148ac8
test: temporarily skip lifecycle case 2
zhongkechen Nov 13, 2026
c0eb43a
fix: record recovery cohort admission
zhongkechen Nov 13, 2026
45d6a43
fix: scope recovery faults to the original request
zhongkechen Nov 13, 2026
a1d43d4
test: prioritize case 7 during diagnostics
zhongkechen Nov 13, 2026
014bb17
fix: observe admission after exceptional exit
zhongkechen Nov 13, 2026
58bf505
test: restore lifecycle case 2
zhongkechen Nov 13, 2026
e428872
refactor: remove redundant cohort ID
zhongkechen Nov 13, 2026
044fbd8
refactor: consume shared LMI workflow and driver
zhongkechen Nov 14, 2026
1b0093e
chore: update shared LMI fixes pin
Nov 16, 2026
ceae5d1
fix(lmi): wait for the declared admission budget
zhongkechen Nov 16, 2026
7fe54bd
Merge branch 'feat/python-lmi-shared-suite' into pr-931
zhongkechen Nov 16, 2026
db4313e
test(lmi): repin the shared conformance revision
zhongkechen Nov 16, 2026
4eeb4ed
test(lmi): use the real clock for replay evidence
zhongkechen Nov 16, 2026
8ac1528
test(lmi): repin the shared conformance revision
zhongkechen Nov 16, 2026
717be40
fix(lmi): isolate evidence publication from SDK calls
zhongkechen Nov 16, 2026
2d458e9
test(lmi): cover the deadline cooperative exit
zhongkechen Nov 16, 2026
5bbf980
test(lmi): repin the shared conformance revision
zhongkechen Nov 16, 2026
7bdd930
Merge branch 'main' into feat/python-lmi-shared-suite
zhongkechen Nov 16, 2026
d08f256
Merge branch 'main' into feat/python-lmi-shared-suite
zhongkechen Nov 17, 2026
268ade0
test(lmi): seal native request evidence
zhongkechen Nov 20, 2026
5fc72e8
Merge commit 'e7c09658bcf95ecab27d0d0c565d11b21dc5883e' into lmi931-l…
zhongkechen Nov 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/scripts/lmi-ci.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
# Python commands for the shared LMI reusable workflow's stage contract.
set -euo pipefail

case "${1:?Expected a workflow stage}" in
setup)
uv pip install hatch==1.16.5
hatch run lmi:shared-setup
;;
harness)
hatch run lmi:lint
hatch run lmi:python - <<'PYTHON'
from pathlib import Path
import aws_durable_execution_sdk_python as sdk

source = Path('packages/aws-durable-execution-sdk-python/src').resolve()
assert Path(sdk.__file__).resolve().is_relative_to(source), sdk.__file__
PYTHON
hatch run lmi:shared-tests --junitxml=lmi_tests/artifacts/unit.xml
;;
build) hatch run lmi:build ;;
deploy)
hatch run lmi:python -m lmi_tests.deploy deploy \
--run-id "$LMI_SHARED_RUN_ID" \
--runtime "$LMI_RUNTIME" --stack-name "$LMI_STACK_NAME"
;;
run) hatch run lmi:shared ;;
collect) hatch run lmi:python -m lmi_tests.deploy collect ;;
cleanup) hatch run lmi:shared-cleanup ;;
summary) hatch run lmi:python -m lmi_tests.summary ;;
*) printf 'Unknown LMI workflow stage: %s\n' "$1" >&2; exit 2 ;;
esac
46 changes: 46 additions & 0 deletions .github/workflows/lmi-e2e-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: LMI e2e tests

on:
push:
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
branches: [main]
paths:
- '.github/workflows/lmi-e2e-tests.yml'
- '.github/scripts/lmi-ci.sh'
- 'lmi_tests/**'
- '!lmi_tests/**/*.md'
- 'packages/aws-durable-execution-sdk-python/src/**'
- 'packages/aws-durable-execution-sdk-python/pyproject.toml'
- 'packages/aws-durable-execution-sdk-python-testing/src/**'
- 'packages/aws-durable-execution-sdk-python-testing/pyproject.toml'
- 'pyproject.toml'
workflow_dispatch:
inputs:
region:
description: Region containing the test LMI capacity provider
default: us-west-2
type: string

permissions:
contents: read

jobs:
lifecycle:
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
Comment thread
zhongkechen marked this conversation as resolved.
Resolved

This comment was marked as outdated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer agreement is still pending. The design and its limitations are documented for review.

permissions:
Comment thread
zhongkechen marked this conversation as resolved.
Resolved
contents: read
id-token: write
uses: aws/aws-durable-execution-conformance-tests/.github/workflows/lmi-lifecycle.yml@41878367192be996cb3414939e15270febedb210
with:
conformance-ref: 41878367192be996cb3414939e15270febedb210
sdk-script: .github/scripts/lmi-ci.sh
runtime-label: Python 3.14
lambda-runtime: python3.14
stack-name: python-lmi-e2e
region: ${{ inputs.region || 'us-west-2' }}
secrets:
TEST_ROLE_ARN: ${{ secrets.TEST_ROLE_ARN }}
TEST_ACCOUNT_ID: ${{ secrets.TEST_ACCOUNT_ID }}
TEST_LAMBDA_EXECUTION_ROLE_ARN: ${{ secrets.TEST_LAMBDA_EXECUTION_ROLE_ARN }}
CAPACITY_PROVIDER_ARN: ${{ secrets.CAPACITY_PROVIDER_ARN }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,5 @@ dist/
**/template.generated.json

.env

/.conformance/
3 changes: 3 additions & 0 deletions lmi_tests/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
artifacts/
build/
__pycache__/
170 changes: 170 additions & 0 deletions lmi_tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Python LMI lifecycle conformance

This PR adapts the Python SDK to the seven shared lifecycle requirements in
[conformance #186](https://github.com/aws/aws-durable-execution-conformance-tests/pull/186).
The exact dependency is pinned in `shared_contract.py` and the workflow. Each
shared requirement declares one scalar `scenario`; the Python adapter reads the
shared matrix directly rather than maintaining a second requirement catalog.

| Requirement | Scenario | Target | Invocation roles |
| --- | --- | --- | --- |
| `lmi-lifecycle-1` | `native-overlap` | c2 | victim, peer |
| `lmi-lifecycle-2` | `normal-return` | c1 | victim |
| `lmi-lifecycle-3` | `deadline-during-step` | c1 | victim |
| `lmi-lifecycle-4` | `peer-recovery` | c2 | victim, peer, recovery |
| `lmi-lifecycle-5` | `suspension-retry` | c1 | victim, resume |
| `lmi-lifecycle-6` | `warm-reuse` | c1 | victim, warm |
| `lmi-lifecycle-7` | `normal-failure` | c1 | victim |

The shared package owns assertions, AWS transport, persistent deployment, scenario
choreography, source normalization, strict reports, scheduling and retirement.
This repository owns the real Python SDK handlers, observation hooks, ZIP build
and thin runtime bindings. The workflow definition also lives in the shared
repository; `.github/workflows/lmi-e2e-tests.yml` calls its pinned
`lmi-lifecycle.yml`, using `.github/scripts/lmi-ci.sh` for Python stage commands.
OTel/exporter tests and the previous Python-specific matrix are deferred.
There is no local conformance profile or validator that replaces a shared rule.

## Local validation

Install Hatch and Python 3.14. Clone the shared repository into `.conformance`
and check out the exact `CONFORMANCE_REVISION` from `shared_contract.py`:

```bash
git clone https://github.com/aws/aws-durable-execution-conformance-tests.git .conformance
git -C .conformance checkout --detach 41878367192be996cb3414939e15270febedb210
hatch run lmi:shared-setup
hatch run lmi:lint
hatch run lmi:shared-tests
hatch run lmi:build
```

`shared-setup` verifies the commit and rejects edited shared package sources.
It installs only the shared core runner. The LMI environment contains the checked
out core SDK and testing library; the Lambda bundle contains the core SDK and
its dependencies plus fixture/observation modules. It does not include the OTel
SDK, an OTel plugin, local catalogs, validators or the shared runner itself.

Python unit tests cover pinned workflow delegation, the Python artifact/manifest
binding, fixture observations and bootstrap. Shared driver regression tests now
live in the conformance repository and cover case ownership, cancellation, source
normalization, driver ordering and cleanup.

A local plan preview defaults to `lmi_tests/artifacts/preview` and requires a
new output directory. Run and cleanup use `lmi_tests/artifacts/shared` by default.

Local e2e tests use the actual public
SDK and its in-process test service, replacing only AWS transport and evidence
storage. They verify checkpoint/replay and preserve genuine admission failures.
These controls do not establish real LMI placement or deployed acceptance.

## Cloud execution

Use a dedicated test account with the configured LMI capacity provider and Lambda
execution role. The workflow validates the account before deployment. The
persistent functions use Python 3.14, arm64, 2048 MiB and native concurrency 1 and
2, with one environment each. SDK artifact and configuration digests are retained
in every manifest and checked before invoking either target.

Run cloud tests through the workflow's `workflow_dispatch` action or a trusted
same-repository PR. Its fixed `lmi-e2e-shared-capacity-provider` concurrency group
serializes cloud jobs across PRs and refs from deployment through final cleanup,
with `cancel-in-progress: false`. GitHub Actions is the only cross-run concurrency
control. Direct CLI calls do not acquire a lock and are not serialized with CI.

`shared plan` writes the seven-case manifest without invoking functions. `shared
run` writes under `lmi_tests/artifacts/shared/standard/`: the manifest, coverage,
JSON/JUnit reports, launch journal and retained case evidence. Each invocation
uses a case-owned control/evidence prefix derived from `runId` and `caseId`.
The case hash uses only `caseId` within the run prefix. Plans, payloads and evidence
have no additional cohort identifier. Generate new run artifacts with this pinned
revision; inspect or clean up older artifacts with their original adapter revision. A service resume is journaled under
the same durable execution; it is not another driver Invoke.

Trusted same-repository pull requests and manual workflow runs exercise the cloud
suite. Fork and Dependabot pull requests run only the unprivileged local harness.
There is no S3 lease, fence token or persistent quarantine record. Existing S3
ownership objects are not read and cannot block a later workflow run.

## Actual observations and failure handling

`fixture.py` uses public durable APIs for the scenarios. Its instrumentation
observer captures actual operation IDs, parents, stored outcomes and user-task
lifetimes. The service-client wrapper forwards every SDK request and response;
it never supplies synthetic checkpoints or changes SDK cancellation flags.
The instrumentation observer is a measurement tool, not a plugin conformance test.

For normal return/failure, the adapter records the actual durable wrapper boundary
and probes the original context after that boundary. The operation probe uses
`DurableContext.step`; the checkpoint probe calls the SDK's
`context.state.create_checkpoint(is_sync=False)` admission path without issuing a
raw Lambda request. SDK closure/revocation errors are recorded as rejection only when no user code
or new SDK request ran. Probes run after both wrapper return and wrapper raise. An accepted checkpoint, another error, or a still-pending operation
remains visible. Probe tasks are tracked separately; a timeout cannot make them
quiescent. Current SDK gaps are failures, not expected passes or exclusions.

The deadline fixture holds real I/O across the runtime deadline and cooperatively
exits after it. A runtime clock alone never supplies a `platform_timeout` event:
that event requires the original request's service `InvocationCompleted` history.
The recovery victim first waits in a checkpointed preparation step. The driver
starts the peer halfway through the victim's native timeout, verifies the original
cohort and the peer's remaining recovery window, then releases preparation to
admit the fault. Preparation checkpoints the original native request ID, so
service retries keep the same durable sequence without reinjecting the victim's
fault or admission probes. Recovery overlap excludes expired original requests
and their retries. The original peer remains held until recovery completes. A still-running
durable execution is collected as such; terminal-state waiting and stops belong
to retirement after assertions. Releasing a victim fault cannot establish the
preceding cleanup or recovery assertion.

The warm fixture creates invocation-owned late work inside a step. A later
invocation must find it in the same process; it observes the old callback using
the original request identity. Failure to reuse that process is a placement or
collection failure, not permission to select a replacement victim.

`shared_evidence.py` supplies Python error expectations and process identity to
the shared source-evidence normalizer.
Service payload wrappers are decoded consistently with checkpoint snapshots;
truncated results are rejected. Replay compares a real resume's initial
checkpoints to the final service history and counts actual user-function entries.
Missing observations are never filled from expected answers.
Each native request writes a final protocol 2 seal after its wrapper and
registered work finish and all captured observations reach S3. A warm owner
can seal after its wrapper returns, while its callback finishes in the reused
process. Lost records, live probes, and a publisher killed at the native
deadline leave the request unsealed. If work still publishes after a seal,
its later sequence invalidates the seal when the driver reads it.

The fixture stops pre-seal publication once retry and read-back work reaches
five cumulative seconds, after the in-flight write and its read-back finish.
Successful first-attempt uploads do not use that budget. Slow successful
uploads can still consume the 60-second native invocation window. Exhausting
the retry budget leaves collection incomplete without replacing the SDK result.
Late writes after a seal still run so they can invalidate it. The published
workflow pin does not enforce terminal seals. Seal enforcement is tested
against the paired runner. The published pin also uses its original per-operation
replay counts. Logical map inputs and late-only replay effects require the
paired runner's policy support.

The shared behavior schema asks for one scenario-owned `task_exit`. Other user
and probe tasks remain in the same evidence as `observed_task_start/exit`, with
their original `sourceKind`. The independent retirement snapshot restores all
original task kinds, so a pending task cannot disappear from the audit.
Coordination barriers retain their complete `controlRecords` journal separately
from injected victim `faultsCreated`; releasing a healthy peer is not victim
fault recovery.

Run `hatch run lmi:shared-cleanup` for manual cleanup of the case manifests.
With the published pin, pass the same `--output` to run and cleanup when
using a custom case output directory.
When recovering an interrupted deployment with a journal-aware conformance
revision, set `LMI_SHARED_RUN_ID` to the run ID recorded in
`lmi_tests/artifacts/deployment-update.json`. Keep the journal in its original
artifact directory until reconciliation completes.
Cleanup releases only the current run's controls and requests stops only for its
identified executions. It validates run, case and function identity before those
requests. It records `NO_CASES`, `CLEANUP_REQUESTED` or `FAILED` in `cleanup.json`;
completed cleanup requests do not claim that every task retired. Assertion and
retirement failures remain in the shared reports. The shared runner can prevent
reuse of an unretired target within the current run, but that state is not stored
as a lock for future workflow runs.
Empty file added lmi_tests/__init__.py
Empty file.
19 changes: 19 additions & 0 deletions lmi_tests/cloud.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
"""Python fixture binding to the shared AWS driver."""

from aws_durable_execution_conformance_tests.lmi.cloud import Cloud as SharedCloud
from lmi_tests.deploy import ARTIFACTS, manager


class Cloud(SharedCloud):
def __init__(
self, manifest, configuration=None, *, artifacts_dir=None, case_context=None
):
super().__init__(
manifest,
configuration,
artifacts_dir=artifacts_dir or ARTIFACTS,
case_context=case_context,
verify_profile=manager().verify,
)
Loading
Loading