Repository navigation
Description
Objective
Complete the agreed SDK changes, resolve ownership and supported-lifecycle decisions, and validate Durable Functions execution and observability on Lambda Managed Instances (LMI). Keep work and telemetry attributable to the correct invocation while preserving checkpoint/replay behavior and healthy concurrent invocations.
Status checked 2026-11-12 using linked issues, PRs, reviews, and existing CI records, with main-branch code review for Java, JavaScript/TypeScript, Python, and shared conformance. The Rust entry summarizes its linked tracker; it is not a new code or deployment audit. Implementation merges, closed issues, local tests, and deployed LMI acceptance are recorded separately below. No new LMI deployment was run for this status review.
The epic has 24 direct sub-issues: 14 open and 10 closed. This includes active implementation and validation work, plus closed lifecycle and documentation tasks. Java #839 and the JS disposal/deadline/guidance issues are directly attached, along with the Rust work below.
Runtime boundaries
LMI environments remain active between invocations, and an invocation timeout does not forcibly terminate the function's code. Already-started work may retain capacity or produce effects after the timeout.
- JavaScript/TypeScript: asynchronous invocations can overlap inside a Node.js worker; continuations and shared resources must retain invocation ownership.
- Java: runtime threads share a JVM and handler object; task lifetime, thread-local scopes, and executor capacity require coordination.
- Python: the runtime uses separate worker processes, each serving one request at a time; blocked invocation-owned threads can still retain a worker.
Native invocation concurrency is distinct from SDK map/parallel concurrency. An invocation is one handler attempt; a durable execution may span multiple invocations. Parent completion, handler result, wrapper return, PENDING, and platform timeout are distinct boundaries. Remaining designs must establish which layer owns a behavior and what guarantee is possible before prescribing a cross-SDK cancellation mechanism.
Remaining implementation and design work
| Area | Tracker | Current scope and implementation status |
|---|---|---|
| JS checkpoint disposal and step-signal cleanup | JS #927; #1101 | Both remain open. Current main still admits checkpoints after disposal and does not abort unawaited step work on normal handler return/failure. #1114 remains open; its current proposal closes checkpoint admission, abandons queued work on disposal, and adds disposal-time step-signal cleanup while preserving in-flight request completion. |
| JS deadline termination and operation admission | JS #927; #1102 | Both remain open. #1202 now proposes deadline-driven termination, cooperative step abort, and operation/serialization admission gates. It reports real-LMI probe results, but is not merged. It does not establish a bound on in-flight checkpoint waiting or queued checkpoint draining. |
| JS telemetry after stop or execution timeout | JS #1127 | The issue remains open. #1202 proposes documentation clarifying that RETRYING does not guarantee a later invocation and that plugins must end remaining operations in onInvocationEnd. Its deadline probe does not validate the separate stop/execution-timeout report. |
| Java deadline admission and cooperative cancellation | Java #838 | #733 remains open. Current main records a deadline but does not enforce it at operation/checkpoint admission. Admission fencing, cancellation, and result/failure correctness belong here; this does not establish hard caller-return or worker-reclamation bounds. |
| Java cleanup bounds and worker recovery | Java #726 | Current main can wait indefinitely for actual task exit or checkpoint transport shutdown. Agree SDK/runtime ownership, achievable budgets, and a supported recovery policy for work that outlives cancellation, then implement that contract while preserving actual owner/task cleanup and owner End ordering. Safe forced termination of arbitrary non-cooperative Java code is not an established SDK guarantee. |
| Java bounded-pool progress and capacity | Java #839 | Shared and nested bounded pools can starve queued descendants. Define supported nesting, admission, and resource limits; per-invocation fixed pools alone do not solve arbitrary nested blocking. |
| Java invocation-owned executors | Java #840 | #735 remains open and stacked on #733. Its proposed opt-in executor isolation is not in main; root and child work still share the configured executor. Broader nested-pool progress stays in #839; hard cleanup and recovery bounds stay in #726. |
| Python lifecycle ownership, deadlines, and recovery | Python #741 | Current main does not enforce the invocation deadline and retains unbounded wrapper/checkpoint waits and branch-pool cleanup. Keep these gaps separate from the unresolved SDK/runtime ownership and worker-recovery contract. #783 remains open, but its review requests closure and reconsideration of the architecture; its partial controller is not an accepted complete resolution. |
| Python new operations under completed/orphaned parents | Python #741 — consolidated scope | #641 is closed as a duplicate of the linked Python tracker. Implementation and validation remain open: current main's guard checks the operation's own ID and does not reliably reject newly created descendants of completed ancestors. The duplicate closure does not mean the bug is fixed. The consolidated scope retains nested map, invoke/callback/wait, NestingType.FLAT, atomic enqueue-race, and normal/ReplayChildren replay requirements. |
| Rust concurrent runtime integration and lifecycle | Rust #145 | Track the supported concurrent entry point/service adapter, invocation deadline and admission behavior, cancellation-safe cleanup, and deployed LMI isolation/recovery evidence. The runtime-specific implementation and acceptance work remains open. |
The JS lifecycle proposal #1014 was closed without merging. Since its replacement decision, #1193 has merged cooperative cancellation through StepContext.signal, including early-completing map/parallel work. #1195 merged the function/plugin/transport guidance and closed #1103.
Keep JS #927 focused on remaining gaps and decisions: disposal/deadline changes in #1114 and #1202, unfinished branches of context.promise.race / context.promise.any, automatic invocation cancellation of SDK-owned checkpoint/get-state requests, and checkpoint draining after the deadline. Application I/O can already use StepContext.signal; that does not automatically cancel the SDK's own transport. #1202 explicitly still waits for in-flight checkpoints and permits queued updates to drain; its default client request timeout is 55 seconds, not a worker-recovery bound. The smaller issues do not yet account for every remaining requirement in JS #927. Before closing JS #927 as superseded, assign each residual behavior to a linked implementation or an explicitly accepted supported limit.
Arbitrary non-cooperative I/O, event-loop-blocking synchronous code, and suspension leaving the original business stack/finally unresolved are documented JS limits. Do not continue to track forced cancellation or artificial stack unwinding as promised SDK capabilities.
For Java, #733 targets main and #735 currently targets the #733 branch. If those proposals proceed, merge #733 before retargeting #735. The task-registry and original harness prerequisites are already incorporated through #782; they are not a pending #728 → #731 merge stack.
Remaining LMI validation and shared conformance
| Tracker | Remaining work and current status |
|---|---|
| Java #727 | The base harness is merged, but lifecycle/recovery and both OTel views still need final deployed evidence with proven same-JVM overlap and genuine runtime X-Ray headers. #823 remains open; its latest LMI cloud check is skipped. Main's cloud workflow is manually triggered, so the requested every-main-push validation is not implemented. The incorporated harness does not establish acceptance. |
| JS #1017 | Lifecycle and deployed OTel/X-Ray validation remain open. #928 was closed without merging. Follow the review decision: design the remaining tests around the final disposal/deadline changes and state what deployed tests prove beyond unit/composed tests before deciding whether to close or rescope this tracker. #1202 reports a real-LMI main/branch deadline probe with signal-aware and signal-ignoring steps; this is partial evidence, not complete JS #1017 acceptance. Same-worker overlap, both OTel views, genuine runtime carriers, and the remaining lifecycle/replay cases still need appropriate evidence. Historical Core 2 results do not validate current Core 3 / OTel 2 artifacts. |
| Python #742 | Runtime-specific lifecycle, worker recovery, replay, and observability acceptance remain open. #743 remains open with review and coverage gaps; its latest LMI cloud check is skipped. The proposal now has 76 cases, but no current full deployed acceptance. The historical 28-case run (16 passed, 12 failed) does not validate the expanded suite or current artifacts. Align recovery assertions with the contract to be agreed in Python #741. |
| Conformance #161 | Implement the optional language-neutral LMI suite, runtime profiles, shared scenario coordinator and versioned evidence, resource leases/quarantine, strict acceptance that fails when applicable mandatory coverage is missing, and SDK adoption with evidence from at least two distinct runtime models. No shared-suite implementation is present in main. Independent SDK harnesses do not replace this work. |
Work already landed
- Java: #782 merged invocation-local plugins, runtime X-Ray capture, the actual-task registry, and handler-owner cleanup/End ordering, incorporating feat: add invocation-scoped task lifecycle foundation #731 and test: add LMI cloud lifecycle and concurrency regressions #728. #765 merged documentation clarifications. #761 and #762 are closed; their remaining deployed LMI observability acceptance is in [Feature]: Add LMI cloud end-to-end tests for concurrency, lifecycle and OpenTelemetry #727.
- JavaScript/TypeScript: #924 merged invocation-local plugin state, runtime X-Ray propagation, and cross-invocation Workflow Insight isolation. #950, #951, and #916 are closed. The separate #955 and #921 proposals were closed in favor of the integrated implementation; they were not merged fixes. Deployed LMI observability acceptance remains in JS #1017.
- JavaScript/TypeScript cancellation and guidance: #1193 merged
StepContext.signalfor invocation termination and non-virtual child-context completion, including early-completingmap/parallelwork. #1195 merged LMI guidance for function, plugin, and custom transport authors, including precise signal/FLAT-context semantics; #1103 and #1196 are closed. These changes landed on November 9. Deadline-driven termination and disposal-time signal cleanup remain in the open PRs above.
Additional closed JS lifecycle tasks are attached: shared-logger invocation attribution #1027 and polling lifetime/stale-timer cleanup #794. Their closed status does not establish acceptance for the final SDK artifacts.
These completed prerequisites do not establish bounded recovery for arbitrary non-cooperative work or complete this epic's LMI acceptance.
Invariants for remaining changes and validation
- Preserve invocation ownership, healthy peers, and caller-owned executors/clients/providers/exporters. Define cooperative boundaries and finite SDK wait/I/O budgets where supported. Do not claim that cancellation, process freezing, or unsafe thread termination can safely stop arbitrary user work or undo accepted external side effects.
- For Java, preserve actual task/owner cleanup and matching owner-thread End ordering. Across SDKs, distinguish logical cancellation, actual exit, caller completion, and restored capacity; an early response or exception is not proof of cleanup or recovery.
- Preserve service-acknowledged terminal outcomes when later cleanup or metadata-pagination deadlines expire. Retain original failure meaning, checkpoint ordering, stored successes/failures, operation identity, and appropriate side-effect counts across replay/retry. Do not fabricate success or
PENDINGfor unacknowledged critical checkpoints. - Preserve each language's suspension model. Java owner cleanup must not be imposed on JS by artificially unwinding a suspended business stack.
- Validate both OTel views, late callbacks, cleanup/flush failures, and shared-provider lifetime. Record expected timeout/export losses without promising lossless or acknowledged export. Preserve Workflow Insight attribution and delivery semantics under overlap.
- Capture invocation-local X-Ray headers at entry, before Java worker dispatch. The runtime carrier is authoritative even when empty, malformed, or unreadable; use ordinary-Lambda fallback only when unavailable. Preserve
Root,Parent, andSampled, custom/legacy extractors, and context restoration without changing shared global carriers.
Epic completion criteria
- Each remaining behavior has an implemented, accepted resolution or an explicitly agreed supported limit with linked follow-up ownership. Record the applicable SDK version and configuration; resolve pending design decisions rather than inferring agreement from a proposed implementation.
- Intended implementation combinations are merged and pass required review, compatibility, and regression checks. Preserve negative controls and fixed-behavior assertions.
- Public API tests cover initial invocation, timeout, early branch completion, in-flight work, normal/failing return, suspension/resume, and retry while retaining the invariants above.
- Final deployed LMI evidence establishes the required native sharing boundary, same-environment overlap, deadline/admission behavior, supported worker-capacity recovery, and sustained task/resource behavior. Record commits/artifacts, deployed configuration, environment/worker/process and invocation identities, lifecycle timing, service histories, logs, JUnit, and telemetry evidence.
- Tests distinguish setup/placement/collection failures from SDK failures, and attempted requests from service acceptance. Emergency releases, unrelated retries, replacement environments, eventual cleanup, skipped checks, or incomplete evidence do not satisfy a recovery bound. Agree test budgets explicitly without presenting them as product SLAs.
- Required OTel/X-Ray and Workflow Insight scenarios pass with the final compatible artifacts, including sampling, replay, overlapping invocations, present-but-empty/malformed/unreadable carriers versus unavailable-carrier fallback, ownership, and failure paths. Label controlled compatibility fixtures separately from genuine runtime cases.
- Shared conformance/profile/coordinator requirements and SDK adoption in conformance #161 are complete, with required runtime-model evidence and remaining per-SDK gaps visible.
- Compatibility/migration guidance, release validation, and release notes are complete. Link final acceptance runs and released core/plugin/provider/layer versions from the relevant trackers.
Activity