Repository navigation
feat(extras): add experimental decider and llm operations - #1218
Merged
Merged
Conversation
Add llm(), which returns ask, check, classify, label, and score. Each call is one durable step that returns a typed answer. check, classify, label, and score use a decision model such as Strands Decider when one is configured, and Amazon Bedrock structured output otherwise. ask uses a Bedrock text model. A decision model answer below unsureBelow goes to the text model.
check, classify, label, and score now need a decision model. Without one they reject with a TypeError before any durable operation. The Bedrock structured-output fallback and unsureBelow are removed. The default ask model moves to Claude Haiku 5.5, the newest Haiku on Bedrock. ask sends a temperature only when set, because Haiku 5.5 rejects one. classify accepts at most 255 options and score at most 10 levels, the decision model's limits, and the TSDoc of each operation lists its limits.
check, classify, label, and score use a decision model, not a large language model. So they move to decider(), and llm() keeps only ask. Both factories live in the new @aws/durable-execution-sdk-js-extras/ai subpath, which replaces /llm. Step names become decider.check, decider.classify, decider.label, decider.score, and llm.ask. Subtypes move to AiOperationSubType: DeciderCheck, DeciderClassify, DeciderLabel, DeciderScore, and LlmAsk. Errors extend AiError: DeciderError, DeciderInvalidResponseError, and LlmMaxTokensError. decider() throws when it has no endpoint, transport, or STRANDS_DECIDER_ENDPOINT.
- Retry a request that the invocation deadline stops: Bedrock and decider aborts become AiTimeoutError, which defaultAiRetryStrategy retries. - Fail ask with LlmStoppedError when the reply stops for a reason other than its end, such as a guardrail. The guardrail message is no longer returned as the answer. - Document that the decision model refuses an input over its window with HTTP 422, which is not retried. The server no longer cuts it. - Validate the decider endpoint URL, keep its query and credentials out of checkpointed errors, and do not follow redirects. - Report a non-JSON decider reply or a score outside the scale as DeciderInvalidResponseError. - Escape every case and spacing of the closing input tag in ask. - Keep config values when a per-call override field is undefined. - Name the classify and label lists `choices`, and accept a plain string array as a score scale.
- Refuse a decider endpoint with a user name or password. fetch() rejects such a URL with an error that repeats the password, and the error was checkpointed and retried. Credentials belong in headers. - Name only the cause's type in the "could not be reached" message, because the cause's message can repeat the full URL. The cause stays on the error. - Wrap only the abort itself as AiTimeoutError. A real error that arrives after the deadline, such as HTTP 401, keeps its message and its retry decision. - Retry LlmStoppedError for malformed_model_output and malformed_tool_use, where a new reply can succeed. - Escape every "</input" prefix in the ask input, whatever follows it.
- Stop re-exporting ./ai from the package root. The ai operations are imported from @aws/durable-execution-sdk-js-extras/ai only, as the README shows. Without this, a root import of microvm pulled in the Bedrock client import, and esbuild failed when the optional peer was not installed. - Load @aws-sdk/client-bedrock-runtime through one import() with .catch, so a bundler leaves a missing package to run time. decider() then works without it, and ask fails with an AiError that names the package. - Name the system error code, such as ECONNREFUSED, in the checkpointed "could not be reached" message. Only the message is checkpointed, and the code holds no URL. - Drop the signal.reason check in isAbort. The abort error names already cover it.
ask said "needs the package. Install it" for any import() failure. It now says that only for MODULE_NOT_FOUND and ERR_MODULE_NOT_FOUND, and otherwise says the package could not be loaded, with the original error as the cause. Tests cover both cases and check that decider() still works without the package.
decider({ agentCore: { runtimeArn } }) sends each request through the AWS
SDK's InvokeAgentRuntime, signed with the function's credentials. AgentCore
Runtime forwards it to the server's POST /invocations route, which the
Strands Decider server answers like /v1/systemone.
The call assumes the model is deployed and ready, and does not check it. A
non-2xx status from the server becomes DeciderError with that status, a
reply that is not JSON becomes DeciderInvalidResponseError, AWS SDK errors
reach the retry strategy unchanged, and a deadline abort becomes
AiTimeoutError.
@aws-sdk/client-bedrock-agentcore is a new optional peer dependency, loaded
on first use like the Bedrock Runtime client. decider() takes only one of
agentCore, endpoint, and transport.
Stacked on the parked `feat/extras-llm-operations` branch (#1160). It is not for main until Bedrock supports decider models. ## Why A Strands Decider can run on a server, on SageMaker, on Lambda, or on AgentCore Runtime. The [Strands Decider deploy guides](https://github.com/strands-labs/strands-decider/tree/main/deploy) show that every host takes the same request body. Only the way the call is sent and signed changes. So each host is one transport factory, and `decider()` does not change when a host is added. ## What changes | Host | Use | AWS call | Permission | |---|---|---|---| | Any server | `endpoint` | `fetch` to `/v1/systemone` | none | | SageMaker | `transport: sageMakerTransport({ endpointName })` | `InvokeEndpoint` | `sagemaker:InvokeEndpoint` | | Lambda | `transport: lambdaTransport({ functionName, qualifier })` | `Invoke`, request as the event | `lambda:InvokeFunction` | | AgentCore Runtime | `transport: agentCoreTransport({ runtimeArn })` | `InvokeAgentRuntime` | `bedrock-agentcore:InvokeAgentRuntime` | - `agentCoreTransport` replaces the `agentCore` config key. `decider()` now takes only `endpoint` or `transport`. - `AgentCoreInvokeClient` is renamed to `AwsSdkClient`, because all three transports use it. - `@aws-sdk/client-sagemaker-runtime` and `@aws-sdk/client-lambda` are new optional peer dependencies, loaded on first use. - SageMaker `ModelError` becomes a `DeciderError` with the server status. So a 422 is not retried. - The Lambda function fails the invocation with `ValueError` or `ValidationError` when it refuses a request. That becomes a `DeciderError` with status 422. Any other function error is retried. - `defaultAiRetryStrategy` also retries the Lambda and SageMaker throttling and server errors. Bedrock support will be one more transport when the API exists. ## Testing - `npm test` in the extras package: 515 passed, 1 skipped. New fake-client tests cover each transport, its error mapping and its missing-SDK error. - The built `dist/ai.mjs` keeps all four AWS SDK imports as runtime `import()`. - Not tested against a real SageMaker endpoint or Lambda function.
# Conflicts: # package-lock.json # packages/aws-durable-execution-sdk-js-extras/README.md # packages/aws-durable-execution-sdk-js-extras/package.json # packages/aws-durable-execution-sdk-js-extras/rollup.config.mjs
…o vitest esbuild leaves a missing package to run time only when a .catch() sits directly on its literal import(). The lazySdk helper from #1213 moved the .catch() into the helper. So a customer who bundled /ai with esbuild needed all four AWS SDK clients installed, even with only an endpoint. - Each transport and llm() now attach sdkLoadError() with .catch() directly on their own import(). once() caches the loaded module. - lazyClient() replaces the three copies of the default-client cache. - sdkLoadError() searches the whole cause chain for the missing-module code, because a loader can wrap the error. - bundle.test.ts bundles a copy of src/ai with esbuild in a directory without node_modules. It fails with the #1213 code. - missing-sdk.test.ts checks that each transport creates one default client on first use and reuses it. main moved the package to vitest and to the compute/microvm subpath. So the ai tests now use vitest, the AI reference moved to docs/ai.md as main did for docs/microvm.md, and package-dependencies.test.ts lists the four ai clients as optional peers that load on first use.
This comment has been minimized.
This comment has been minimized.
…fetch replies in ai tests The lockfile from the merge bumped shared AWS SDK and Smithy packages. It now matches main, plus the two new clients at 3.1164.0. On Node.js 22, reading a Response body inside a step of the local test runner with skipTime runs out of memory. So the ai tests return a plain object from the fetch mock.
This comment has been minimized.
This comment has been minimized.
The lookup of getRemainingTimeInMillis ran before the try, so a getter that throws failed the call. It now runs inside the try, as in the core SDK. A new test also covers the ERR_MODULE_NOT_FOUND code.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
yaythomas
requested changes
Nov 13, 2026
Contributor
|
The Python counterpart has already merged. Its corresponding follow-up is tracked in Python SDK #934: fix AI operation execution boundaries. |
Fixes for yaythomas's review. - check and label reject a probability outside 0 to 1, and clamp one within 0.01, as the Python SDK does. - decider() rejects headers together with transport. - decider and llm check the step name (1 to 256 printable ASCII characters) before they create the step. - Each operation serializes its input once and passes that string into the step. - A core SDK whose step context has no signal fails the call with AiError instead of losing cancellation without a sign. - The DeciderTransport docs say a transport must stop when abortSignal aborts.
This comment has been minimized.
This comment has been minimized.
# Conflicts: # package-lock.json # packages/aws-durable-execution-sdk-js-extras/package.json # packages/aws-durable-execution-sdk-js-extras/rollup.config.mjs
This comment has been minimized.
This comment has been minimized.
Codex AI reviewTwo new findings and one existing deferred coverage gap:
Static inspection only; no tests or AWS calls were run. Suppressed repeats
Reviewed commit |
yaythomas
approved these changes
Nov 16, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the parked
/aiwork to main: #1160 (decider()andllm()) and #1213 (SageMaker and Lambda transports), merged with current main, plus fixes for the open review threads on #1213.What it adds
@aws/durable-execution-sdk-js-extras/ai. Every export is marked@experimental.decider().checkdecider.checkDeciderCheckbooleandecider().classifydecider.classifyDeciderClassifydecider().labeldecider.labelDeciderLabeldecider().scoredecider.scoreDeciderScorellm().askllm.askLlmAskendpointtransport: sageMakerTransport({ endpointName })sagemaker:InvokeEndpointtransport: lambdaTransport({ functionName, qualifier })lambda:InvokeFunctiontransport: agentCoreTransport({ runtimeArn })bedrock-agentcore:InvokeAgentRuntimeThe four AWS SDK clients are optional peer dependencies, loaded on first use. Reference:
packages/aws-durable-execution-sdk-js-extras/docs/ai.md.Fixes on top of #1213
.catch()sits directly on the literalimport(). ThelazySdkhelper in feat(extras): add SageMaker and Lambda transports for decider() #1213 moved it into the helper. So bundling/aiwith esbuild failed unless all four clients were installed. Eachimport()now has its own.catch(sdkLoadError(...)).bundle.test.tsbundles a copy ofsrc/aiwhere no client can be resolved. It fails with the feat(extras): add SageMaker and Lambda transports for decider() #1213 code and passes now.lazyClient()replaces the three copies (hln33).docs/ai.md.Changes from merging main
docs/ai.md, as main did fordocs/microvm.md. The README links to it.package-dependencies.test.tslists the four ai clients. It checks that the package root and/aiload without them.Testing
npm test(tsc and vitest): 514 passed, 1 skipped (the live test, which needsAI_LIVE=1).npm run buildandnpm run lintpass.Python counterpart: aws/aws-durable-execution-sdk-python#928.
Design note
Problem. A durable function often needs one typed decision (yes or no, one of N, a score) or one short text answer from a model. Calling a model inside
context.stepby hand repeats the same work in every function: prompt shape, reply parsing, retries, the invocation deadline, and checkpoint size.Cause. The core SDK gives steps, not model calls. No existing library gives model calls that are durable steps on Lambda. Typed-answer libraries are not durable. Durable platforms ship model layers only for their own runtimes.
Options considered.
context.step+ client pattern onlyai()namespace with all operationsdecider()andllm()factories under/ai(chosen)decider()config key per host (agentCore,sageMaker, ...)DeciderConfig.endpointor onetransport(chosen)decider()does not change.Removed before release. The
agentCoreconfig key (nowagentCoreTransport), andAgentCoreInvokeClient(nowAwsSdkClient). Neither was released.Defaults.
llm()uses a pinned Bedrock model ID, moved forward in later releases.decider()has no default host, because no managed host exists yet. It usesSTRANDS_DECIDER_ENDPOINTwhen no option is set. Step names and subtypes are fixed, because a change breaks executions in flight.Experimental. Every export is
@experimental.