Repository navigation
fix(agent-runner): reconcile an environment's LLM env (key, base URL, model) on every start - #653
Conversation
A static environment bakes the creating agent's GOOSE_MODEL, model API key and OPENAI_HOST into its Pod and, apart from GOOSE_MODEL, never touches them again. A rotated key or an edited model therefore never reaches the Pod: it keeps calling the provider with a credential the agent row no longer holds, and every attach dies with ACP "session/prompt: Authentication required". Add GOOSE_MODEL's two siblings to the keys ensureEnvironmentInfraEnv already reconciles on every StartEnvironment — OPENAI_HOST and the provider's own key var, whose name varies by provider and so comes in as the new EnvironmentConfig.LLMKeyEnvVar from the executor. Still gated on a matching GOOSE_PROVIDER: a model id, base URL or key var belonging to another provider's agent is meaningless to the one the Pod is frozen to, so that case stays as it was.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
ℹ️ No blocking issues — a couple of suggestions inline.
Reviewed changes
EnvironmentConfig.LLMKeyEnvVar— new field naming the provider's model-key env var, since the name varies by provider and can't be inferred fromEnvalone.infraEnvKeysFor(docker + k8s) — appendsOPENAI_HOSTandLLMKeyEnvVarto the reconciled keys when the frozenGOOSE_PROVIDERmatches, so a rotated key, moved base URL or edited model follows the attaching agent instead of staying frozen.coldStartEnvironment— populatesLLMKeyEnvVarfromresolveProviderEnv, empty forprovider_cli.- Tests — k8s coverage for same-provider refresh and different-provider preservation.
ℹ️ Per-agent key reconciliation restarts a shared environment on alternating attaches
Because the model key now follows each attaching agent, an environment attached by two agents that share GOOSE_PROVIDER but hold different keys will patch its Pod template (k8s) or recreate its container (docker) on every alternating attach, restarting whatever it was serving. This extends the existing GOOSE_MODEL behavior rather than introducing a new mechanism, and the PR already calls out the different-provider case — worth confirming that same-provider, different-key environments are intended to serve only one agent at a time.
Technical details
# Per-agent key reconciliation across attaches
## Affected sites
- services/agent-runner/internal/sandbox/docker/environment.go:573 — key/OPENAI_HOST added to reconcile keys
- services/agent-runner/internal/sandbox/k8s/environment.go:599 — same
- services/agent-runner/internal/sandbox/docker/environment.go:612 — stale env triggers a full container recreate
- services/agent-runner/internal/sandbox/k8s/environment.go:692 — stale env patches the pod template (rolling restart)
## Open questions for the human
- Are same-provider, different-key agents ever expected to attach to one environment? If not, no action needed. If yes, the key can't be a per-attach reconciled value without thrash; it would need to be part of the frozen identity (like `GOOSE_PROVIDER`/`PACA_AGENT_ID`).deepseek-v4.1-flash (free via Pullfrog for OSS) ｜ 𝕏
| // The model key's env var NAME varies by provider, so a backend cannot | ||
| // infer it from Env alone — see EnvironmentConfig.LLMKeyEnvVar. | ||
| keyEnvVar := "" | ||
| if cfg.AgentType != agent.AgentTypeProviderCLI { |
There was a problem hiding this comment.
The provider_cli branch also has a native auth key: buildProviderCLIContainerEnv writes ANTHROPIC_API_KEY/GEMINI_API_KEY/OPENAI_API_KEY from CLIAPIKeySecret when CLIAuthMode == "api_key". Leaving LLMKeyEnvVar empty for these agents keeps a rotated CLI key frozen on the Pod — the same failure this PR fixes for LLM agents.
Technical details
# provider_cli native key is not reconciled
## Affected sites
- services/agent-runner/internal/executor/executor.go:798 — LLMKeyEnvVar forced empty for provider_cli
- services/agent-runner/internal/executor/executor.go:501-515 — buildProviderCLIContainerEnv sets the adapter's native key env var from CLIAPIKeySecret
- services/agent-runner/internal/executor/providercli/*.go — APIKeyEnvVar() (claude-code→ANTHROPIC_API_KEY, codex→OPENAI_API_KEY, gemini-cli→GEMINI_API_KEY)
## Required outcome
- A provider_cli agent in api_key auth mode has its native key env var reconciled on attach, like the LLM path.
## Suggested approach (optional)
- In coldStartEnvironment, when AgentType == provider_cli and CLIAuthMode == api_key, look up providercli.Get(cfg.CLIProvider).APIKeyEnvVar() and use it as LLMKeyEnvVar (same empty-string guard for unsupported adapters / non-api_key mode).| } | ||
|
|
||
| // envMapOf reads backendRef's Deployment container env into a map. | ||
| func envMapOf(t *testing.T, m *Manager, name string) map[string]string { |
There was a problem hiding this comment.
envMapOf is a duplicate of the existing deploymentEnvMap helper just above (only the namespace source differs). Reuse deploymentEnvMap and drop this helper.
pikann
left a comment
pikann
left a comment
There was a problem hiding this comment.
LGTM! Thank you for the contribution! 🚀
Fixes #652.
Symptom: a conversation attached to a durable environment died at its first model call with
executor: acp session/prompt: Authentication required, while the gateway recorded401 KeyNotFoundError— the Pod was calling the provider with a key the agent row had stopped holding days earlier.Cause: a static environment bakes the creating agent's LLM env at Pod creation.
ensureEnvironmentEnvbackfills it once and short-circuits forever after;ensureEnvironmentInfraEnvreconcilespacaInfraEnvKeysplusGOOSE_MODELon every start but leaves the model API key andOPENAI_HOSTfrozen. So a rotated key, a moved base URL or a model change never reaches the Pod, and every later attach uses a credential that no longer exists.Change
services/agent-runner/internal/sandbox/environment.go— newEnvironmentConfig.LLMKeyEnvVar: the provider's key env var name varies by provider (OPENAI_API_KEY,ANTHROPIC_API_KEY, …), so a backend cannot infer it fromEnvalone.services/agent-runner/internal/sandbox/{k8s,docker}/environment.go—infraEnvKeysFornow also returnsOPENAI_HOSTandcfg.LLMKeyEnvVarnext toGOOSE_MODEL, so both backends reconcile them on everyStartEnvironment(set, update, or remove when the agent no longer has the value). Still gated on a matchingGOOSE_PROVIDER: another provider's model id, base URL or key var is meaningless to the Pod's frozen provider, so that case is unchanged.services/agent-runner/internal/executor/executor.go— pass the key var name fromresolveProviderEnv, empty for aprovider_cliagent (which has no model key of its own).pacaInfraEnvKeys,StartEnvironment'sEnvcontract andinfraEnvKeysForupdated to match;PACA_AGENT_ID/PACA_PROJECT_IDstay deliberately frozen.Tests (
internal/sandbox/k8s/environment_test.go)TestEnsureEnvironmentInfraEnv_RefreshesSameProviderLLMKeys: a Pod holdingsk-revoked+ an old model + the gateway base URL gets the agent's current key, model and host, with a user env var (GITHUB_TOKEN) left alone.TestEnsureEnvironmentInfraEnv_LeavesLLMKeysWhenProviderDiffers: another provider'sGOOSE_MODEL/key var/base URL are not applied — thePACA_*keys still reconcile.Verification (local,
services/agent-runner)go build ./...,go vet ./...,gofmt -lclean;go test -count=1 ./internal/sandbox/... ./internal/executor/...passes.GOOSE_MODELonly) fails on exactly the reported failure mode:deployment env["OPENAI_API_KEY"] = "sk-revoked", want "sk-fresh"plus the staleOPENAI_HOST.golangci-lintis not installed on this workstation — CI is the gate for it.Remaining limit (unchanged by this PR):
GOOSE_PROVIDERitself is still frozen to the first agent, so two agents with different providers cannot share one environment; the second one's attach still runs against the first one's provider. That is the frozen-identity design, not something this fix silently changes — worth its own decision if environments are meant to serve a whole agent fleet.