Skip to content

fix(agent-runner): cancel a stopped turn's goose run instead of abandoning it - #655

Merged
pikann merged 2 commits into
Paca-AI:masterfrom
siedlegroup:fix/acp-cancel-abandoned-run
Oct 29, 2026
Merged

pikann merged 2 commits into
Paca-AI:masterfrom
siedlegroup:fix/acp-cancel-abandoned-run

Conversation

@kayahk

@kayahk kayahk commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #654

A stopped turn left its goose run running, so the next message in the same chat was rejected:

executor: acp session/prompt: Invalid params: session already has active run `run_…`; use _goose/unstable/session/steer

and the conversation ended failed with the user's message unprocessed. Prompt only stopped waiting on cancellation — its doc comment even said the transport had no session/cancel, which is no longer true of goose 1.50.1. Reproduced against a live paca-env-… pod: a second session/prompt while a sleep 60 tool call was in flight returns exactly that -32602, and session/cancel clears the run again in ~2–4 s.

Changes

services/agent-runner/internal/acp:

  • Prompt cancels on every exit that did not receive the turn's terminal response (user Stop, timeout_minutes budget, maxToolCalls cap, dead stream) via a new cancelRun → notify("session/cancel", …). The notification form is what goose's on_cancel handles, and it is the same call apps/acp-bridge's Runner.Interrupt/acpclient.Cancel makes.
  • Cancel-and-retry, bounded when session/prompt is rejected because a previous run is still unwinding: a message typed a second after pressing Stop lands in that window, and failing it loses what the user wrote. Budget is maxRunActiveRetries = 2 waits of 2 s each, sized from the measurement below; a rejected prompt costs no model call. A run that never clears still fails the turn, and is cancelled on the way out.
  • post grew a notify sibling (no id) sharing one send; rpcNotification/cancelParams in types.go.
  • Prompt's frame loop moved to awaitTurn so the retry doesn't duplicate it. Its behaviour is unchanged.

No api or web change: the stop already reaches the runner, it just stopped there.

Tests

services/agent-runner/internal/acp/client_test.go:

  • TestPrompt_CancelledSendsSessionCancel — a cancelled turn sends exactly one session/cancel, without an id, carrying the session's id. Sabotage (cancel stubbed out): session/cancel sent 0 times, want exactly 1 (an abandoned run left active rejects the session's next prompt).
  • TestPrompt_RetriesAfterActiveRunRejection — goose's verbatim -32602 on the first prompt, success on the retry, exactly one cancel in between. Sabotage (retry removed): Prompt: Invalid params: session already has active run run_58072af2-…; use _goose/unstable/session/steer.
  • TestPrompt_GivesUpAfterTheRetryBudget — every prompt rejected: exactly maxRunActiveRetries+1 attempts, then goose's own error, with a cancel on giving up too. Sabotage: session/prompt sent 1 times, want 3 (initial + 2 retries).
  • TestPrompt_SuccessSendsNoSessionCancel — a normal turn sends none, since a chat conversation reuses its session.

Gates

In services/agent-runner: go build ./..., go vet ./..., go test -race $(go list ./... | grep -v '/test/e2e') — all pass. golangci-lint is not installed locally, so that gate is CI's.

Behaviour is verified against goose 1.50.1 by probe, not by a rebuilt image: this repo's paca-agent-runner image still needs a build and pin bump before the deployment stops hitting this.

…oning it

A stop (or the turn timeout, or the tool-call cap) only ended Prompt's wait;
the session/prompt already sent to goose kept running, and goose rejects the
session's next session/prompt with -32602 "session already has active run
...; use _goose/unstable/session/steer" until that run ends. A Stop followed
by a new message in the same chat therefore ended the conversation failed
with the user's message unprocessed.

Send session/cancel (the notification form goose's on_cancel handles, same as
apps/acp-bridge's Runner.Interrupt) on every non-terminal exit, and retry the
prompt once after cancelling when it was rejected by a run still unwinding.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ No critical issues — one rough edge worth a look.

Reviewed changes

  • Prompt cancels on non-terminal exits — a new answered flag plus a defer sends session/cancel whenever the turn's terminal response wasn't received (ctx cancel, turn timeout, maxToolCalls, dead stream), so a stopped run is no longer left active to reject the session's next prompt with -32602.
  • One cancel-and-retry on the run-active rejection — isRunActiveError matches goose's -32602 + "session already has active run", and Prompt cancels, waits runActiveRetryDelay (2s), then re-posts the prompt once. The former inline frame loop is factored into awaitTurn (behaviour unchanged).
  • New notify sibling + wire types — post/notify share one send; rpcNotification (id-less) and cancelParams are added, matching the notification form apps/acp-bridge's acpclient.Cancel already uses.

Pullfrog  | Fix all ➔ | Fix 👍s ➔ | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

Comment thread services/agent-runner/internal/acp/client.go
… retry

Review catch: the delay comment claimed a cancelled run clears in under two
seconds, while the measurement was 1.8-3.8s (the probe polled every 2s). One
retry at 2s would still fail a message for a run on the slow side of that
range. Allow up to two retries, and pin the budget with a test that a run
which never clears still fails the turn with goose's own error.

@kayahk kayahk left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair catch — the two numbers disagreed and the retry was too tight for the slower half of the range.

What I actually measured (goose 1.50.1, sleep 60 tool call in flight, probe polling every 2 s) is: session/cancel → next prompt accepted 1.8–3.8 s later. The poll interval is the resolution, not an upper bound, so "under two seconds" in the comment was wrong.

Fixed in 42bf724:

  • retry budget is now up to two attempts (maxRunActiveRetries = 2), one cancel each, so 2 s + 2 s covers the measured window;
  • the comment states the 1.8–3.8 s measurement and that it comes from a 2 s poll, instead of claiming a bound I did not measure;
  • TestPrompt_GivesUpAfterTheRetryBudget pins the budget: with goose rejecting every prompt, the turn makes exactly 3 attempts and then fails with goose's own -32602, and session/cancel is sent on giving up too — a turn never leaves its run active. A rejected prompt costs nothing (no model call), so the only budget being spent is waiting.

go build ./..., go vet ./..., go test -race (whole module) pass; the retry tests were re-sabotaged to confirm they fail without the fix.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Retry budget widened from one attempt to a bounded three — the run-active rejection now retries up to maxRunActiveRetries = 2 (2s + 2s waits), which covers the measured 1.8–3.8s session/cancel unwind window instead of the single 2s wait that lost the message on the slow side of that range.
  • Retry-delay comment reconciled with the measurement — it now states the observed 1.8–3.8s range and that the 2s probe interval is the resolution, not an upper bound, rather than claiming an unmeasured "under two seconds".
  • New TestPrompt_GivesUpAfterTheRetryBudget — pins the budget (exactly maxRunActiveRetries+1 prompts), the cancel before each retry, and the final cancel on giving up, so a turn that never clears still fails with goose's own -32602 without leaving its run active.

The prior review's single concern (the 2s delay versus the ~2–4s reproduction) is addressed by 42bf724; the resulting two-wait budget matches the measurement, and the give-up path preserves the invariant that a turn never exits with its run still active.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@pikann pikann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you for the contribution! 🚀

@pikann
pikann merged commit e27b23b into Paca-AI:master Oct 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent-runner: a stopped turn leaves its run active, so the next message in the same chat fails with "session already has active run"

2 participants