Skip to content

feat(codex): mid-turn steering via turn/steer (Steering composer option + ↳ Steered badge)#906

Open
swear01 wants to merge 10 commits into
tiann:mainfrom
swear01:fix/issue-888-mid-turn-steering
Open

feat(codex): mid-turn steering via turn/steer (Steering composer option + ↳ Steered badge)#906
swear01 wants to merge 10 commits into
tiann:mainfrom
swear01:fix/issue-888-mid-turn-steering

Conversation

@swear01

@swear01 swear01 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Closes #888

Summary

Adds mid-turn steering for Codex: when the operator opts in, a message sent while a turn is running is delivered into the active turn via the app-server turn/steer RPC, instead of waiting for the whole turn to finish.

  • New session option steeringMode (queue default | steer), exposed as a Steering radio section in the composer settings popover next to Permission Mode. Codex-only (auto-hidden for other flavors).
  • Codex: a queued message that arrives while turnInFlight is steered in via turn/steer (with the expectedTurnId precondition). Falls back to the normal queue-until-turn-end path on activeTurnNotSteerable (review/compact turns), an expectedTurnId race, or an older codex without the method.
  • Steered messages get a ↳ Steered badge in the web UI (like Codex's native GUI/TUI marks mid-turn messages). The CLI is the authoritative source: it re-emits messages-consumed with a steered flag on a successful steer; the hub relays it live; the web renders the label.

Why Codex-only

Claude Code's real-time steering is interactive-TUI only — it is not exposed through the Agent SDK / stream-json interface HAPI drives (upstream anthropics/claude-agent-sdk-typescript#70; verified empirically that mid-turn stdin input is ignored). So Claude is intentionally excluded until session.send() ships there. See the investigation notes on #888.

Also fixed

  • Queued-clock bug: because steered messages are consumed immediately, the hub's message echo arrives already-invoked; mergeMessages was copying the optimistic queued status onto that delivered message, pinning the queued clock. Now a message with invokedAt set never renders as queued.

Scope notes

  • The steered badge is a live marker (relayed over SSE, preserved across in-session refetches); it is not persisted across a full page reload (that would need a SQLite steered column + migration). Easy follow-up if desired.

Test plan

  • Protocol unit tests for getSteeringModeOptionsForFlavor (codex-only)
  • codexRemoteLauncher tests: turn/steer fires in steer mode; no steer in queue mode
  • mergeMessages tests: no queued inherited onto an invoked echo; queued kept while not-yet-invoked; stuck state normalized
  • Typecheck clean (cli / web / hub / shared)
  • Full suite green (web 967, shared 83, + cli/hub)
  • Live e2e against a real Codex session: turn/steer fires (model acted on the steered instruction in the same turn), messages-consumed carries steered: true, and the ↳ Steered badge renders on the steered bubble.

🤖 Generated with Claude Code

swear01 added 5 commits June 14, 2026 03:05
…odex-only)

Protocol: STEERING_MODES/SteeringMode + getSteeringModeOptionsForFlavor (codex
only — Claude steering is TUI-only, not in the Agent SDK; see tiann#888). Wire
steeringMode through session schemas, socket keepalive type, apiTypes request,
resume target, and type re-exports.

Web: new 'Steering' radio section in the composer settings popover next to
Permission Mode (queue | steer), api.setSteeringMode + useSessionActions
mutation + SessionChat handler, en/zh-CN locales. Hub route + CLI turn/steer
land next.
Hub: persist/sync steeringMode through sessionCache (runtime config + broadcast),
syncEngine (handleSessionAlive, applySessionConfig, resume targets), rpcGateway,
cli socket session-alive payloads, and a /sessions/:id/steering-mode REST route
(codex-only via isSteeringSupportedForFlavor).

CLI: carry steeringMode on the keepalive runtime (sessionBase + apiSession wire),
CodexSession setter, runCodex SetSessionConfig accept + resolve + echo in applied.
codexRemoteLauncher: in 'steer' mode, deliver a queued message into the active
turn via app-server turn/steer (expectedTurnId precondition) instead of waiting
for the turn to end; on activeTurnNotSteerable / turn-ended / unknown-method,
fall back to the normal queue-until-turn-end path. Wake the parked loop on
mid-turn enqueue only in steer mode (no-op for default queue mode).

Tests: codexRemoteLauncher steer + queue-mode coverage; session-stub gains
getSteeringMode/setSteeringMode.
A message sent while a turn is running gets an optimistic 'queued' status. In
steer mode the CLI consumes it immediately, so the hub's message echo arrives
already invoked (invokedAt set). mergeMessages copied the optimistic 'queued'
onto that already-invoked server echo, pinning the queued/pending clock on a
message that was actually delivered (steered into the active turn).

Fix mergeMessages: never carry a 'queued' status onto a message with invokedAt
set, and normalize any stuck queued+invoked state to 'sent' at all return paths.
Normal queue-mode messages (not yet invoked) keep their queued clock.
Like Codex's native GUI/TUI marks mid-turn messages, surface when a message was
steered into an active turn. The CLI is the authoritative source: on a
successful turn/steer it re-emits messages-consumed with a steered flag (the
protocol's userMessage item carries no steer marker). The hub relays the flag
live (not persisted); the web sets DecryptedMessage.steered, carries it through
normalize -> block -> assistant-ui metadata, and UserMessage renders a small
'↳ Steered' label in the footer. Survives in-session refetches via mergeMessages
preservation.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Steering swallows Codex control commands — when steering mode is enabled, the new loop branch drains any queued batch while a turn is in flight and immediately calls turn/steer before handleSpecialCommand runs. /clear and /compact are intentionally enqueued as isolated control commands, but this path ignores steerBatch.isolate and sends the literal command text into the active Codex turn, so the thread is not reset/compacted. Evidence: cli/src/codex/codexRemoteLauncher.ts:2979, related command detection in cli/src/codex/runCodex.ts:213 and handling in cli/src/codex/codexRemoteLauncher.ts:2926.
  • [Minor] Local resume drops the saved steering mode — the response schema now includes steeringMode, but the Codex resume dispatcher still calls runCodex without it, and runCodex therefore starts from queue via the loop default. A session resumed locally after choosing steer in the web UI silently loses that setting. Evidence: shared/src/resume.ts:20, related dispatcher at cli/src/commands/resume.ts:93.

Questions

  • None.

Summary

  • Review mode: initial
  • Two issues found: one behavior regression for Codex special commands while steering is enabled, and one resume-state propagation gap.

Testing

  • Not run (automation): attempted targeted Bun tests, but this environment has no bun binary.

HAPI Bot

// instead of waiting for it to finish. On failure (turn ended, or a
// non-steerable review/compact turn) fall back to the normal path:
// process it as a fresh turn once the active turn terminates.
if (!pending && turnInFlight && session.getSteeringMode() === 'steer'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This branch steers every queued batch while a turn is active, including isolated Codex control commands. runCodex enqueues /clear and /compact as isolated messages so handleSpecialCommand can interrupt/reset/compact, but this branch drains them first and sends the literal text to turn/steer. That leaves the old thread alive instead of executing the command.

Suggested fix:

if (!pending && turnInFlight && session.getSteeringMode() === 'steer'
    && this.currentThreadId && this.currentTurnId && session.queue.size() > 0) {
    let steeredLocalIds: string[] = [];
    const autoOnBatchConsumed = session.queue.onBatchConsumed;
    session.queue.onBatchConsumed = (localIds) => { steeredLocalIds = localIds; };
    const steerBatch = await session.queue.waitForMessagesAndGetAsString(this.abortController.signal);
    session.queue.onBatchConsumed = autoOnBatchConsumed;

    if (steerBatch?.isolate || (steerBatch && parseCodexSpecialCommand(steerBatch.message).type)) {
        if (steeredLocalIds.length > 0) {
            session.client.emitMessagesConsumed(steeredLocalIds);
        }
        pending = steerBatch;
        await waitForTurnOrRecovery(this.abortController.signal);
        continue;
    }

    // existing turn/steer path
}

Comment thread shared/src/resume.ts
permissionMode: PermissionModeSchema.optional(),
collaborationMode: CodexCollaborationModeSchema.optional()
collaborationMode: CodexCollaborationModeSchema.optional(),
steeringMode: SteeringModeSchema.optional()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] Adding steeringMode to the resume target makes the hub return it, but the CLI resume dispatcher does not pass it into runCodex. Since the Codex loop defaults missing steering to queue, a session configured as steer silently reverts after local resume.

Suggested fix:

// cli/src/codex/runCodex.ts
export async function runCodex(opts: {
    // ...existing fields
    collaborationMode?: EnhancedMode['collaborationMode'];
    steeringMode?: SteeringMode;
    existingSessionId?: string;
    workingDirectory?: string;
}): Promise<void> {
    // ...
    let currentSteeringMode: SteeringMode = opts.steeringMode ?? 'queue';
}

// cli/src/commands/resume.ts
await runCodex({
    existingSessionId: base.existingSessionId,
    workingDirectory: base.workingDirectory,
    resumeSessionId: base.resumeSessionId,
    startedBy: base.startedBy,
    permissionMode: base.permissionMode as CodexPermissionMode | undefined,
    model: target.model ?? undefined,
    modelReasoningEffort: (target.modelReasoningEffort ?? undefined) as ReasoningEffort | undefined,
    collaborationMode: target.collaborationMode,
    steeringMode: target.steeringMode
})

…s; thread steeringMode through resume

- [major] Steering drained isolated control commands (/clear, /compact, /goal)
  and injected their literal text via turn/steer, so the thread was never
  reset/compacted. Now control commands (isolate or parsed special/goal) bypass
  the steer path and run through handleSpecialCommand/handleGoalCommand
  immediately, matching queue-mode behavior. Regression test added.
- [minor] runCodex now accepts steeringMode and the local resume dispatcher
  passes target.steeringMode, so a 'steer' session no longer reverts to 'queue'
  after local resume.
@swear01

swear01 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review — both findings addressed in a2669aa:

[Major] Steering swallowed control commands. Fixed. The steer branch now detects control commands (steerBatch.isolate || parseCodexSpecialCommand(...).type || parseGoalCommand(...) !== null) and hands them straight to the normal loop path instead of turn/steer, so /clear / /compact / /goal run through handleSpecialCommand / handleGoalCommand (interrupt the active turn + reset/compact) — matching queue-mode behavior. Note the deferral is immediate (not await waitForTurnOrRecovery), since a pre-queued control command already interrupts the active turn in queue mode (the park is skipped while queue.size() > 0). Added a regression test: does not steer isolated control commands (/clear) while steering mode is "steer" asserts no turn/steer call and that the clear path runs (interruptedTurns + thread reset).

[Minor] Resume dropped steering mode. Fixed. runCodex now accepts steeringMode and the local resume dispatcher passes target.steeringMode, so a session configured as steer keeps it after local resume.

Re-validated: cli typecheck clean; codexRemoteLauncher + runCodex tests pass (59).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Minor] Mid-turn messages can miss steering while turn/steer is in flight — the loop only wakes when session.queue.setOnMessage fires, but after the first batch is collected it awaits trySteerActiveTurn(...) before installing waitForTurnOrRecovery(...). If another message arrives during that await, wakeLoop() sees no waiter, then line 3020 parks until the active turn completes even though session.queue.size() > 0; that message falls back to normal queue timing instead of being steered into the still-running turn. Evidence: cli/src/codex/codexRemoteLauncher.ts:3020, with wake registration at cli/src/codex/codexRemoteLauncher.ts:1844.

Suggested fix:

if (turnInFlight && !this.shouldExit) {
    if (session.getSteeringMode() === 'steer' && session.queue.size() > 0) {
        continue;
    }
    await waitForTurnOrRecovery(this.abortController.signal);
}
continue;

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • The prior control-command and local-resume issues are addressed. One remaining steering responsiveness race found.

Testing

  • Not run (automation): bun is not installed in this environment.

HAPI Bot

}
}
if (turnInFlight && !this.shouldExit) {
await waitForTurnOrRecovery(this.abortController.signal);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] A message that arrives while trySteerActiveTurn(...) is awaiting the app-server RPC can miss the wakeup. During that await loopWakeWaiter is null, so session.queue.setOnMessage calls wakeLoop() but there is no waiter to resolve; then this branch immediately waits for the active turn to finish even if session.queue.size() > 0. That leaves the new message queued until turn completion instead of steering it into the still-running turn.

Suggested fix:

if (turnInFlight && !this.shouldExit) {
    if (session.getSteeringMode() === 'steer' && session.queue.size() > 0) {
        continue;
    }
    await waitForTurnOrRecovery(this.abortController.signal);
}
continue;

…C raced a new arrival

PR tiann#906 follow-up [minor]: a message arriving while trySteerActiveTurn awaited
the app-server RPC could lose its wakeLoop() (no waiter installed yet); the loop
then parked until turn completion despite queue.size() > 0, missing the steer.
Before parking, re-loop when in steer mode with queued messages (guarded by
!pending so the non-steerable fallback still parks and processes as a new turn).
@swear01

swear01 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in the latest commit. Before parking at the end of the steer branch, the loop now re-loops when getSteeringMode() === 'steer' && queue.size() > 0, so a message that arrived during the trySteerActiveTurn RPC await (and lost its wakeLoop()) gets steered into the still-running turn instead of waiting for turn completion. Guarded with !pending so the non-steerable / turn-ended fallback still parks and processes the deferred message as a fresh turn. cli typecheck clean; launcher tests pass (57).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] Failed steer acks the fallback message before it is actually delivered — when turn/steer returns false, this new branch emits messages-consumed immediately, then stores the batch in pending and waits for the current turn to finish. The hub stamps those localIds as invoked on receipt, so the web no longer treats the message as queued even though it has not been sent as the fallback turn yet. If the CLI exits during that waiting window, the message is also no longer recoverable as queued because the DB already has invokedAt. Evidence: cli/src/codex/codexRemoteLauncher.ts:3014.
    Suggested fix:
    let pendingConsumedLocalIds: string[] = [];
    
    // steer fallback
    pending = steerBatch;
    pendingConsumedLocalIds = steeredLocalIds;
    
    // when the pending fallback is actually about to run through the normal path
    const pendingLocalIds = pendingConsumedLocalIds;
    pendingConsumedLocalIds = [];
    if (pendingLocalIds.length > 0) {
        session.client.emitMessagesConsumed(pendingLocalIds);
    }

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • The prior wakeup race is addressed. One remaining fallback-path issue can hide or lose a message after a failed steer attempt.

Testing

  • Not run (automation)

HAPI Bot

Comment thread cli/src/codex/codexRemoteLauncher.ts Outdated
// Fell back to a normal turn (turn ended / non-steerable):
// mark consumed so the queued bar clears, then reprocess.
if (steeredLocalIds.length > 0) {
session.client.emitMessagesConsumed(steeredLocalIds);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This fallback path marks the message consumed before the fallback turn actually starts. When trySteerActiveTurn() returns false, the batch has only been moved into pending; the code then waits for the current turn to finish before processing it. Emitting messages-consumed here makes the hub stamp invokedAt, so live/refetched clients stop showing it as queued while it is still just pending in memory. If the CLI exits before the active turn ends, the DB now says the message was invoked even though the fallback turn was never sent.

Suggested fix:

let pendingConsumedLocalIds: string[] = [];

// steer fallback
pending = steerBatch;
pendingConsumedLocalIds = steeredLocalIds;

// when the pending fallback is actually about to run through the normal path
const pendingLocalIds = pendingConsumedLocalIds;
pendingConsumedLocalIds = [];
if (pendingLocalIds.length > 0) {
    session.client.emitMessagesConsumed(pendingLocalIds);
}

…tually delivered

PR tiann#906 follow-up [major]: when turn/steer returned false, the fallback emitted
messages-consumed immediately while the message only sat in 'pending' awaiting
the active turn's end. That stamped invokedAt prematurely (hiding it as
not-queued) and lost it if the CLI exited mid-wait. Now the fallback holds the
localIds and emits the consumed ack only when the message is pulled for its real
turn, so it stays queued/recoverable until actually sent.
@swear01

swear01 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in the latest commit. The steer-fallback no longer emits messages-consumed up front. Its localIds are held in deferredSteerFallbackLocalIds and the ack is emitted only when the message is pulled from pending for its real turn — so while it waits for the active turn to end it stays queued (no premature invokedAt) and remains recoverable if the CLI exits. cli typecheck clean; launcher tests pass (57).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Minor] Steered marker is lost when the stored message replaces the optimistic row — the new same-id merge path preserves steered, but the normal optimistic-to-server replacement is by localId and only carries status/invokedAt forward. Because the hub intentionally does not persist steered, the ↳ Steered badge disappears as soon as the server echo/refetch replaces the optimistic row. Evidence: web/src/lib/messages.ts:79.

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • The prior fallback-consumption issue is fixed. One remaining UI-state issue can drop the live steered badge during normal message reconciliation.

Testing

  • Not run (automation)

HAPI Bot

Comment thread web/src/lib/messages.ts
if (existing.invokedAt != null && msg.invokedAt == null) {
preserved.invokedAt = existing.invokedAt
}
if (existing.steered && !msg.steered) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] This preserves the live-only steered marker only when the incoming server copy has the same id. The usual optimistic replacement path below matches by localId: the optimistic row has id === localId, the stored echo has a DB id, and lines 100-138 only carry status/invokedAt forward before filtering the optimistic row out. Since the hub does not persist steered, the ↳ Steered badge will disappear on the first server echo/refetch after the consume event.

Suggested fix:

const optimisticSteeredByLocalId = new Set<string>()

for (const msg of merged) {
    if (msg.localId && isOptimisticMessage(msg) && incomingStoredLocalIds.has(msg.localId)) {
        if (msg.steered) {
            optimisticSteeredByLocalId.add(msg.localId)
        }
    }
}

// when applying localId-preserved fields to the stored message
if (optimisticSteeredByLocalId.has(msg.localId) && !msg.steered) {
    update.steered = true
}

…econciliation

PR tiann#906 follow-up [minor]: the optimistic-to-server replacement carried status
and invokedAt forward but not steered. Since the hub never persists steered, the
↳ Steered badge vanished when the server echo/refetch replaced an optimistic row
that had been marked steered. mergeMessages now preserves steered in that path
too (tests added).
@swear01

swear01 commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in the latest commit. mergeMessages now carries the live steered marker from the optimistic row onto the replacing server echo (alongside status/invokedAt), so the ↳ Steered badge survives the optimistic→server reconciliation even though the hub doesn't persist steered. Added two regression tests (optimistic→echo replacement; refetch that omits it). web typecheck clean; messages tests pass (6).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • None.

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • No issues found in the latest full diff. The prior bot finding about losing the live steered marker during optimistic-to-server reconciliation is addressed in web/src/lib/messages.ts and covered by web/src/lib/messages.test.ts.
  • Residual risk: I did not run the full suite in this automation pass.

Testing

  • Not run (automation)

HAPI Bot

…urn-steering

# Conflicts:
#	cli/src/agent/sessionBase.ts
#	cli/src/api/apiSession.ts
#	cli/src/codex/runCodex.ts
#	hub/src/socket/handlers/cli/sessionHandlers.ts
#	hub/src/sync/sessionCache.ts
#	shared/src/modes.test.ts
#	web/src/api/client.ts
#	web/src/components/AssistantChat/HappyComposer.tsx
#	web/src/components/SessionChat.tsx
#	web/src/hooks/mutations/useSessionActions.ts

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • None.

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • No issues found in the latest full diff. I reviewed the Codex steering path, hub messages-consumed propagation, web reconciliation/rendering, and the added coverage around steering markers and queued-status normalization.
  • Residual risk: I did not run the full suite in this automation pass.

Testing

  • Not run (automation)

HAPI Bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(codex,claude): mid-turn steering — deliver queued messages at step boundaries, not only on full-turn completion

1 participant