Skip to content

fix(om): honor background posts requested while acknowledgement is pending - #291

Merged
egil merged 1 commit into
mainfrom
egil/om-290-background-drain
Oct 10, 2026
Merged

egil merged 1 commit into
mainfrom
egil/om-290-background-drain

Conversation

@egil

@egil egil commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Background posts and matching reminders received while dispatch or acknowledgement owns the outbox now preserve one follow-up request. Newly committed work dispatches promptly after acknowledgement instead of waiting for RetryDelay. Repeated requests coalesce; an explicit request can retry the whole pending snapshot once, while ordinary failures retain retry cadence.

 background request during delivery/acknowledgement
- change timer only; acknowledgement can overwrite it with RetryDelay
+ retain request until a fresh snapshot consumes it
 acknowledgement completes
+ schedule requested drain immediately

Queued acknowledgement callbacks whose batch was already consumed by a foreground post now leave the follow-up timer alone. Public interfaces, storage formats, reminder policies, and the separate acknowledgement turn are unchanged.

Evidence

  • Before: the gated regression on the unmodified processor timed out waiting five seconds for B with a two-minute retry delay. Reversing only the stale-acknowledgement guard also made the foreground follow-up test time out waiting for C.
  • After: all seven new hosted scenarios and all 988 Messaging solution tests pass in Release. The tests use real Orleans timers, with gates at the timer boundary to force the problematic ordering; they cover background/reminder triggers, coalescing, no duplicate successful delivery, failure retry cadence, and foreground takeover.
  • Release CI build: 0 warnings, 0 errors. All four packages pack successfully and pass the existing NuGet validation rules, including SourceLink.
  • Independent review of final commit 7cca9ce5: Standards: no actionable findings. Spec: no findings. All GitHub checks pass. An existing Journaling fault-injection test failed on the first CI run and passed on rerun; the outbox suite passed both runs.
  • Stable candidate: NBGV produces 1.1.4 from v1.1.3 plus this fix. All 988 tests pass at patch commit 2c6ab7f4; its Release build has zero warnings/errors and all four packages validate. The core DLL extracted from Egil.Orleans.Messaging.1.1.4.nupkg also passes all seven new hosted regressions.

Merge Danger

Door: two-way

The change can be reverted without a storage migration.

Blast Radius: Outbox

Explicit requests during a failed batch permit one prompt retry of the pending snapshot. Without a further request, remaining failures wait for the configured retry delay.

Exceptional exits (processing timeout, cancellation, or throwing acknowledgement callback) preserve recovery at RetryDelay, as specified in the approved plan. They retain the request flag until the next retry takes a fresh snapshot; they do not schedule an immediate exceptional retry.

Refs #290.

Release handoff

The reviewed fix is backported to egil/om-290-1.1-patch, commit 2c6ab7f453017f13f1ae5cf5c96db2b51578c6eb. The local release/egil-orleans-messaging/v1.1 branch points at that commit. Its only difference from this PR's head is the stable version file.

Maintainer merge and publication remain pending. Publishing that commit to release/egil-orleans-messaging/v1.1 invokes the existing build/test/package-validation/release workflow and publishes stable 1.1.4 packages (Journaling remains 1.1.4-preview). The publishing branch has not been pushed. After publication, verify the NuGet package's source commit and regression behavior, then record the published version in #290.

…nding

Background posts and matching outbox reminders received during delivery or
acknowledgement now request one immediate follow-up drain. Newly committed
messages no longer wait for RetryDelay after the current batch completes.
Repeated requests coalesce, and failures keep their configured retry cadence
unless a caller explicitly requests another drain of the pending snapshot.

Queued acknowledgement callbacks already consumed by a foreground post no
longer overwrite its immediate follow-up schedule. Dispatch and acknowledgement
remain separate turns, and pending batches are not delivered twice.

Refs #290
@egil
egil force-pushed the egil/om-290-background-drain branch from 9ebecc3 to 7cca9ce Compare October 10, 2026 17:34
@egil
egil marked this pull request as ready for review October 10, 2026 17:39
Copilot AI balanced review requested due to automatic review settings October 10, 2026 17:39
@claude

claude Bot commented Oct 10, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Exceptional dispatch and acknowledgement paths still overwrite explicit follow-up requests with the retry delay.

1 open finding
What changed in this PR

Updates outbox draining so background requests made during delivery or acknowledgement trigger a prompt follow-up dispatch.

Changes:

  • Tracks coalesced follow-up drain requests.
  • Preserves timers during stale acknowledgements.
  • Adds hosted regression coverage and timer controls.
File Description
OutboxProcessor.cs Tracks and schedules requested follow-up drains.
OutboxProcessorDrainRequestTests.cs Covers delivery, acknowledgement, retries, and foreground takeover.
OutboxDrainRequestGrain.cs Provides the regression-test grain and synchronization gates.
GatedOutboxTimerRegistry.cs Adds controllable Orleans timer decoration.
MessagingTestClusterFixture.cs Installs the timer decorator.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@egil
egil merged commit 8f3b206 into main Oct 10, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants