[swuferhong]

Purpose

Linked issue: close #3859

Treat rebalance timeout as a non-final state and continue tracking timed-out bucket migrations without blocking subsequent tasks.

Fence stale timeout and NotifyLeaderAndIsr events by rebalance attempt and request epochs. Reconcile migrations idempotently from persisted assignment and LeaderAndIsr state, and recover unfinished tasks after coordinator restart.

Persist cancellation intent before changing local state, drain admitted tasks safely, retry final-state persistence, and treat deleted buckets as completed.

Brief change log

Tests

API and Format

Documentation

[platinumhamburg]

LGTM +1.

I have left three non-blocking suggestions concerning operational configurability and regression coverage.

[fxbing]

Thanks for your contribution! I've run into this issue as well and left a few comments for you to take a look at.

[fxbing]

Could rebuilding all plans as pending leave existing migrations outside the admission limit during recovery?

A local Manager/serde probe reproduced this with maxTrackedTimedOutTasks=1:

1. Start bucket A, leave its assignment at the union state, and move it to TIMEOUT. Further admission is correctly blocked.
2. Serialize and reload the persisted task, then recover a new manager while preserving A's union assignment.
3. Exercise a recovery order where a previously unstarted bucket B appears before A.

B is then admitted while A remains pending. Both buckets have union assignments, but A's execution key and rebalance-plan lookup return null.

The probe used 12 colliding TableBucket keys—new TableBucket(i + 1L, 31 * (11 - i)), for i=0..11—to exercise a different iteration order after deserialization. It used the real Manager/serde/context with a fake executor, so this demonstrates the admission and tracking behavior rather than actual TabletServer traffic.

[swuferhong]

Agreed. Recovery now classifies every persisted bucket before admitting any new work. Buckets that are no longer at origin are restored directly as tracked TIMEOUT attempts with an execution key and an immediate reconciliation event. Only after all buckets have been classified does normal admission run, so an intermediate migration is discovered before the maxTrackedTimedOutTasks check can admit another origin task.