[lukevalenta]

Why

A sequencer commits a checkpoint before calling its application callback. If that callback fails after the checkpoint is durable, the failure is logged but its input is lost. MTC landmark publication is such a callback: losing it leaves a checkpoint that cannot later acquire the landmark-dependent objects expected by clients.

This change persists the callback input before invoking a callback configured for durable delivery. It clears that record only after success, and retries it before a subsequent checkpoint is sequenced. Durable callback delivery is idempotent because the callback may run more than once after a callback or cleanup failure.

When a durable callback retry keeps failing, the sequencer skips new sequencing and increments checkpoint_callback_retry_failures_total so the condition can be alerted on.

CT keeps its best-effort callback delivery and does not persist callback inputs. It does, however, adopt the fatal checkpoint-upload handling and checkpoint re-upload recovery needed to keep its existing state consistent after storage failures.

Compatibility

This changes Durable Object state and callback delivery only. It does not change checkpoint text, signed-note signatures, tile paths, entry bundles, or HTTP responses. It remains wire-compatible with the current versionless c2sp.org/tlog-tiles specification and c2sp.org/tlog-checkpoint@v1.1.0.

[lbaquerofierro]

The description says CT keeps its existing behavior, but CT also changes here: a failed checkpoint upload to R2 is now fatal, and load() re-uploads the checkpoint. Maybe add a line about that to the description, with the reason?