You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug(storage): context-offload page reclamation never retries an unfinished WAL checkpoint #6149
reclaimFreePages in packages/storage/src/sqlite-context-offload-store.ts (added in #5855) runs PRAGMA wal_checkpoint(PASSIVE) only in the call where incremental_vacuum reclaimed pages:
PASSIVE does not wait for readers. If another connection holds an older read snapshot while the last batch commits, the checkpoint stops early and the truncated pages stay in the WAL. The freelist is already empty at that point, so every later maintenance tick returns at the first check and never retries the checkpoint. The pages are reclaimed logically, but context-offload.sqlite does not shrink on disk until some unrelated write happens to trigger an autocheckpoint. On an idle install, that may never happen.
@Astro-Han reproduced the same pattern with real SQLite in the review of #6083, which copies this logic for runtime.sqlite: after 536 pages were drained with a reader pinned, the file stayed at 3,334,144 bytes; one extra PASSIVE checkpoint brought it to 1,138,688 bytes.
This is not data corruption: reads, the freelist accounting and quick_check are all correct. The problem is only that the disk space is not returned.
Expected
Once the freelist drains, a checkpoint that could not finish is retried later without blocking. The maintenance lane still never takes the write lock on an idle tick, never waits on readers, and never runs a full VACUUM.
Suggested fix
Read the row that wal_checkpoint(PASSIVE) returns (busy, log, checkpointed). If busy = 1 or checkpointed < log, keep an in-memory "checkpoint pending" flag. On the next tick, if the freelist is empty but the flag is set, run PASSIVE again outside any write transaction, and clear the flag once it completes. Return hasMore: false so the lane stays on its idle cadence instead of spinning while a reader holds its snapshot.
Add a regression test that holds a read transaction across the final batch, releases it, and asserts that the file shrinks on the next idle call.
Ideally runtime.sqlite (#6083) and context-offload share one small helper, so the two lanes do not drift apart. See my review on #6083.
Automated contribution plan, prepared with OpenAI Codex:
The empty-freelist return currently skips the only PASSIVE checkpoint path, so a checkpoint interrupted by a pinned reader loses its retry opportunity. I plan to keep the checkpoint completion state in memory, read the busy/log/checkpointed result, and retry a pending PASSIVE checkpoint on later idle calls after the freelist drains. The retry stays outside write transactions and returns hasMore: false so readers cannot cause a busy maintenance loop.
I will put the checkpoint-state logic in a small internal storage helper that the runtime lane can reuse, while limiting this change to context-offload: #6083 currently remains runtime-only. I will add the requested real-SQLite regression holding a read transaction across the final reclamation batch, then releasing it and checking that the next idle call shrinks the main file, alongside the existing idle-lock and bounded-batch coverage.
Thanks @Dante-dan, assigned. The plan matches what I had in mind: keep the checkpoint state in memory, retry PASSIVE outside write transactions after the freelist drains, return hasMore: false, and add the pinned-reader regression. Keeping the helper reusable but scoped to context-offload is fine. #6083 can adopt it for runtime.sqlite once both land. Please say Fixes #6149 in the PR.
Activity
Dante-dan commentedon Oct 18, 2026
Automated contribution plan, prepared with OpenAI Codex:
The empty-freelist return currently skips the only PASSIVE checkpoint path, so a checkpoint interrupted by a pinned reader loses its retry opportunity. I plan to keep the checkpoint completion state in memory, read the
busy/log/checkpointedresult, and retry a pending PASSIVE checkpoint on later idle calls after the freelist drains. The retry stays outside write transactions and returnshasMore: falseso readers cannot cause a busy maintenance loop.I will put the checkpoint-state logic in a small internal storage helper that the runtime lane can reuse, while limiting this change to context-offload: #6083 currently remains runtime-only. I will add the requested real-SQLite regression holding a read transaction across the final reclamation batch, then releasing it and checking that the next idle call shrinks the main file, alongside the existing idle-lock and bounded-batch coverage.
Dante-dan commentedon Oct 18, 2026
take
liugddx commentedon Oct 18, 2026
Thanks @Dante-dan, assigned. The plan matches what I had in mind: keep the checkpoint state in memory, retry PASSIVE outside write transactions after the freelist drains, return
hasMore: false, and add the pinned-reader regression. Keeping the helper reusable but scoped to context-offload is fine. #6083 can adopt it forruntime.sqliteonce both land. Please sayFixes #6149in the PR.