Skip to content

Fast set-based inserts and drop_chunks retention for db-timescale - #83

Merged
divi255 merged 4 commits into
mainfrom
fix/db-timescale-set-based-insert-drop-chunks
Oct 29, 2026
Merged

divi255 merged 4 commits into
mainfrom
fix/db-timescale-set-based-insert-drop-chunks

Conversation

@Citizen4our

@Citizen4our Citizen4our commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Synopsis

Large Timescale event flushes in eva-db-timescale were slow enough to hit sqlx’s hard-coded 2s slow-statement warning, and retention cleanup stayed expensive (DELETE over hypertables).

Solution

  • Replace per-row PL/pgSQL insert loops with a set-based UNNEST / INSERT … SELECT body for both insert_state_history_events and try_insert_state_history_events (stable OID lock order; vanished OIDs fail loudly instead of silent drops).
  • Prefer Timescale drop_chunks for hypertables; keep DELETE only for plain tables after an EXISTS probe.
  • Add optional config slow_statement_threshold (seconds; 0 disables; default remains 2s warn).
  • Coordinate OID cleanup with in-flight inserts via a shared/exclusive PostgreSQL advisory transaction lock, so cleanup cannot delete an OID between the OID insert and the events insert of a batch.
  • Add unit tests and ignored Timescale integration tests.

@Citizen4our Citizen4our self-assigned this Oct 19, 2026
@Citizen4our
Citizen4our marked this pull request as ready for review October 19, 2026 11:21
@Citizen4our
Citizen4our requested a review from divi255 October 19, 2026 11:21
@Citizen4our
Citizen4our changed the base branch from stable to main October 29, 2026 12:49
@Citizen4our
Citizen4our force-pushed the fix/db-timescale-set-based-insert-drop-chunks branch from 393884b to 6e7a37c Compare October 29, 2026 12:52
@divi255
divi255 merged commit d61f015 into main Oct 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants