feat: Add conversation tracing infrastructure with session tracking and message persistence#51
Open
stefanskoricdev wants to merge 13 commits into
Open
feat: Add conversation tracing infrastructure with session tracking and message persistence#51stefanskoricdev wants to merge 13 commits into
stefanskoricdev wants to merge 13 commits into
Conversation
stefanskoricdev
force-pushed
the
feat/conversations
branch
from
35d6e2e to
502bef1
Compare
stefanskoricdev
changed the title
stefanskoricdev
force-pushed
the
feat/conversations
branch
from
e953cf8 to
be35055
Compare
Member
|
Please collapse all migration to a single one, and we need to wait both for turso to fix the multiprocess wal and claude code agent trace impl |
Add session_id: Option<String> to TouchedLine for traceability. Extend parse_patch(input, session_id) and hunk parsing to stamp each added/removed touched line with the provided session ID. Update patch parser consumers and tests to pass explicit None where no session context exists, and session IDs where available. Keep patch/domain docs in sync with the parser and model contract. Co-authored-by: SCE <sce@crocoder.dev>
Convert intersect_patches from .filter().cloned() to .filter_map() so that when a post-commit hunk line matches an available line, the resulting overlapping line inherits the session_id from the matched entry in the first patch (a). This ensures session provenance is preserved through intersection, matching the existing model_id inheritance pattern. Co-authored-by: SCE <sce@crocoder.dev>
- Introduce ConversationRelated with schema-aligned fields (type, url) and add optional Conversation.related to the conversation payload model - Omit related when absent via skip_serializing_if = "Option::is_none" and initialize new entries with related: None in trace construction - Update context/sce/agent-trace-minimal-generator.md to reflect the current payload/type shape Co-authored-by: SCE <sce@crocoder.dev>
…sion_id provenance Conversation.related was always None despite the ConversationRelated domain type existing in the schema. Downstream consumers need session provenance to trace which sessions contributed to each AI-authored conversation. Extract non-empty session_id values from touched lines on the matched intersection_patch hunk, deduplicate with deterministic BTreeSet ordering, and emit conversation.related as session link entries. The related field is omitted (None) when no included lines provide session_id values. Co-authored-by: SCE <sce@crocoder.dev>
Add Agent Trace DB migrations for session messages and append-only parts, including natural-key message upserts, chronological lookup indexes, and updated_at triggers. Expose typed Rust helper payloads for message upserts and part inserts, and update Agent Trace DB context to reflect the current schema and helper surface. Co-authored-by: SCE <sce@crocoder.dev>
Add a new `sce hooks conversation-trace` subcommand that reads normalized snake_case STDIN envelopes (message.updated / message.part.updated), validates required fields, and persists messages via AgentTraceDb::upsert_message() or parts via AgentTraceDb::insert_part() without writing context/tmp artifacts. The CLI schema, command routing, and context documentation are updated to reflect the new subcommand. Co-authored-by: SCE <sce@crocoder.dev>
Remove the parent messages.text column from the fresh Agent Trace DB schema and message upsert path. Conversation trace message.updated payloads now persist only parent message metadata, while body text remains owned by message.part.updated rows through parts.text. Co-authored-by: SCE <sce@crocoder.dev>
Introduce a new `ConversationTrace` hook subcommand that validates normalized snake_case `message.updated` and `message.part.updated` STDIN envelopes and persists them to AgentTraceDb — messages via insert-ignore-duplicate `(session_id, message_id)` semantics and parts via append-only inserts. The generated OpenCode agent-trace plugin now calls this hook for every captured message event: `message.updated` handoff runs before the existing diff-trace flow, and `message.part.updated` events trigger only conversation-trace without invoking diff-trace. Supporting changes: - Rename `UpsertMessageInsert` → `InsertMessageInsert` with `ON CONFLICT ... DO NOTHING` instead of upsert-update - Enable `experimental_multiprocess_wal(true)` on local Turso connections for safe concurrent `sce` process access - Update context docs to reflect conversation-trace behavior, insert-ignore-duplicate semantics, plugin handoff contract, and multiprocess WAL mode Co-authored-by: SCE <sce@crocoder.dev>
Introduce configurable retry policies for connection open and query operations across local_db, agent_trace_db, and auth_db Turso adapters. Retry parameters (max_attempts, timeout_ms, initial_backoff_ms, max_backoff_ms) are configured under policies.database_retry.<db>.connection_open|query and initialized at app startup. - Add run_with_retry_sync to resilience.rs for synchronous retry with timeout classification and capped exponential backoff - Add db_config_key() to DbSpec trait implemented by all three DB specs for per-adapter config lookup - Add DATABASE_RETRY_CONFIG OnceLock with hardcoded defaults (3 attempts, 5s/3s timeouts, 100ms-1s/100ms-500ms backoff) - Wrap TursoDb/EncryptedTursoDb constructors and execute/query/query_map in config-driven retry loops - Add database_retry schema to sce-config-schema.pkl and regenerate derived config/schema/sce-config.schema.json - Change AgentTraceDb insert_message to update summary_diffs on duplicate (session_id, message_id) conflicts instead of ignoring the write; align embedded migration count with existing migrations 013 and 014 (updated_at triggers) - Fix encryption_key.rs doc comment indentation Co-authored-by: SCE <sce@crocoder.dev>
…messages Co-authored-by: SCE <sce@crocoder.dev>
…-row inserts
Replace the single-event STDIN envelope for `sce hooks conversation-trace`
with a typed batch envelope `{ type, payloads }` supporting homogeneous
`message.updated` / `message.part.updated` item arrays.
- Parse top-level `type` discriminator with `payloads` array; reject
item-level `type` fields to enforce homogeneous batches
- Validate per-item fields with skipped-item diagnostics that preserve
valid sibling items for persistence
- Persist valid `message.updated` batches through one multi-row
`insert_messages()` call with duplicate-ignore semantics
- Persist valid `message.part.updated` batches through one multi-row
`insert_parts()` append-only call
- Log multi-row insert failures as whole-batch skipped without
row-by-row fallback; report deterministic attempted/persisted/skipped
counts on success
- Update agent-trace plugin to send one-element typed batch envelopes
- Regenerate OpenCode config plugin assets
- Update context docs to reflect the typed batch contract
Co-authored-by: SCE <sce@crocoder.dev>
The plugin now extracts diff patches from `message.updated` events
and traces them as `part_type: "patch"` entries in the conversation
trace. When `summary.diffs` exist, builds a `-patch` variant payload
set (one parent `message.updated` + per-diff `message.part.updated`)
and dispatches all payloads concurrently via `Promise.all`.
An in-memory dedup cache (`processedDiffsMessageIds` Set keyed by
`"${sessionID}:${messageID}"`) prevents duplicate processing of
diff-bearing events while still allowing non-diff events for the same
message pair to be processed normally.
Refactored `extractDiffEntries` helper centralises the `summary.diffs`
access pattern, shared by both `buildPatchConversationTracePayloads`
and the existing diff-trace extraction. Diff entry iteration also
switches from `as` casts to `"patch" in entry` type narrowing.
On the Rust side:
- `PartType::Patch` variant added to the enum
- SQL migration `009_create_parts.sql` updated CHECK constraint to
accept `'patch'`
- Hook parser updated to accept `"patch"` as a valid part type
Co-authored-by: SCE <sce@crocoder.dev>
stefanskoricdev
force-pushed
the
feat/conversations
branch
from
883ced3 to
40effa0
Compare
Member
|
This is blocked by tursodatabase/turso#7350 |
Switch the CLI Turso dependency from the crates.io 0.6.0 release to the pinned upstream revision used by the Nix flake, and update the lockfiles and package metadata to match the 0.7.0-pre.5 source. Co-authored-by: SCE <sce@crocoder.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.