Background
Sub-issue of #3011. To preserve LLM prefix-cache stability (a key requirement called out in the Hermes memory architecture), the inject_memories builtin maintains a frozen snapshot of injected memory text and rebuilds it only on well-defined events.
Motivation
Without snapshotting, every memory write or every minor DB change could invalidate the model's prefix cache mid-session, hurting cache hit rate and latency. Mirroring Hermes' two-tier approach gives us:
- A frozen snapshot used to build the injected block on each turn
- A live mutable state in the DB for reads/writes
- Explicit invalidation on memory writes and on context compaction
Scope
Snapshot lifecycle
Session start
│
▼
Load snapshot from DB (initial materialisation)
│
▼ ← used by inject_memories on every turn
Memory write (add_memory / update_memory / delete_memory)
│
▼ invalidate → rebuild on next turn
Context compaction event
│
▼ invalidate → rebuild on next turn
Where it lives
- Snapshot is in-process only — never persisted
- Stored alongside the agent's runtime state (one snapshot per agent per session)
- Thread-safe with RWMutex (reads on turn_start are hot path)
Implementation Checklist
Acceptance Criteria
Background
Sub-issue of #3011. To preserve LLM prefix-cache stability (a key requirement called out in the Hermes memory architecture), the
inject_memoriesbuiltin maintains a frozen snapshot of injected memory text and rebuilds it only on well-defined events.Motivation
Without snapshotting, every memory write or every minor DB change could invalidate the model's prefix cache mid-session, hurting cache hit rate and latency. Mirroring Hermes' two-tier approach gives us:
Scope
Snapshot lifecycle
Where it lives
Implementation Checklist
[]database.UserMemory+ generation counterInvalidate()method that bumps the generation counteradd_memory,update_memory,delete_memory) to callInvalidate()pkg/runtime/compactor/(or wherever the compaction signal is emitted) to callInvalidate()inject_memoriesbuiltin reads from snapshot; on stale generation, refetches from DB and storesAcceptance Criteria
add_memoryinvalidates the snapshot → next turn reflects the new memoryupdate_memoryinvalidates the snapshot → next turn reflects the changedelete_memoryinvalidates the snapshot → next turn no longer shows the deleted entrygo test -race