Repository navigation
Add opt-in dreaming: cited dreams consolidated from agent history - #282
Merged
Merged
Conversation
A dream run (`memex dream`) sends redacted excerpts of recent sessions, saved Claude and Codex memories, and the existing dreams to an agent CLI the user already has (claude, codex, or a configured command) and stores what it concludes as dreams: single statements, each citing the transcript records it rests on. Everything is behind `[dream] enabled = true`, off by default. With it off no run starts, the daemon schedules none, and search returns no dreams. - Memex decides what is written. The executor cites short handles for the records it was shown; a dream needs at least one conversation record memex actually offered, excerpts and hashes are cut from the index, and a dream the executor does not mention carries over unchanged. Revising or retracting needs new record evidence. - The store is immutable generations under `<root>/dreams/` with an atomic CURRENT pointer and a diff per generation. Claude and Codex memory files are never written. - `memex dream lint` re-finds each citation by record ID, native event ID, identical text in the session, then excerpt search, rewrites moved IDs, and marks lost evidence instead of dropping it. - `--content all` (CLI and MCP) returns dreams beside the conversations and memories they cite; `--content dreams` returns them alone. - `[dream] auto = true` lets the daemon start a run after enough new sessions and a quiet period.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds dreaming: an offline pass that reads recent conversations from every agent memex indexes (Claude Code, Codex, and the rest), plus saved Claude and Codex memories, and stores what a model concludes as dreams. A dream is one statement (a correction the user keeps giving, a recurring mistake, a memory that recent work contradicts, a preference, a project fact, a lesson) together with the transcript records it rests on.
Opt-in
Everything is behind
[dream] enabled = true, which is off by default. With it off:memex dreamandmemex dream --dry-runstop with a message naming the key; no executor runs and nothing is written.autosays.--content allreturns what it did before, and--content dreamsfails saying dreaming is off.memex dream --print-promptworks either way, because it sends nothing; it shows what a run would send. Daemon scheduling is a second opt-in (auto = true) on top ofenabled.How a run works
memex dreambuilds one prompt (existing dreams, memory sections, the user and assistant messages of primary sessions since the last run; tool calls, tool output, and reasoning are not read), passes it through the egress redactor, and makes one call to an agent CLI the user already has:claude,codex, or[dream] command. Memex holds no API keys. Claude runs with--safe-mode --no-session-persistence --tools ""; Codex has no flag to remove its shell tool, so it runs in its read-only sandbox.The executor only proposes. It answers with
add/reinforce/revise/retractoperations citing short handles (r12,m3) for what it was shown, and memex decides what is written:supersededorretracted.Store
<root>/dreams/generations/<id>/holdsdreams.json, a renderedDREAMS.md,diff.md, and the executor's raw answer, with an atomicCURRENTpointer. Each run publishes a new generation and never modifies an earlier one. Claude and Codex memory files are never written.Citations and lint
A citation carries machine, record ID and its scheme, source, session, source path, native event ID, role, timestamp, a hash of the normalized text, and an inline excerpt.
memex dream lintre-finds each source by record ID in its session, then native event ID, then identical text in the session, then excerpt search (recorded asremapped). Moved IDs are rewritten withrid_prevkept; a source that is gone is markedevidence_lostand the dream keeps its excerpt; an unreachable machine isunverified, not lost; an edited memory that no longer contains the passage issource_revised. Every run lints first.Reading
With dreaming enabled,
--content allin the CLI and MCPsearchreturns dreams askind: "dream"hits beside the conversations and memories they cite, merged by reciprocal rank with at most five per search.--content dreamsreturns them alone. A project or--cwdscope returns that project's dreams plus the ones that hold across projects. Search without--contentis unchanged. The search skill and MCP guidance tell agents a dream is a conclusion to check against its cited record, not a source.Not in this change
max_sessions/max_prompt_chars; they are skipped and counted in the report.Verification
cargo fmt --checkandcargo clippy -- -D warningspass, and the full test suite passes, including new unit tests for the operation rules and the citation resolver andtests/dream_cli.rs, which drives the CLI end to end with a shell script standing in for the model. The realclaudeandcodexCLIs were checked to accept the executor flags with a trivial prompt. No dream run has been done over real history yet, so the quality of the dreams a given model produces is untested.The single commit's subject says "claims"; the entries were renamed to "dreams" before this was opened, and the code and docs use that name throughout.