Skip to content

Add opt-in dreaming: cited dreams consolidated from agent history - #282

Merged
nicosuave merged 2 commits into
mainfrom
cross-tool-dreaming
Nov 10, 2026
Merged

nicosuave merged 2 commits into
mainfrom
cross-tool-dreaming

Conversation

@nicosuave

@nicosuave nicosuave commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Adds dreaming: an offline pass that reads recent conversations from every agent memex indexes (Claude Code, Codex, and the rest), plus saved Claude and Codex memories, and stores what a model concludes as dreams. A dream is one statement (a correction the user keeps giving, a recurring mistake, a memory that recent work contradicts, a preference, a project fact, a lesson) together with the transcript records it rests on.

- Use bun, never npm, in the shop repository. `drm_3f9a…` (evidence supported)
  - 2026-11-09 claude user on local · `memex show --record-id rid1_… --machine local`
    > always use bun in this repo, never npm.

Opt-in

Everything is behind [dream] enabled = true, which is off by default. With it off:

  • memex dream and memex dream --dry-run stop with a message naming the key; no executor runs and nothing is written.
  • The daemon schedules no runs, whatever auto says.
  • --content all returns what it did before, and --content dreams fails saying dreaming is off.

memex dream --print-prompt works either way, because it sends nothing; it shows what a run would send. Daemon scheduling is a second opt-in (auto = true) on top of enabled.

How a run works

memex dream builds one prompt (existing dreams, memory sections, the user and assistant messages of primary sessions since the last run; tool calls, tool output, and reasoning are not read), passes it through the egress redactor, and makes one call to an agent CLI the user already has: claude, codex, or [dream] command. Memex holds no API keys. Claude runs with --safe-mode --no-session-persistence --tools ""; Codex has no flag to remove its shell tool, so it runs in its read-only sandbox.

The executor only proposes. It answers with add / reinforce / revise / retract operations citing short handles (r12, m3) for what it was shown, and memex decides what is written:

  • A handle memex did not offer is dropped; an operation left without evidence is rejected and listed in the report.
  • A dream needs at least one conversation record. A saved memory alone is not new ground truth (a contradiction may instead cite two sources of any kind).
  • Existing dreams are not evidence, and transcripts of dream runs are never read back.
  • A dream the executor does not mention carries over unchanged. Revising or retracting needs new record evidence, and the old dream stays as superseded or retracted.
  • Excerpts and text hashes are cut from the index, not taken from the model.

Store

<root>/dreams/generations/<id>/ holds dreams.json, a rendered DREAMS.md, diff.md, and the executor's raw answer, with an atomic CURRENT pointer. Each run publishes a new generation and never modifies an earlier one. Claude and Codex memory files are never written.

Citations and lint

A citation carries machine, record ID and its scheme, source, session, source path, native event ID, role, timestamp, a hash of the normalized text, and an inline excerpt. memex dream lint re-finds each source by record ID in its session, then native event ID, then identical text in the session, then excerpt search (recorded as remapped). Moved IDs are rewritten with rid_prev kept; a source that is gone is marked evidence_lost and the dream keeps its excerpt; an unreachable machine is unverified, not lost; an edited memory that no longer contains the passage is source_revised. Every run lints first.

Reading

With dreaming enabled, --content all in the CLI and MCP search returns dreams as kind: "dream" hits beside the conversations and memories they cite, merged by reciprocal rank with at most five per search. --content dreams returns them alone. A project or --cwd scope returns that project's dreams plus the ones that hold across projects. Search without --content is unchanged. The search skill and MCP guidance tell agents a dream is a conclusion to check against its cited record, not a source.

Not in this change

  • Writing dreams back into Claude or Codex memory, or injecting them at session start.
  • Lexical or embedding clustering of candidates before the model; the model sees recent sessions, memories, and existing dreams directly.
  • Semantic matching and federation for dream search: dreams are matched by keyword and read from the local store.
  • Carrying over sessions beyond max_sessions / max_prompt_chars; they are skipped and counted in the report.

Verification

cargo fmt --check and cargo clippy -- -D warnings pass, and the full test suite passes, including new unit tests for the operation rules and the citation resolver and tests/dream_cli.rs, which drives the CLI end to end with a shell script standing in for the model. The real claude and codex CLIs were checked to accept the executor flags with a trivial prompt. No dream run has been done over real history yet, so the quality of the dreams a given model produces is untested.

The single commit's subject says "claims"; the entries were renamed to "dreams" before this was opened, and the code and docs use that name throughout.

A dream run (`memex dream`) sends redacted excerpts of recent sessions,
saved Claude and Codex memories, and the existing dreams to an agent CLI
the user already has (claude, codex, or a configured command) and stores
what it concludes as dreams: single statements, each citing the
transcript records it rests on.

Everything is behind `[dream] enabled = true`, off by default. With it
off no run starts, the daemon schedules none, and search returns no
dreams.

- Memex decides what is written. The executor cites short handles for
  the records it was shown; a dream needs at least one conversation
  record memex actually offered, excerpts and hashes are cut from the
  index, and a dream the executor does not mention carries over
  unchanged. Revising or retracting needs new record evidence.
- The store is immutable generations under `<root>/dreams/` with an
  atomic CURRENT pointer and a diff per generation. Claude and Codex
  memory files are never written.
- `memex dream lint` re-finds each citation by record ID, native event
  ID, identical text in the session, then excerpt search, rewrites moved
  IDs, and marks lost evidence instead of dropping it.
- `--content all` (CLI and MCP) returns dreams beside the conversations
  and memories they cite; `--content dreams` returns them alone.
- `[dream] auto = true` lets the daemon start a run after enough new
  sessions and a quiet period.
@nicosuave
nicosuave merged commit 057416c into main Nov 10, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant