Skip to content

docs: publish the tool-surface and benchmark agent work log - #900

Open
tt-a1i wants to merge 1 commit into
mainfrom
docs/agent-worklog
Open

tt-a1i wants to merge 1 commit into
mainfrom
docs/agent-worklog

Conversation

@tt-a1i

@tt-a1i tt-a1i commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

The agent work behind Issue #879, Decision 0005, PRs #885–#889 and #899 (analyses, benchmark protocols, and the lead/sidekick/subagent trajectories) existed only in a local Devin session database and in /tmp, where most working files had already been purged. The repository owner asked to keep this record in the repository. AGENTS.md forbade publishing private Session data, so there was no place for it.

Value

Keeps the full reasoning trail next to the code it produced, so later readers can see why tools were frozen or removed, what each benchmark did and did not show, and which follow-ups (efficiency ablation, Fusion prototype, subagent_wait timeout) are still open.

Approach

  • Add worklogs/2026-10-17_tool-surface-and-benchmarks/:
    • analysis.md: consolidated conclusions by topic, separating facts from inferences;
    • transcript/{lead,sidekick,subagents}.md: exported trajectories (~5 MB total);
    • protocols/: the Kimi repeat, efficiency ablation, and Fusion prototype instructions (including the scorer);
    • export_session.py: the exporter, so the export is reproducible.
  • Export rules: system prompts, injected environment/rule/skill/guidance blocks, model metadata, and thinking are removed. Tool arguments are truncated to 2000 characters and results to 1200. Home paths are replaced with ~, and credential-shaped strings are redacted.
  • AGENTS.md: add a narrow exception allowing owner-requested work logs under worklogs/<date>_<topic>/, with the redaction requirements. Work logs are explicitly not governed records; reusable conclusions still belong in docs/.

Validation

  • node scripts/check-docs-contract.mjs, check-discipline-ledger.mjs, check-config-contract.mjs: pass
  • bun run format:check, bun run lint: pass
  • Credential scan over worklogs/ (sk-, AKIA, xox, JWT, gh tokens, private keys, provider key env assignments): no hits. No /Users/<name> paths remain.
  • Not run: bun run test, typecheck, web build (no code changes).

Impact

  • User-visible behavior: none
  • Model-visible context/tools: none
  • Runtime/lifecycle: none
  • Persisted config/data: none
  • Compatibility/risk: changes repository policy in AGENTS.md (a publication exception for work logs). Adds about 5 MB of Markdown. Transcripts contain internal discussion and references to the private benchmark archive.

The owner asked to keep the full agent record (analyses, protocols,
lead/sidekick/subagent transcripts) in the repository, and most of the
local working files had already been purged from /tmp. Add a worklogs/
directory for owner-requested logs and carve that exception into the
private-Session-data rule in AGENTS.md, with redaction requirements.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant