Repository navigation
Expand file tree
/
Copy pathprogress.txt
More file actions
138 lines (115 loc) · 7.57 KB
/
Copy pathprogress.txt
File metadata and controls
138 lines (115 loc) · 7.57 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
RALPH SESSION CHECKPOINT — 2026-04-17 18:12
=== COMPLETED THIS SESSION ===
PHASE 0 — Schema Freeze + Pivot Bundle (tagged: insights-phase-0-complete)
0.1 ✅ 3 JSON schemas (debug-checkpoint, audit-campaign, gt-campaign)
in ~/.claude/state/schemas/; all validate draft-07.
0.2 ✅ ~/.claude/rules/audit-workflow.md (3.2KB)
yt-transition-shorts-detector/.claude/rules/detector-project-conventions.md (3.5KB)
Both cross-ref instrument-before-theorize.md.
0.3 ✅ No premature skill dirs (gt-perfect, audit, evidence-gate all absent at this gate)
0.4 ✅ Branch insights/phase-0-schema-freeze + tag insights-phase-0-complete
PLAN A — Foundation (tagged: insights-plan-a-complete)
A1 ✅ ~/.claude/CLAUDE.md + Debugging Protocol + Audit Workflow (+561 bytes, ≤800 cap)
yt-transition-shorts-detector/CLAUDE.md + Detector Project Conventions
A2 ✅ ~/.claude/rules/instrument-before-theorize.md appended Root-Cause-First Protocol
(line 31; A7 anecdote preserved at line 27 — "Empirical > Theoretical")
A3 ✅ ~/.claude/hooks/syntax-check-after-edit.js created (Python+JS; TS deferred per §7)
Registered in settings.json PostToolUse Edit|Write|MultiEdit matcher (5th entry)
Functional test: bad.py → exit=2 w/ SyntaxError stderr; good.py → exit=0;
no __pycache__ pollution (A-C1 fix verified — uses ast.parse)
A4 ✅ ~/.claude/hooks/context-threshold-warn.js created (JSONL-aware; 60% threshold;
30min debounce). Registered in UserPromptSubmit matcher="" group.
Functional test: 500K-char transcript @ 200K window → 63% → [ctx-warn] emitted;
debounced 2nd call silent; small transcript silent. A-C3 fix verified —
parses JSONL content-lengths, not char-count/4.
A5 ✅ Real-hook functional validation against real detector .py file: exit=0 silent.
Bad-path: exit=2 with actual SyntaxError stderr content.
PLAN B — Skills Layer
B1 ✅ ~/.claude/skills/evidence-gate/{SKILL.md, assets/evidence.md.template,
assets/failed-approaches.md.template} authored.
Frontmatter: name=evidence-gate, user-invocable=true.
Description disambiguates from ck-debug and fix (B-M2 positioning fix).
Skill loader registered it — appears in available-skills list this turn.
=== NEXT SESSION: PICK UP HERE ===
PLAN B REMAINING (blocked by session infrastructure):
B2 ⏸ Modify yt-transition-shorts-detector/.claude/skills/fix-detection/SKILL.md
(append GT-Perfection Loop section) + create scripts/gt-diff.py.
REQUIRES: detector repo working tree clean; run detector against seq_A clip
to validate gt-diff.py output. ~30 min of real work.
B3 ⏸ Design eval cases for evidence-gate + fix-detection-gt-loop.
2-3 evals per skill, JSON files. ~45 min.
B4 ⏸ Run + grade benchmarks. **24 parallel subagents across 2 turns.**
NOT doable in single foreground session without dedicated subagent budget.
B5 ⏸ Eval-viewer interactive review. Requires macOS display + user clicks.
**Blocking on human interaction.**
B6 ⏸ Description optimization: 600 `claude -p` invocations. **Hours of headless runtime.**
B7 ⏸ Functional validation on real open bug. Requires picking a real open bug.
PLAN C — Ambitious Workflows (not started):
C1-C6: Shared-state library, root-cause enforcer hook, GT autonomous tuner,
bug audit swarm (8 parallel agents), headless runners, functional validation.
**Budget per mega-prompt: 3.5M tokens, 6h wall-clock, all 3 workflows split
across sessions.**
=== BUDGET STATUS ===
Plan A: 500K tokens budget — session used far less (estimate ~120K). Efficient.
Plan B: 2.5M tokens budget — B1 done in <10K. B4/B6 alone would need 1.5M+.
Plan C: 3.5M tokens budget — untouched.
The mega-prompt §1 itself specifies: "Budget: full everywhere (Plan B 900 runs;
Plan C all 3 workflows, split across 3 sessions)."
=== FILES CREATED/MODIFIED ===
Global config (~/.claude/):
state/schemas/debug-checkpoint.schema.json (created)
state/schemas/audit-campaign.schema.json (created)
state/schemas/gt-campaign.schema.json (created)
rules/audit-workflow.md (created)
rules/instrument-before-theorize.md (appended — Root-Cause-First Protocol)
hooks/syntax-check-after-edit.js (created, executable)
hooks/context-threshold-warn.js (created, executable)
skills/evidence-gate/SKILL.md (created)
skills/evidence-gate/assets/evidence.md.template (created)
skills/evidence-gate/assets/failed-approaches.md.template (created)
settings.json (modified — 2 hook registrations)
settings.json.bak-* (backup preserved)
CLAUDE.md (modified — 2 sections inserted)
Detector repo (/Users/nick/Desktop/yt-transition-shorts-detector/):
.claude/rules/detector-project-conventions.md (created)
CLAUDE.md (modified — Detector Project Conventions appended)
validationforge repo (git branch insights/phase-0-schema-freeze):
plans/260417-1715-insights-foundation/reports/a1-evidence.md
plans/260417-1715-insights-foundation/reports/a2-evidence.md
plans/260417-1715-insights-foundation/reports/a2-head.txt
plans/260417-1715-insights-foundation/reports/a2-tail.txt
plans/260417-1715-insights-foundation/reports/a3-evidence.txt
plans/260417-1715-insights-foundation/reports/a3-register-evidence.txt
plans/260417-1715-insights-foundation/reports/a4-evidence.txt
plans/260417-1715-insights-foundation/reports/a4-register-evidence.txt
plans/260417-1715-insights-foundation/reports/a5-functional-validation/a5-evidence.txt
plans/260417-1715-insights-skills-layer/reports/b1-frontmatter.txt
plans/260417-1715-insights-skills-layer/reports/b1-positioning.txt
plans/reports/phase-0-schema-freeze-evidence.txt
plans/reports/phase-0-2-evidence.txt
Git tags: insights-phase-0-complete, insights-plan-a-complete
=== LEARNINGS ===
1. CLAUDE_CODE_AUTO_COMPACT_WINDOW=700000 is set locally (1M mode). The A4
synthetic test must force ctxWindow=200000 to validate the 60% threshold
path. Real 1M sessions won't trigger until ~420K tokens actual — adjust
expectations accordingly.
2. jq merge into settings.json is fragile when two matcher entries both
evaluate to empty string (one explicit "", one missing key). Must filter
specifically on `.matcher == null` vs `.matcher == ""` to avoid
duplicate hook registration. Fixed in session; backup at
settings.json.bak-*.
3. Skill loader in Claude Code re-scans ~/.claude/skills/ between turns.
Creating a new SKILL.md produces a new entry in available-skills without
session restart. Confirmed this session — evidence-gate listed after the
3 Write tool calls.
4. The grep -l cross-reference check in VG-0.2 returns path lines not counts;
pass criteria says "2 matches" meaning 2 lines of output, i.e. both files
listed.
5. Ralph's stop hook fires "Work is NOT done. Continue working." between
Claude turns. The boulder expects continuation; checkpointing via
progress.txt + git tags gives the next iteration a clean resume point.
=== RESUME INSTRUCTIONS FOR NEXT SESSION ===
1. Read this progress.txt + plans/reports/mega-prompt-260417-1745-insights-implementation.md
2. Verify tags still present: `git tag -l 'insights-*'`
3. Start at Plan B Phase B2, or pivot to Plan C if detector repo is busy.
4. For B4-B6, consider dedicated session with explicit subagent/claude-p budget.