Repository navigation
feat(spec-loop): require evidence before the plan beat queues a Known gap - #1941
Merged
Merged
Conversation
… gap The plan beat derived work items from Known-gaps bullets, and nothing made it check that a bullet was still true, so a bullet describing shipped code sent the loop at finished work. This is the prompt half of spec-gap-staleness.md (acceptance 1 and 2). - PROMPT_plan.md step 4: a gap becomes a work item only with evidence it is still open, of three kinds (call site, file content, counted value), written into the item as a `*Verified open:*` line. A name match is explicitly not evidence. An unverifiable bullet is neither queued nor deleted but listed under Notes & discoveries as unverified. A bullet with a `gap-closes-when` predicate defers to spec-validate. Step 5 now names the evidence line as part of every work item. - PROMPT_update.md: every Known-gaps bullet the update beat keeps must be re-verified the same way; a closed one is removed or rewritten, an unverifiable one is kept and marked, and a gap with a crisp end state gets a predicate so spec-validate checks it from then on. - The spec-loop fixture suite pins the three evidence kinds and the no-name-match rule in both prompts. Generated-by: Claude Code (Opus 5.5)
… experimental Review fixups: - With the validator half merged (#1940) and the prompts now stating the evidence rule, the spec's "evidence rule in the prompts is not built" Known gap described shipped work: removed, and the spec moves from proposed to experimental (here and in the overview table). - PROMPT_update.md: symbol-called reads Python sources only, so a gap about a shell function gets path-exists or none (narrative) instead of a predicate the validator reports as invalid. Generated-by: Claude Opus 5.5
potiuk
force-pushed
the
plan-beat-evidence-rule
branch
from
November 1, 2026 12:49
e899711 to
86052d8
Compare
potiuk
approved these changes
Nov 1, 2026
potiuk
left a comment
Member
There was a problem hiding this comment.
Thanks — the evidence rule reads well and matches the predicate vocabulary #1940 shipped. I rebased the branch onto main and pushed one small fixup commit, 86052d8 (please pull before pushing again):
tools/spec-loop/specs/spec-gap-staleness.md— with #1940 merged and this PR putting the rule in both prompts, the "evidence rule in the prompts is not built" Known gap described shipped work, the exact staleness the spec exists to prevent. Removed, and the spec moves fromproposedtoexperimental(here and in the overview table), as the PR body anticipated once both halves landed.PROMPT_update.md—symbol-calledreads Python sources only (a directory without*.pyisinvalid), so the update beat now usespath-existsornone (narrative)for a gap about a shell function, which most ofspec-loopis.
GitHub has not yet computed whether this branch merges cleanly with its base.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.More on how Apache Magpie handles maintainer review:
Contributing guide.
9 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
specs/spec-gap-staleness.mddocuments). This is the prompt half of that spec, acceptance criteria 1 and 2.PROMPT_plan.md: a gap becomes a work item only with evidence it is still open: a call site, file content, or a counted value, written into the item as a*Verified open:*line. A name match is explicitly not evidence. An unverifiable bullet is neither queued nor deleted but listed under Notes & discoveries as unverified. A bullet with agap-closes-whenpredicate defers tospec-validate.PROMPT_update.md: every Known-gaps bullet the update beat keeps must be re-verified the same way; a closed one is removed or rewritten, an unverifiable one is kept and marked, and a gap with a crisp end state gets a predicate.Plan item
plan-beat-evidence-rule.Type of change
prek, workflows, validators)Test plan
grep -qi evidenceon both prompts,bash -n tools/spec-loop/loop.sh.test_prompts_carry_the_evidence_rulepins the three evidence kinds, the predicate syntax, and the no-name-match rule in both prompts. Mutation-checked: with the old prompts it fails.bash tools/spec-loop/tests/test_runner_fixtures.shpasses;shellcheckfindings unchanged frommain.prek run --all-files --skip workspace-pytest: every hook passes exceptskill-token-count(tokenizer download blocked in the authoring sandbox; noSKILL.mdtouched). RAT: 0 unapproved.RFC-AI-0004 compliance
Not applicable (prompt text only).
Linked issues
Companion to the
spec-validator-gap-checksPR, which builds thegap-closes-whenchecks these prompts refer to. Independent of it: neither touches the other's files.Notes for reviewers (optional)
spec-gap-staleness.mdis deliberately not edited here; the validator PR rewrites its Known-gaps bullet, and editing it in both would conflict. Once both land, the spec can move fromproposedtoexperimental.*Verified open:*lines in this shape.Generated-by:trailer.