Land on green, and delete an unbacked live-gate assertion instead of adding observation (#241) by ericlitman · Pull Request #242 · ericlitman/open-pstack

ericlitman · 2026-10-21T14:42:29Z

Summary

Open Pstack lands on green, and an unbacked live-gate assertion is answered by deleting it.

merge rule on main after this PR
  every PR        -draft, verify, live-gate, #review-threads-unresolved = 0
  risky path      ^(\.github/|\.mergify(\.ya?ml\Z|/))  ->  also check-success = @unfret-eal/Unfret
  Unfret submits a review  ->  unfret-threads.yml resolves every thread Unfret started
                               so only other reviewers' threads (Greptile, people) hold a merge

 queue_conditions / "Queue ready pull requests"
   -draft
   check-success=verify
-  check-success=Unfret
   check-success=live-gate
   "#review-threads-unresolved = 0"
+"Unfret on risky paths"
+  if: base = main, files ~= ^(\.github/|\.mergify(\.ya?ml\Z|/))
+  check-success = @unfret-eal/Unfret

REVIEW.md: an assertion the live gate publishes that can pass without its behavior is material, and its correction is deletion; a behavior no recipe asserts is an advisory coverage gap. The verifier entry point must back every assertion it publishes with retained evidence. A new Usage-facts contract states the merge policy.
AGENTS.md: the live-proof scope (from mastra-pilot run-these), Merge policy: land on green, the merge rule, and the release rule. A draft pull request is the pause: Mergify reads no hold label here, because no GitHub rule limits who can remove a label.
UPSTREAM.md: deletes the keep-setting sentence from the setup-pstack: on a rerun with a valid sheet and passing probes, steps 4/6/7 should take the sheet as their answers #46 entry and step 6's recipe-update sentence; step 9 carries the release rule.
CHANGES.md:3: the merge description.
.github/workflows/unfret-threads.yml: on each review Unfret submits, resolves every unresolved thread Unfret started. It needs contents: write as well as pull-requests: write, because GitHub refuses resolveReviewThread to the Actions token without it.
.claude/skills/verify-open-pstack/features/registry.json: maps the new workflow nonRuntime, because unknown paths abort the live gate and no Claude Code or Codex surface runs a GitHub workflow. Its live test is its own run on this PR. No recipe changes.

After this PR merges, this session swaps the "Review gate" ruleset (24286952) from requiring Unfret (App 5142537) to requiring Mergify Merge Protections (App 10562), the standard for ericlitman repositories. Until then the ruleset still requires Unfret, so this PR, and anything that merges before the swap, waits for it.

Design decisions and rejected alternatives are in #241's "Current scope" block.

Evidence

Mergify cannot tell thread authors apart. Mergify's simulator (POST /v1/repos/ericlitman/open-pstack/pulls/<n>/simulator) on PRs 98, 114 and 145, which have unresolved Unfret or Greptile threads: review-threads-unresolved ~= ^PRRT_ is true and review-threads-unresolved ~= unfret-finding is false on all three. The attribute holds thread IDs, so a workflow resolves Unfret's threads instead.
The new config validates and evaluates as intended (simulator with this PR's .mergify.yml, byte-identical to the committed file):
- risky-path files ~=: true on chore(tooling): delete comments from the verifier, tests and repository config (#218) #220 (.github/workflows/windows-hooks.yml, .mergify.yml) and build: add root check scripts so Unfret registers and shares open-pstack's checks (#185) #186 (.github/workflows/ci.yml); false on fix(pstack): let Claude Code subagents wait in the foreground (#232, #233) #239 and feat(pstack): sync Cursor pstack 0.15.29 as Open Pstack 1.16.0 (#234) #240.
- check-success = @unfret-eal/Unfret: true on feat(pstack): sync Cursor pstack 0.15.29 as Open Pstack 1.16.0 (#234) #240. @unfret/Unfret and @github-actions/Unfret: false. The App qualifier is what keeps a same-named check from another App out once the ruleset stops binding Unfret.
- A bogus qualifier inside merge_protections is rejected, so the simulator validates that section.
The workflow's thread query (REPOSITORY=ericlitman/open-pstack PR=98, the committed script's query and filter, read-only) returns exactly the two unresolved Unfret threads on release: sync Cursor pstack 0.15.5 as Open Pstack 1.5.0 #98 and neither of its two Greptile threads.
Thread workflow, head 7791942c: Unfret's review triggered it from this PR's own copy (run 39621500336), and it failed: resolveReviewThread returned FORBIDDEN ("Resource not accessible by integration") under pull-requests: write.
Token probe: a push-triggered workflow on a throwaway branch (deleted after), run 39622083381, tried the mutation on two old Unfret threads of merged fix(verify): accept only the resolver launcher and equalize the Reflect fixture openers (#152) #164. With pull-requests: write alone it returned FORBIDDEN; with contents: write added it returned "isResolved":true. Head 986c49aa adds contents: write.
Thread workflow, head 986c49aa: pending; Unfret's review of this head triggers it. This run is the workflow's live test.
Live gate: passed on 7791942c (evidence); passed on 986c49aa (evidence). verify-open-pstack ran from this candidate head with --feature claude:project-skill --feature codex:project-skill --self-test, because registry.json is a verifier file and AGENTS.md and registry.json reach only project-skill; the other paths are nonRuntime. Both sessions passed 4 machine-checked assertions each.

Ruleset 24286952 before the swap:

{"name":"Review gate","enforcement":"active","bypass_actors":[{"actor_id":5,"actor_type":"RepositoryRole","bypass_mode":"pull_request"}],"conditions":{"ref_name":{"exclude":[],"include":["~DEFAULT_BRANCH"]}},"rules":[{"type":"required_status_checks","parameters":{"strict_required_status_checks_policy":false,"do_not_enforce_on_create":true,"required_status_checks":[{"context":"Unfret","integration_id":5142537}]}}]}

Ruleset 24286952 after the swap (applied 2026-10-21 15:42 UTC, right after the merge, so no PR merged without Unfret before the new .mergify.yml was on main). main's required checks are now verify, live-gate and Mergify Merge Protections:

{"name":"Review gate","enforcement":"active","bypass_actors":[{"actor_id":5,"actor_type":"RepositoryRole","bypass_mode":"pull_request"}],"conditions":{"ref_name":{"exclude":[],"include":["~DEFAULT_BRANCH"]}},"rules":[{"type":"required_status_checks","parameters":{"strict_required_status_checks_policy":false,"do_not_enforce_on_create":true,"required_status_checks":[{"context":"Mergify Merge Protections","integration_id":10562}]}}]}

Merge Danger

Door: two-way. Reverting this PR and restoring the ruleset JSON above returns the old rule.

Blast Radius: merge policy.

After the ruleset swap, a PR that touches no risky path merges before Unfret's verdict. Users install from main, so a material defect reaches fresh installs before its fix-forward PR; the release rule delays only the next version bump. Every Unfret thread on every PR is resolved as Unfret posts it. Unfret's check still gates risky-path PRs, and its findings stay in the check and the review service. A PR that touches .github/** still needs an operator to queue it.

Acceptance criteria

Criterion (#241)	Verification
"Items 1 to 6 are on `main`."	This PR. Read back on `main` at merge.
"After this merges, the next PR that touches no risky path, including a verifier-only PR, merges on `verify` and `live-gate` without waiting for `Unfret`."	Deferred proof: needs the next such PR after the merge and the ruleset swap.
"On the next PR whose merged head gets a finding that a live-gate assertion can pass without its behavior, the fix-forward PR only deletes that assertion. Coverage-gap findings get no fix-forward PR."	Deferred proof: needs that finding.

Gate results (head `986c49aa`)

Check	Exit	Result
`bun run test`	0	505 pass, 0 fail
`bun run typecheck`	0	no errors
`bun run static`	0	201 pass, 0 fail (includes the verifier's unit tests and typecheck)
`claude plugin validate plugins/pstack`	0	Validation passed

no-comments: the one comment, on the workflow's contents: write, states GitHub behavior we cannot change and stays. No constraint is left unencoded.

Unfret findings on head `7791942c`

Finding	Class	Disposition
`v1:2ec9ef2e…` the hold gate trusts unauthorized label removal	`verdict-changing`	Fixed by deletion in `986c49aa`: no GitHub rule limits who removes a label, and #241 does not require `hold`.
`v1:9ac6ba0b…` fork PR tokens cannot resolve threads	`fails-loudly`	No change. Unfret leaves no thread on fork PRs (#181, #188 have none), and the repository has no Dependabot. If one ever did, the job fails and the thread holds the merge.
`v1:d191256f…` the workflow is excluded from live verification	`verdict-changing`	Fixed in `986c49aa`: the resolver failed while `live-gate` passed. Its live test is its own run on the PR. The finding's claim that a `pull_request_review` workflow must already be on `main` is false: run 39621500336 ran this PR's copy.

Not verified

That the workflow resolves Unfret's threads on this PR: the probe proved the permission; the run on 986c49aa proves the workflow.
The first land-on-green merge and the ruleset swap, which happen after this PR merges.

Follow-up filed: MASTRA-1134. babysit-pr's risky-path hold check matches only @unfret/Unfret, so babysit-pr waive refuses on an open-pstack risky-path PR until it also matches @unfret-eal/Unfret.

Closes #241

🤖 Generated with Claude Code

Mergify now merges on verify and live-gate. Only a PR that touches .github/ or the Mergify config also waits for Unfret, and a workflow resolves the threads Unfret starts so only other reviewers' threads hold a merge. REVIEW.md answers an unbacked live-gate assertion by deleting it and treats a behavior no recipe checks as an advisory. AGENTS.md carries the live-proof scope and the release rule. Closes #241 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ericlitman · 2026-10-21T14:44:01Z

Live evidence: Open Pstack

Candidate SHA: 7791942c1f494a84f903c3b151f4a47b5f759726
Base SHA: 74da6519569f7a672251782c70d96c40c4ae5186
Result: PASSED — named features verified
Self-test: completed in both harnesses
Planned sessions: 2 (limit 12)
Publisher revision: 7791942c1f494a84f903c3b151f4a47b5f759726
Evidence-set SHA-256: ecb31c7d4587fc721d74d137e9df9135ca93dc75532845ba29cd649b75deb5e3
Installation (claude): CLI 2.1.304 (Claude Code), plugin 1.16.0, candidate 7791942c1f494a84f903c3b151f4a47b5f759726, tree cb0d51f6146e5aeb33e96b9e288baa8912dcd547599421b5661345e0b635409b.
Installation (codex): CLI codex-cli 0.163.0, plugin 1.16.0, candidate 7791942c1f494a84f903c3b151f4a47b5f759726, tree cb0d51f6146e5aeb33e96b9e288baa8912dcd547599421b5661345e0b635409b.
Observation (claude / project-skill): surface claude -p stream-json with the candidate plugin-dir; action recipe project-skill: doctor; result passed: doctor:exit-0 doctor:skill-loaded doctor:doctor-ran doctor:doctor-canonical-pass; 4 assertions machine-checked; transcript SHA-256 b424ca1a6daa64ab73b8b63615ea5cad72bc0bdd9a4022a74e4b26e67cdb7c1e; artifact-set SHA-256 67e11ce898bb139121b4a4794e715c67418bcb83ee299677c0fe4f4a437cdeb5.
Observation (codex / project-skill): surface codex exec --json with the run-owned candidate installation; action recipe project-skill: doctor; result passed: doctor:exit-0 doctor:skill-loaded doctor:doctor-ran doctor:doctor-canonical-pass; 4 assertions machine-checked; transcript SHA-256 c068bdcc914c65e3189c2703b84a506db2823ad7ba165a6fb5af923378b56f88; artifact-set SHA-256 e56836dbd5d596541364503d6ce01cb6cc59c3f38a9650d773973145f57d0e1c.
Policy: every new head requires a fresh run; this verifier does not change PR state, queue, or merge.

mergify · 2026-10-21T14:49:45Z

Merge Protections

🔴 1 of 3 protections blocking · waiting on 🙋 you

	Protection	Waiting on
🔴	🚦 Auto-queue	🙋 you
🟢	Queue ready pull requests	—
🟢	Unfret on risky paths	—

🔴 🚦 Auto-queue

Waiting for

-files~=^\.github/

This rule is failing.

When all merge protections are satisfied and these conditions match, this pull request will be queued automatically.

-files~=^\.github/
base = main

Show 2 satisfied protections

🟢 Queue ready pull requests

Automatically queue non-draft pull requests targeting main after verify and live-gate pass on the head and every review thread is resolved.

#review-threads-unresolved = 0
-draft
check-success=live-gate
check-success=verify

🟢 Unfret on risky paths

Pull requests that touch .github/ or the Mergify config also wait for Unfret.

check-success = @unfret-eal/Unfret

unfret-eal · 2026-10-21T14:57:24Z

✅ Accepted risk

Accepted by ericlitman: #241 (authority 48fa77b)

🔴 High · New resolver workflow is classified nonRuntime, so live-gate passes without exercising it

🟡 Medium · Fork PRs still get a read-only token, so Unfret threads are never resolved

Details

Run: GET /unfret/run/panel:986c49aac0b90f2b1606ff943fbca6193c53c184:BMvLeB_cgc1:msgstO9P1EU.

_{@unfret review · @unfret status · @unfret help}

unfret-eal

🛑 3 blocking findings
🔴 Critical · The hold gate trusts unauthorized label removal
🔴 High · Executable workflow is excluded from live verification
🟡 Medium · Use a writable execution context for fork PR threads
Check

greptile-apps · 2026-10-21T14:59:04Z

[High impact] The PR appears safe to merge under the stated land-on-green policy.

Summary

This PR changes the repository to land ordinary PRs after verify and live-gate pass, provided there is no hold label or unresolved review thread.

Most pull requests can merge on green checks without waiting for Unfret.
The live-gate guidance limits checks to changed behavior and evidence-backed claims.
Upstream sync notes leave out old verifier recipe details.

_{Reviews (1) · Last reviewed commit: "ci, docs: land on green and delete unbac..." · Reviewed by Greptile}

The Actions token needs contents: write for resolveReviewThread: the workflow's run on 7791942 failed FORBIDDEN under pull-requests: write alone, and a probe resolved an Unfret thread once contents: write was added. Mergify cannot tell who removed a hold label, so the pause is a draft pull request instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

ericlitman · 2026-10-21T15:07:47Z

Live evidence: Open Pstack

Candidate SHA: 986c49aac0b90f2b1606ff943fbca6193c53c184
Base SHA: 74da6519569f7a672251782c70d96c40c4ae5186
Result: PASSED — named features verified
Self-test: completed in both harnesses
Planned sessions: 2 (limit 12)
Publisher revision: 986c49aac0b90f2b1606ff943fbca6193c53c184
Evidence-set SHA-256: 93e279a35b418b2f6fc4addc35198a8d19ffb9bfc0a2bbc7c16b44107a9f1dde
Installation (claude): CLI 2.1.304 (Claude Code), plugin 1.16.0, candidate 986c49aac0b90f2b1606ff943fbca6193c53c184, tree cb0d51f6146e5aeb33e96b9e288baa8912dcd547599421b5661345e0b635409b.
Installation (codex): CLI codex-cli 0.163.0, plugin 1.16.0, candidate 986c49aac0b90f2b1606ff943fbca6193c53c184, tree cb0d51f6146e5aeb33e96b9e288baa8912dcd547599421b5661345e0b635409b.
Observation (claude / project-skill): surface claude -p stream-json with the candidate plugin-dir; action recipe project-skill: doctor; result passed: doctor:exit-0 doctor:skill-loaded doctor:doctor-ran doctor:doctor-canonical-pass; 4 assertions machine-checked; transcript SHA-256 378e1513f425689a0bef84852a045fd9c98d80f00666c549160209acedac284e; artifact-set SHA-256 e7306010bb4d39bb169367562f85809ceaf0e84cbbb53e4944463d5da559130c.
Observation (codex / project-skill): surface codex exec --json with the run-owned candidate installation; action recipe project-skill: doctor; result passed: doctor:exit-0 doctor:skill-loaded doctor:doctor-ran doctor:doctor-canonical-pass; 4 assertions machine-checked; transcript SHA-256 9b3c97f54d4b2ccc4aff4bede7928c2bc6d9e2c5c57634a6f1e3743fc6ecc667; artifact-set SHA-256 866b95b7be192237b359aac7a05c97a7ca38cbd6d463c7c707c5147b60663a2e.
Policy: every new head requires a fresh run; this verifier does not change PR state, queue, or merge.

unfret-eal

🛑 2 blocking findings
🔴 High · New resolver workflow is classified nonRuntime, so live-gate passes without exercising it
🟡 Medium · Fork PRs still get a read-only token, so Unfret threads are never resolved
Check

ericlitman · 2026-10-21T15:20:38Z

Operator waiver for head 986c49a (#241).

No open blocking finding is material. Both fail closed: each failure holds a merge until someone resolves a thread by hand, and none merges anything wrongly.

v1:9ac6ba0b…, "Fork PRs still get a read-only token, so Unfret threads are never resolved". Class: fails-loudly. On a fork PR the resolve job fails, and Unfret's thread stays unresolved, so #review-threads-unresolved = 0 holds the merge. Unfret left no thread on the repository's two fork PRs (docs(skills): require a smallest-change option, rewrite relays for the user, widen interrogate triggers #181, feat(skills): add in-your-own-words and sharpen from Cursor pstack 0.15.24 (#187) #188), and the repository has no Dependabot configuration.
v1:d191256f…, "New resolver workflow is classified nonRuntime, so live-gate passes without exercising it". Class: fails-loudly. The resolver's live test is its own run on this PR: Actions run 39624429134 on this head succeeded, and github-actions[bot] resolved all three Unfret threads on Land on green, and delete an unbacked live-gate assertion instead of adding observation (#241) #242. No Claude Code or Codex surface runs a GitHub workflow, so live-gate cannot exercise it. If a later change broke the resolver, Unfret's threads would stay unresolved and Mergify's thread condition would hold every PR until someone resolved them; that PR itself touches .github/, so it also waits for Unfret and an operator queue.

No consequence is security, secret exposure, data loss, or a gate or waiver that publishes without its run.

ericlitman · 2026-10-21T15:26:22Z

@Mergifyio queue

ericlitman · 2026-10-21T15:27:18Z

@Mergifyio refresh

mergify · 2026-10-21T15:40:32Z

mergify · 2026-10-21T15:41:34Z

refresh

✅ Pull request refreshed

ericlitman mentioned this pull request Oct 21, 2026

Live gate: delete an unbacked assertion instead of adding observation, and land on green with the verifier off the risky-path list #241

Closed

unfret-eal Bot reviewed Oct 21, 2026

View reviewed changes

Comment thread .mergify.yml Outdated

Comment thread .github/workflows/unfret-threads.yml

Comment thread .claude/skills/verify-open-pstack/features/registry.json

unfret-eal Bot reviewed Oct 21, 2026

View reviewed changes

ericlitman added the unfret:986c49aac0b90f2b1606ff943fbca6193c53c184 label Oct 21, 2026

mergify Bot added the queued label Oct 21, 2026

mergify Bot merged commit 9046eec into main Oct 21, 2026
8 checks passed

mergify Bot removed the queued label Oct 21, 2026

ericlitman mentioned this pull request Oct 21, 2026

Measure Greptile's review-on-every-push trial against Unfret (2026-10-15 to 2026-10-29) #189

Open

8 tasks

Conversation

ericlitman commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Evidence

Merge Danger

Acceptance criteria

Gate results (head 986c49aa)

Unfret findings on head 7791942c

Not verified

Uh oh!

ericlitman commented Oct 21, 2026

Live evidence: Open Pstack

Uh oh!

mergify Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Protections

🔴 🚦 Auto-queue

🟢 Queue ready pull requests

🟢 Unfret on risky paths

Uh oh!

unfret-eal Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Accepted risk

Uh oh!

unfret-eal Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

ericlitman commented Oct 21, 2026

Live evidence: Open Pstack

Uh oh!

unfret-eal Bot left a comment

Choose a reason for hiding this comment

Uh oh!

ericlitman commented Oct 21, 2026

Uh oh!

ericlitman commented Oct 21, 2026

Uh oh!

ericlitman commented Oct 21, 2026

Uh oh!

mergify Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Queue Status

Uh oh!

mergify Bot commented Oct 21, 2026

✅ Pull request refreshed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ericlitman commented Oct 21, 2026 •

edited

Loading

Gate results (head `986c49aa`)

Unfret findings on head `7791942c`

mergify Bot commented Oct 21, 2026 •

edited

Loading

unfret-eal Bot commented Oct 21, 2026 •

edited

Loading

greptile-apps Bot commented Oct 21, 2026 •

edited

Loading

mergify Bot commented Oct 21, 2026 •

edited

Loading