Measure Greptile's review-on-every-push trial against Unfret (2026-10-15 to 2026-10-29) #189

Goal

Decide whether Greptile earns its place on this repository after a two-week trial of reviews on every push.

Why

An assessment on 2026-10-15 adjudicated all 88 Greptile inline findings from PR 29 to PR 184 against the code and against Unfret. On Eric's and mast-factory's PRs from 2026-09-30 to 2026-10-14 (the period when both ran):

• Greptile posted 58 findings. 13 were material, and Unfret reported 11 of those 13 on the same head.
• Greptile alone drove 9 fixes. One was material: ci: queue PRs in Mergify once verify and Unfret pass #101's "PR queues before live verdict", which became the exact-head live-gate requirement.
• Unfret reported 229 distinct defects on those PRs. Greptile matched 25 of them.
• 19 of 88 findings were not defects, or were upstream-owned or pre-existing behavior.
• Greptile reviewed only the head that each PR opened with. Unfret finds most of its defects on later fix heads.

Since #163 (2026-10-12), Mergify queues a PR only when #review-threads-unresolved = 0. Each Greptile thread now blocks the queue until someone resolves it.

Trial settings (changed by Eric on 2026-10-15)

• Automatic reviews: on new pushes (was: on PR opened).
• Default review tier: Auto.
• Custom Instructions added, Confidence Score off, Sequence Diagram off (2026-10-15; see the comment below).
• Filters: none. Outside contributors' PRs are allowed again, so Greptile reviews them too.
• Any later change gets a dated note on this issue.

Acceptance criteria

• Re-run the assessment over every PR opened from 2026-10-15 to 2026-10-29. Use the same rubric and scripts as the 2026-10-15 record. Adjudicate every Greptile finding against the code and against Unfret. Report two groups: Eric's and mast-factory's PRs, and outside contributors' fork PRs.
• Outside fork PRs get no Unfret check (seen on docs(skills): require a smallest-change option, rewrite relays for the user, widen interrogate triggers #181 and feat(skills): add in-your-own-words and sharpen from Cursor pstack 0.15.24 (#187) #188), so Greptile is their only automated reviewer. For that group, report validity and materiality of each finding, and whether the contributor acted on it.
• Split Greptile findings by head: the opening head vs later heads. A finding on a later head is the trial's effect.
• For each later-head finding, report validity, materiality, the Unfret match (same defect, related, or none), and who drove the fix.
• Report Greptile threads per PR. Report the time from the last required check passing (verify, Unfret, live-gate) to the last Greptile thread being resolved. That time is the merge delay Greptile causes under Require resolved review threads before Mergify queues a PR #163.
• Compare the not-a-defect plus upstream rate with the 19-of-88 baseline.
• Record Greptile credit use for the period from Greptile's Usage page.
• Recommend one option: keep reviews on every push, return to reviews when a PR opens, or remove Greptile. Give the evidence for the recommendation.

Not a code change. This issue closes when the comparison is posted on it and Eric picks an option.