Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 29 additions & 33 deletions .github/scripts/pull-request-dashboard/RATIONALE.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,15 +414,14 @@ the implementation understandable and operationally cheap.
check suite on a PR that was already with its reviewers from looking like a
stalled handoff. A push clears the clock, because new code means new checks
and a review that has to run again.
- Only a gate that has reported nothing on the current head runs the clock. A
Copilot review that covers the head but left open findings holds the PR, yet
it is not missing: it answered, and the threads it left are the author's to
clear. Because the clock is carried across a trip back to the author, counting
those findings would make every author who takes more than four hours over
review comments look like a gate GitHub lost, and the report would say a
review never arrived when it is sitting on the PR. A review that is missing or
that only covers older code still counts, because Copilot has said nothing
about the code being reviewed.
- Only a gate that has reported nothing on the current head runs the clock.
Copilot findings that still need author action hold the PR through discussion
routing, but the review is not missing: it answered. Because the clock is
carried across a trip back to the author, counting those findings would make
every author who takes more than four hours over review comments look like a
gate GitHub lost, and the report would say a review never arrived when it is
sitting on the PR. A review that is missing or that only covers older code
still counts, because Copilot has said nothing about the code being reviewed.
- An expired hold routes the pull request and says so in its status comment,
which is the whole of what the dashboard can do about it. The gates raise no
alarm of their own. Each way a gate can go missing has its own cause and none
Expand All @@ -446,26 +445,23 @@ the implementation understandable and operationally cheap.
would hold every ready PR with its author waiting for a review that never
runs, so only branches with automatic review are listed and PRs targeting
other branches route normally.
- Every unresolved, non-outdated thread Copilot started is an author action,
even after an author reply or a later clean review. This ownership bypasses
generic praise and author-reply classification and ends only when the thread
is resolved or outdated. The gate therefore never needs its expiration path
to retain these findings.
- Findings are counted from unresolved, non-outdated review threads Copilot
started, not from the comment count on its review. A review's comment count
never shrinks, so it keeps counting feedback the author has since addressed
and holds the PR on work that is already done.
- The gate's re-request path covers two states. A stale review means the author
pushed, which is the one change a re-review can respond to. Findings on the
current head sit on unchanged code, so asking Copilot to look at it again
would reach the same verdict and be requested again on the next pass; those
threads stop counting only when the author resolves them or GitHub marks them
outdated after the referenced code moves. A later clean review does not close
an existing thread. A push makes the review stale and triggers a re-request,
but the push alone does not prove that a finding was fixed.
- The other state is a missing first review. The dashboard requests it on the
first eligible refresh using its GitHub App installation token. Initial reviews
and re-reviews share the same delivery checks and confirmation path.
- Copilot threads use the same discussion classification as human review
threads. A completed author reply hands the action to reviewers; a commitment
to more work keeps it with the author. Explicit resolution is not required
for that handoff because a reply explains the author's response without
requiring a separate administrative step. The thread remains visible as
unresolved until GitHub marks it resolved or outdated.
- The gate waits only for a missing or stale review. Counting open threads
independently would override discussion classification and keep completed
author replies assigned to the author. Actionable findings already keep the
PR with its author through ordinary discussion routing.
- A stale review means the author pushed, so a re-review can inspect new code.
Findings on the current head do not trigger re-review requests: another
review is not a substitute for responding to the finding. A later clean
review alone does not close an existing thread or change its pending action.
- The dashboard requests a missing first review on the first eligible refresh
using its GitHub App installation token. Initial reviews and re-reviews share
the same delivery checks and confirmation path.
- Copilot's generic error notice is a failed attempt, not a completed review or
delivery confirmation. It leaves the review missing or stale so the next
eligible refresh can retry on the same head without changing the requester.
Expand All @@ -474,15 +470,15 @@ the implementation understandable and operationally cheap.
from GitHub reviews, not refreshes, and a new head starts a fresh budget.
Delivery rechecks the live count so an error arriving after evaluation cannot
cause another request. Pending requests still prevent duplicates. Required
checks and unresolved Copilot findings still apply; the status comment
explains when routing stops waiting after repeated errors. The four-hour
timeout remains a backstop for reviews that never report.
checks and Copilot findings needing author action still apply; the status
comment explains when routing stops waiting after repeated errors. The
four-hour timeout remains a backstop for reviews that never report.
- GitHub's quota-exhausted notice is a failed attempt even though GitHub records
it as a review of the current head. It does not count as a completed review
or confirm delivery. Without a configured fallback token, it immediately
releases the Copilot gate and disables automatic requests for that PR,
including after later pushes. Required checks and actionable feedback still
apply, including unresolved, non-outdated Copilot threads.
apply, including Copilot threads that still need author action.
- The optional `COPILOT_REVIEW_FALLBACK_TOKEN` changes the requester after quota
exhaustion and keeps the Copilot gate active under its normal rules. The
fallback records existing quota notice IDs before requesting, so only a new
Expand Down
15 changes: 2 additions & 13 deletions .github/scripts/pull-request-dashboard/classification_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,6 @@ class ClassificationDiscussion:
pr_author: str = ""
source_kind: str = ""
candidate_feedback: tuple[CandidateFeedback, ...] = ()
strict_author_action: bool = False
selected_comment_index: int | None = None
selected_activity_timestamp: str = ""
ignored_comment_index: int | None = None
Expand Down Expand Up @@ -465,7 +464,6 @@ def from_record(cls, record: Mapping[str, Any]) -> ClassificationDiscussion:
pr_author=str(record.get("pr_author") or ""),
source_kind=str(record.get("source_kind") or ""),
candidate_feedback=candidate_feedback,
strict_author_action=bool(record.get("strict_author_action")),
)

def with_comments(
Expand Down Expand Up @@ -1701,7 +1699,7 @@ def prepare_praise_candidates(
return tuple(
discussion
for discussion in discussions
if not discussion.strict_author_action and _could_be_praise(discussion)
if _could_be_praise(discussion)
)


Expand Down Expand Up @@ -1781,16 +1779,7 @@ def resolve_review_thread_policy(
selected.activity_timestamp
or selected.comment.timestamp
)
if discussion.strict_author_action:
resolved[discussion_id] = ClassificationSuccess(
discussion.identity,
ActionDecision(
DiscussionAction.AUTHOR,
"This unresolved Copilot finding remains author work until "
"the thread is resolved or outdated.",
),
)
elif dropped and not comments:
if dropped and not comments:
resolved[discussion_id] = ClassificationSuccess(
discussion.identity,
ActionDecision(
Expand Down
58 changes: 9 additions & 49 deletions .github/scripts/pull-request-dashboard/copilot_review.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
Check,
Review,
ReviewRequest,
ReviewThread,
normalize_review_requests,
normalize_reviews,
)
Expand Down Expand Up @@ -100,48 +99,11 @@ def copilot_reviews_disabled(facts: DashboardFacts) -> bool:
)


def open_copilot_findings(
review_threads: Sequence[ReviewThread],
) -> tuple[ReviewThread, ...]:
# A review's own comment count never shrinks, so it still counts findings
# the author has since addressed. Unresolved threads are the live ones:
# GitHub marks a thread outdated once its anchor lines change, which is how
# the rest of the dashboard already recognises a pushed fix.
return tuple(
thread
for thread in review_threads
if (
not thread.is_resolved
and not thread.is_outdated
and thread.comments
and is_copilot_reviewer(thread.comments[0].actor)
)
)


def open_copilot_finding_count(
review_threads: Sequence[ReviewThread],
) -> int:
return len(open_copilot_findings(review_threads))


def open_copilot_finding_urls(
review_threads: Sequence[ReviewThread],
) -> tuple[str, ...]:
urls: list[str] = []
for thread in open_copilot_findings(review_threads):
url = thread.comments[0].url
if url and url not in urls:
urls.append(url)
return tuple(urls)


def copilot_review_status(
reviews: Sequence[Review],
head_sha: str,
review_threads: Sequence[ReviewThread],
) -> tuple[bool, bool, bool]:
"""Return (review exists, review is stale, review left open findings)."""
) -> tuple[bool, bool]:
"""Return (review exists, review is stale)."""
copilot_reviews = [
review
for review in reviews
Expand All @@ -150,16 +112,16 @@ def copilot_review_status(
and not is_copilot_error(review)
]
if not copilot_reviews:
return False, False, False
return False, False
if not head_sha:
# Without a head to compare against, staleness is unknowable, and
# acting on incomplete data would hold the pull request on a guess.
return True, False, False
return True, False
stale = not any(
review.commit_id == head_sha
for review in copilot_reviews
)
return True, stale, open_copilot_finding_count(review_threads) > 0
return True, stale


def copilot_review_outstanding(facts: DashboardFacts, *, enabled: bool) -> bool:
Expand All @@ -170,10 +132,9 @@ def copilot_review_outstanding(facts: DashboardFacts, *, enabled: bool) -> bool:

def copilot_review_unreported(facts: DashboardFacts, *, enabled: bool) -> bool:
# Whether the gate is still waiting for Copilot to say anything about the
# current head. Findings are an answer, not a silence: the threads they
# leave are the author's to clear, and the dashboard already routes the
# pull request to the author for them. Only a review that is missing or
# that covers older code is a report that has not arrived.
# current head. Findings are an answer, not a silence: actionable threads
# belong to discussion routing. Only a review that is missing or that
# covers older code is a report that has not arrived.
if not enabled or copilot_reviews_disabled(facts):
return False
return not facts.copilot_review_exists or facts.copilot_review_stale
Expand Down Expand Up @@ -269,9 +230,8 @@ def copilot_review_request_landed(
# Copilot can finish a short review before the last read, which takes it
# back out of the pending requests. A review of the current head proves the
# request landed just as well as a pending one does.
review_exists, review_stale, _findings = copilot_review_status(
review_exists, review_stale = copilot_review_status(
normalize_reviews(fetch_pr_reviews(owner, repo_name, pr_number)),
head_sha,
[],
)
return review_exists and not review_stale
Original file line number Diff line number Diff line change
Expand Up @@ -149,12 +149,9 @@ def deliver_copilot_review_requests(
)
requests.pop(key, None)
continue
review_exists, review_stale, _review_findings = (
copilot_review_status(
reviews,
snapshot.head_sha,
snapshot.review_threads,
)
review_exists, review_stale = copilot_review_status(
reviews,
snapshot.head_sha,
)
if review_exists and not review_stale:
print(
Expand Down
8 changes: 3 additions & 5 deletions .github/scripts/pull-request-dashboard/dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -163,11 +163,9 @@
reviewed; eligible PRs
without a review need an
initial review request.
copilot_review_needed bool The review is stale or Copilot
owns open review threads,
meaning unresolved threads
GitHub has not marked
outdated.
copilot_review_needed bool The review is stale. Findings
use normal discussion
classification and routing.
created_at str (iso)
last_activity_at str (iso) Latest substantive activity by a
PR participant, never earlier
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,10 +210,6 @@ def _group_review_threads(source: DiscussionInput) -> list[dict[str, Any]]:
or f"review-discussion-{len(discussions) + 1}"
),
"discussion_kind": "review-comment-thread",
"strict_author_action": bool(
root_comment
and root_comment.actor.is_copilot_reviewer
),
"path": discussion.path or None,
"line": discussion.line,
"resolved": False,
Expand Down Expand Up @@ -732,11 +728,7 @@ def _review_thread_pending_actions(
)
discussion_id = classification.identity.discussion_id
discussion = by_id.get(discussion_id)
action = (
DiscussionAction.AUTHOR
if (discussion or {}).get("strict_author_action")
else decision.action
)
action = decision.action
comments = (discussion or {}).get("comments") or []
if action is not DiscussionAction.NONE and comments:
entry = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
copilot_review_status,
is_copilot_quota_failure,
is_copilot_reviewer,
open_copilot_finding_urls,
)
from dashboard_contracts import (
CopilotReviewFallback,
Expand Down Expand Up @@ -228,14 +227,9 @@ def _compute_facts(
)
api_author = pr.author.login
head_sha = snapshot.head_sha
(
copilot_review_exists,
copilot_review_stale,
copilot_review_findings,
) = copilot_review_status(
copilot_review_exists, copilot_review_stale = copilot_review_status(
source.reviews,
head_sha,
snapshot.review_threads,
)
override = dashboard_override_facts(
DashboardOverrideInput(source.issue_comments),
Expand Down Expand Up @@ -284,7 +278,7 @@ def _compute_facts(
copilot_review_error_count=copilot_review_error_count(source.reviews, head_sha),
copilot_review_exists=copilot_review_exists,
copilot_review_stale=copilot_review_stale,
copilot_review_needed=copilot_review_stale or copilot_review_findings,
copilot_review_needed=copilot_review_stale,
is_maintenance_bot=_is_maintenance_bot_author(api_author),
author_can_act=_author_can_act(pr.author, author),
is_draft=pr.is_draft,
Expand Down Expand Up @@ -317,7 +311,6 @@ def _compute_facts(
def _author_action_discussion_urls(
discussions: list[dict[str, Any]],
pending_actions: dict[str, dict[str, Any]],
additional_urls: tuple[str, ...] = (),
) -> tuple[str, ...]:
by_id = {
discussion["discussion_id"]: discussion for discussion in discussions
Expand All @@ -331,9 +324,6 @@ def _author_action_discussion_urls(
url = (discussion or {}).get("discussion_url") or ""
if url and url not in urls:
urls.append(url)
for url in additional_urls:
if url and url not in urls:
urls.append(url)
return tuple(urls)


Expand Down Expand Up @@ -632,11 +622,6 @@ async def evaluate_pull_request(
_author_action_discussion_urls(
review_threads,
pending_actions,
(
open_copilot_finding_urls(pr_source.review_threads)
if facts.copilot_review_outstanding
else ()
),
)
),
author_action_top_level_feedback_urls=(
Expand Down
2 changes: 1 addition & 1 deletion .github/scripts/pull-request-dashboard/state.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
STATUS_COMMENT_ROLLOUT_STATE_VERSION = 3
# Rendered status-comment behavior. Increment when existing comments need to
# adopt a change; hourly runs durably roll it out to all open PRs.
STATUS_COMMENT_REVISION = 21
STATUS_COMMENT_REVISION = 22
FULL_PUBLISH_STATE_VERSION = 1
INITIAL_BACKFILL_COMPLETE_KEY = "initial_backfill_complete"
_state_dir: Path | None = None
Expand Down
Loading
Loading