Skip to content

Allow skipping report generation when ending - #374

Open
chien1120yc wants to merge 1 commit into
sysprog21:mainfrom
chien1120yc:issue-100-skip-report
Open

chien1120yc wants to merge 1 commit into
sysprog21:mainfrom
chien1120yc:issue-100-skip-report

Conversation

@chien1120yc

@chien1120yc chien1120yc commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Add a "Skip report and exit" option to the ending overlay, following the minimal scope agreed in #100. Candidates can confirm that they want to discard the evaluation report and return to the lobby without waiting for report generation to finish.

Changes

  • Display the Skip option while the evaluation report is being generated.
  • Require confirmation before discarding the report and exiting.
  • Send a skip_report control packet to interrupt pending report generation.
  • Add frontend guards to ignore late report packets and block subsequent history saves after Skip.
  • Hide the Skip option once report delivery or recovery begins.
  • Preserve recording consent and existing recordings.

Screenshots

Ending overlay: The Skip option is available while the evaluation report is being generated.

image(20261026-065640)

Confirmation dialog: Candidates must confirm that the report will be discarded without being saved.

image(20261026-065721)

Testing

  • Manually verified the Skip flow returns to the lobby without adding a report to history.

Close #100

cubic-dev-ai[bot]

This comment was marked as resolved.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI fails on the indent gate: interview.html has formatting drift. Run make indent, then amend and force-push.

Check CONTRIBUTING.md carefully.

Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from 465a554 to 9be6dd7 Compare October 19, 2026 18:16
cubic-dev-ai[bot]

This comment was marked as resolved.

@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from 9be6dd7 to ae7790e Compare October 19, 2026 19:15
@jserv
jserv requested review from ColtenOuO and alanhc October 19, 2026 19:24
Comment thread src/livekit.rs Outdated
Comment thread web/interview.js Outdated
Comment thread web/interview.js Outdated
Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from ae7790e to 3e984a7 Compare October 21, 2026 07:50
cubic-dev-ai[bot]

This comment was marked as resolved.

@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch 3 times, most recently from 128b5ac to 783838d Compare October 21, 2026 13:57
Comment thread src/livekit/report.rs Outdated
Comment thread src/livekit/report.rs Outdated
Comment thread src/livekit/report.rs Outdated
Comment thread src/livekit/report.rs Outdated
Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch 2 times, most recently from f525b28 to b929494 Compare October 21, 2026 16:46
@chien1120yc

Copy link
Copy Markdown
Author

Fixed. I ran make indent and updated the PR accordingly.

The latest GitHub Actions run now passes all checks, including the formatting and browser gates. Thank you for pointing this out.

@chien1120yc chien1120yc changed the title Allow skipping report generation after an interview Allow skipping report generation when ending Oct 22, 2026
@chien1120yc

Copy link
Copy Markdown
Author

I rechecked the PR against CONTRIBUTING.md and corrected the PR title and description.

make hooks succeeded, and ./scripts/test.sh exited with code 0. The gate reported skipped lanes for unavailable local dependencies. All GitHub Actions checks are passing.

The review fixes are included in b929494. The PR is ready for re-review. Thank you!

Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from b929494 to 3fb1835 Compare October 22, 2026 15:01
Comment thread web/interview.js Outdated
@chien1120yc chien1120yc reopened this Oct 25, 2026
Comment thread web/interview.js Outdated
Comment thread web/interview.js Outdated
Comment thread web/history.js Outdated
Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from be73a88 to c7724f5 Compare October 25, 2026 15:23
Comment thread src/accounts/reports.rs Outdated
Comment thread src/livekit/report.rs Outdated
Comment thread web/lib.js Outdated
@chien1120yc

Copy link
Copy Markdown
Author

Hi @jserv,

While addressing the review comments, I noticed that supporting Skip during report generation introduces substantial complexity around cancellation, acknowledgements, late reports, and account cleanup.

I understand that Issue #100 explicitly chose the ending-overlay Skip behavior. However, given the implementation and maintenance costs revealed during this PR, I'd like to ask whether an alternative is worth reconsidering:

After clicking "End Interview", users could choose either to generate the report through the existing workflow or to end the session without starting report generation.

This would change the original interaction, but it might avoid much of the cancellation and cleanup logic.

Would you prefer to retain the agreed scope, or would it be worth exploring this simpler flow before continuing with the remaining changes?

I'm happy to follow the original design if that remains preferable.

@jserv

jserv commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

I understand that Issue #100 explicitly chose the ending-overlay Skip behavior. However, given the implementation and maintenance costs revealed during this PR, I'd like to ask whether an alternative is worth reconsidering:

Once you take ownership of a task, you should follow through unless completing it is infeasible.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before asking someone to review, you MUST resolve any regressions flagged by GitHub Actions.

@chien1120yc

Copy link
Copy Markdown
Author

Understood. I'll continue working on Issue #100 and see it through to completion. Thank you for the guidance.

jserv
jserv previously requested changes Oct 25, 2026

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase latest main branch and resolve conflicts.

@jserv

jserv commented Oct 25, 2026

Copy link
Copy Markdown
Contributor

Keep the ending-overlay Skip flow accepted in #100. The acknowledgement and cleanup must ensure that a late report cannot be published or saved after Skip succeeds. Simplify where possible while preserving those guarantees, and fix the missed mutants in the mutants (0), (1) and (3) shards before re-review. Skip discards the evaluation report; it does not withdraw recording consent or delete an existing recording.

Comment thread src/livekit/report.rs Outdated
Comment thread web/interview.js Outdated
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from c7724f5 to 5f71e0d Compare October 26, 2026 07:34
Add a confirmed skip action to the ending overlay so candidates
can discard a report during generation and return to the lobby.

Handle cancellation in the agent, guard late report handling,
and cover the behavior with Rust and browser tests.

Close sysprog21#100
@chien1120yc
chien1120yc force-pushed the issue-100-skip-report branch from 68de7d3 to cbc41d8 Compare October 26, 2026 07:39
@chien1120yc

Copy link
Copy Markdown
Author

I've reworked this PR after realizing that I had misunderstood the of #100.

The previous implementation included more than the issue required, so I simplified the changes to focus on the agreed "Skip report and exit" behavior: allowing candidates to confirm that they want to discard the report and leave immediately without saving it.

I've also updated the PR description, added screenshots, and revised the tests to reflect the new implementation.

Thank you for the previous feedback.

@jserv
jserv dismissed stale reviews from themself October 26, 2026 11:22

No merge conflicts at the current head.

/// the original candidate rather than a bystander or an unattributed sender.
#[test]
fn only_the_original_candidate_can_skip_the_report() {
let payload = serde_json::to_vec(&serde_json::json!({ "type": "skip_report" })).unwrap();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test builds the skip_report packet by hand, unlike the retry test above it, which takes its packet from tests/fixtures/control.json. Nothing ties skipReportPayload() in web/lib.js to what skip_request accepts, so a renamed type on either side passes both suites. Add a { name: "skip report", payload: lib.skipReportPayload() } case to controlCases() in scripts/gen-wire-fixtures.mjs, regenerate the fixture, and read the payload from it here the way only_the_original_candidate_can_request_report_regeneration does.

Comment thread web/interview.js
"End this interview without waiting for a report? Any report in progress will be discarded and will not be saved.";

/// Longest the page waits for the skip to leave before it goes anyway. The
/// agent also stops when the candidate disconnects, so a packet that does not

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The skip packet races the disconnect. leaveRoom runs as soon as publish resolves, and the agent's LiveKit SDK identifies the packet's sender from its participant roster (handle_data in livekit 0.9.6). If the candidate's departure is processed first, the packet arrives with participant: None and skip_request rejects it. The fallback this comment describes does not exist either: nothing stops generate_report_bounded when the candidate disconnects, and wait_for_skip discards the departure event, so generation runs to the end. Have the agent acknowledge the skip and wait for that acknowledgment (bounded by this timeout) before leaving, and correct the comment.

Comment thread web/interview.js
globalThis.setTimeout(resolve, SKIP_REPORT_SEND_WAIT_MS),
),
]);
leaveRoom();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the report fails during this wait, offer moves the phase to report_recovery, so leaveRoom() takes its recovery branch. That calls reportRecovery.finish(), which calls finalize, which reaches receiveReport, and receiveReport returns early because reportSkipped is set. The page stays on the recovery overlay with the skip button hidden. Clicking "Save incomplete report and leave" then does nothing: finish() returns once its report is cleared, and the phase never leaves report_recovery. A candidate who confirmed Skip is stuck until they close the tab. Leave directly here instead: stop recovery without finalizing, then run the teardown and navigation from leaveRoom.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase the current branch onto the upstream default branch and resolve any conflicts. Run /codetrial-review in AI agents (use $codetrial-review in Codex) and resolve all identified issues. Rework the series into minimal, logically coherent commits, squash closely related commits, and ensure all commit messages follow the project’s rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow users to end an interview immediately without generating a report

3 participants