Add server-selected strict interview mode by ZuoTso · Pull Request #285 · sysprog21/codetrial

ZuoTso · 2026-10-11T19:14:59Z

Summary

Introduce an opt-in Strict Interview mode for more honest self-assessment, reducing the ability to repeatedly practice a familiar problem through URL selection or page refresh.

Strict assigns problems on the server based on the selected difficulty, while Practice remains unchanged. A normal same-tab refresh starts a new attempt and excludes the previous assignment when its receipt is available. Session recovery is not included.

Testing

./scripts/test.sh passed (1,141 tests, no failures or skips; 49 hook checks). Manually verified with make web.

Closes #185

Summary by cubic

Adds an opt-in Strict Interview mode that assigns a matching problem server-side after media preflight, closing #185 by preventing repeated practice of a known problem via URL or refresh. Practice mode keeps its current behavior; Strict requires GitHub sign-in.

Selection and refresh behavior

Difficulty remains a multi-select choice; the server draws from matching problems, excluding the previous assignment.
A signed receipt in session storage remembers the previous problem per tab; clearing or replaying it resets the exclusion, and storage failure does not start a new sequence.
Reloading returns to preflight, then starts a fresh attempt with a different problem; admission is refused with HTTP 409 if none remains.
Strict ignores URL problem selection, does not fall back to offline practice, and restores no interview progress.
Strict always starts a fresh room, bypassing a pinned development room.

^{Written for commit 5c53026. Summary will update on new commits.}

ColtenOuO

Squash the commit and remove Co-authored-by: xxx in commit message.

Addressing review feedback doesn't require adding a new commit message.

ZuoTso · 2026-10-11T21:36:44Z

Squash the commit and remove Co-authored-by: xxx in commit message.

Addressing review feedback doesn't require adding a new commit message.

Squashed into "95a5fd3" and removed the "Co-authored-by" trailer.

jserv · 2026-10-11T23:12:07Z

+    let choices: Vec<_> = problems
+        .iter()
+        .filter(|problem| difficulties.contains(&problem.difficulty))
+        .filter(|problem| Some(problem.variant().page) != previous)


Excluding only previous means the pool never shrinks: each reload is a fresh uniform draw from every other problem at those levels. A candidate who already knows some of them can keep reloading until one of those comes up, which is the "practice a familiar problem" case #185 is about. Each reload also sends out a new agent while the old room keeps its capacity slot. If that is acceptable for a self-assessment mode, say so in the limits paragraph of docs/interview-selection.md. Otherwise, a follow-up could store the draws in a bounded, signed set (or keep them on the server per account) so that reloading moves through the pool instead of re-rolling.

This is an intentional limitation of the opt-in self-assessment mode. Excluding only the previous problem reduces immediate repetition but does not prevent deliberate re-rolling. Strict retains the existing agent cleanup policy, accepting temporary capacity overlap. Both trade-offs are now documented in docs/interview-selection.md.

jserv · 2026-10-11T23:13:39Z

Squashed into "95a5fd3" and removed the "Co-authored-by" trailer.

You don't have to make statements like this, as GitHub always tracks commits.

jserv · 2026-10-23T07:08:03Z

#440 adds a 10 s deadline on /api/token (TOKEN_REQUEST_TIMEOUT_MS) and rewrites the resulting TimeoutError to "The interview connection timed out. Offline practice is ready." Your Strict branch in the same catch shows error.message while refusing offline practice, so once both land, a Strict start that times out would say offline practice is ready. Whichever of the two lands second needs the timeout to reach the Strict branch with wording of its own; please plan for that when rebasing.

jserv

Rebase latest main branch and resolve conflicts.

ColtenOuO · 2026-10-23T17:05:39Z

    await connectLiveKit(connection, preflight, presenting);
    state.connected = true;
+    if (strict) updateRunAvailability();
+    if (strict && !whiteboard) nodes.editor.disabled = false;


This turns the editor back on without checking anything else. A pause or a behavioral round_state can arrive while connectLiveKit is still running, and this line then makes the editor editable when it should stay locked. Suggest nodes.editor.disabled = state.paused || codingClosed();, like the other places that turn the editor back on.

ColtenOuO · 2026-10-23T17:05:40Z

      "connection",
-      providerUiState("degraded", "The interview connection dropped.").message,
+      strict
+        ? "The Strict Interview connection dropped."


When the start fails after the room was joined, the catch block in connect calls joined.disconnect(). That fires this handler later, which replaces the specific failure banner (the timeout or Egress refusal) with this generic one. The catch has already cleared state.room, so setting the banner only when state.room === room would keep the real reason on screen.

ColtenOuO · 2026-10-23T17:05:40Z

+  // waits for a signed-in account and suggested difficulties to settle.
+  start.disabled =
+    (selectionMode === "practice" && !problem) ||
+    (selectionMode === "strict" &&


On a deployment that requires login, a signed-out candidate who picks Strict sees Start labeled "Use GitHub to start", but this condition keeps it disabled (accountHistory !== true). So the sign-in-on-Start path through signInFirst is never reached, and clicking does nothing. Either let Strict through when signInFirst is set, or change the label for Strict.

Practice needs reproducible problem links, while strict interviews should prevent candidates from choosing known problems. Preserve difficulty selection in both modes and start a new strict attempt on reload instead of recovering the previous interview.

ColtenOuO

LGTM, very nice PR :) Thanks for your contribution and patience.

jserv · 2026-10-23T23:38:26Z

Thank @ZuoTso for contributing!

This comment was marked as resolved.

Sign in to view

ColtenOuO requested changes Oct 11, 2026

View reviewed changes

ZuoTso force-pushed the fix-interview-refresh branch from 8b0e3df to 95a5fd3 Compare October 11, 2026 21:31

jserv reviewed Oct 11, 2026

View reviewed changes

jserv mentioned this pull request Oct 12, 2026

Preserve random draws across interview navigation #280

Closed

ZuoTso force-pushed the fix-interview-refresh branch from 95a5fd3 to 6e9c9b4 Compare October 13, 2026 04:20

jserv requested a review from ColtenOuO October 13, 2026 04:53

This comment was marked as outdated.

Sign in to view

ZuoTso force-pushed the fix-interview-refresh branch from 6e9c9b4 to afd7a92 Compare October 13, 2026 16:28

jserv reviewed Oct 13, 2026

View reviewed changes

Comment thread web/app.js

ZuoTso force-pushed the fix-interview-refresh branch 2 times, most recently from 2a49315 to e03a0e2 Compare October 14, 2026 09:18

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

ZuoTso force-pushed the fix-interview-refresh branch from e03a0e2 to 08a21b1 Compare October 19, 2026 05:48

This comment was marked as outdated.

Sign in to view

ZuoTso force-pushed the fix-interview-refresh branch from 08a21b1 to 29c232f Compare October 19, 2026 09:49

This comment has been minimized.

Sign in to view

ZuoTso force-pushed the fix-interview-refresh branch from 29c232f to abc6eef Compare October 20, 2026 04:16

jserv reviewed Oct 21, 2026

View reviewed changes

Comment thread web/interview.js Outdated

This comment has been minimized.

Sign in to view

jserv reviewed Oct 22, 2026

View reviewed changes

Comment thread src/web/token.rs Outdated

jserv reviewed Oct 22, 2026

View reviewed changes

Comment thread web/interview.js Outdated

This comment was marked as outdated.

Sign in to view

jserv reviewed Oct 23, 2026

View reviewed changes

Comment thread web/interview.js Outdated

jserv mentioned this pull request Oct 23, 2026

Bound interview token startup and no-show cleanup #440

Merged

ZuoTso force-pushed the fix-interview-refresh branch from abc6eef to cdcc80e Compare October 23, 2026 10:10

ColtenOuO reviewed Oct 23, 2026

View reviewed changes

Comment thread web/interview.js Outdated

Comment thread web/interview.js

jserv reviewed Oct 23, 2026

View reviewed changes

Comment thread web/interview.js Outdated

jserv requested changes Oct 23, 2026

View reviewed changes

ZuoTso force-pushed the fix-interview-refresh branch from cdcc80e to 3624b7c Compare October 23, 2026 16:11

ColtenOuO reviewed Oct 23, 2026

View reviewed changes

ZuoTso force-pushed the fix-interview-refresh branch from 3624b7c to 5c53026 Compare October 23, 2026 17:12

ColtenOuO approved these changes Oct 23, 2026

View reviewed changes

ColtenOuO requested a review from jserv October 23, 2026 18:13

jserv merged commit 85135ba into sysprog21:main Oct 23, 2026
13 checks passed

Conversation

ZuoTso commented Oct 11, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Summary by cubic

Uh oh!

This comment was marked as resolved.

Uh oh!

ColtenOuO left a comment

Choose a reason for hiding this comment

Uh oh!

ZuoTso commented Oct 11, 2026

Uh oh!

jserv Oct 11, 2026

Choose a reason for hiding this comment

Uh oh!

ZuoTso Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

jserv commented Oct 11, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

This comment has been minimized.

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

jserv commented Oct 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jserv left a comment

Choose a reason for hiding this comment

Uh oh!

ColtenOuO Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

ColtenOuO Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

ColtenOuO Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

ColtenOuO left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jserv commented Oct 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ZuoTso commented Oct 11, 2026 •

edited by cubic-dev-ai Bot

Loading