Skip to content

feat(sdk): accept a host-supplied severity rubric for Standard scans - #2253

Merged
Aismit merged 2 commits into
mainfrom
aismit/BE-pass-rubric-to-hosted-scans
Nov 2, 2026
Merged

Aismit merged 2 commits into
mainfrom
aismit/BE-pass-rubric-to-hosted-scans

Conversation

@Aismit

@Aismit Aismit commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

A host can now choose the severity rubric for an individual Standard scan and verify which rubric the scan used. Previously, sending a rubric with the hosted v2 run request failed input validation. Omitting it let the scan run, but gave the host no way to supply its grading criteria.

For example, a host can send revision 3 of a team's rubric with the Markdown and its SHA-256. The SDK checks the hash before dispatch, sends those grading criteria to the model, and includes the same snapshot in the terminal receipt. The host can compare that receipt with its original request.

Changes

  • Accept an optional severityRubric on the hosted Standard scan request, using the same snapshot shape as severity reclassification. Hash the exact UTF-8 Markdown, including any trailing newline.
  • Include the snapshot in execution.run and in the terminal receipt. Give the executor its own copy so later edits cannot change the receipt.
  • A valid request canceled during checkout verification now returns a canceled receipt before executor dispatch. Previously this surfaced as an invalid-parameters error. Malformed inputs still fail validation.
  • Document how a host saves the snapshot before the first attempt and supplies it on later attempts. Each attempt has its own execution and receipt.

Testing

  • Built the plugin and SDK, checked the TypeScript declarations, and ran the SDK type and formatting checks.
  • All 46 existing severity reclassification tests passed.
  • Exercised immediate cancellation through the CLI with both a valid and an invalid rubric. Verified two attempts in separate state directories received the same host-saved rubric and distinct request IDs.
  • Checked the packaged CLI installation and runtime.
  • Ran the packaged CLI against a host adapter on a synthetic repository with a fixed configuration response and deterministic inference. The host saw the supplied rubric at dispatch, accepted the completed receipt, and verified the saved report and findings against their hashes. In a second run, the host rejected a terminal receipt containing a different rubric with a valid hash.

Risk and rollout

Hosts can adopt the new field after upgrading their CLI. With the field omitted, hosted scans keep their previous prompt and wire format. A supplied rubric governs severity grading. The scan's evidence, validation, and reporting rules still apply.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@github-actions github-actions Bot added the enhancement New feature or request label Nov 2, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review ✅ Completed 2026-11-02T17:52:48.434159Z ef94e0c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Aismit
Aismit merged commit 205684e into main Nov 2, 2026
74 checks passed
@Aismit
Aismit deleted the aismit/BE-pass-rubric-to-hosted-scans branch November 2, 2026 20:24
@github-actions github-actions Bot mentioned this pull request Nov 2, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants