Repository navigation
Prompt injection test: see what your agent's permission stops, from signed receipts - #7
Merged
Merged
Conversation
…m signed receipts. examples/injection-test hides the five AgentDojo attack templates (verbatim, MIT notice kept) in a vendor note a hosted agent is asked to summarize. Each attack runs under a permission scoped to the task and under a broad one, plus a clean control and an assume-breach case where the attacker's goal is the instruction itself. Every run is graded from the terminal receipt's signed tool activity: resisted, refused before it ran, held for approval, or dispatched; utility under attack is graded too. verify re-checks checksums, receipt signatures, run binding and every grade offline. --payload plants custom attack text for bounty submissions; --share makes records link-only. Grants are minted per run, expire in two hours, and are revoked at the end. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aDTfwQ3nirTB9pDCaFZmo
…n is a sample. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aDTfwQ3nirTB9pDCaFZmo
…asses the reader-language check. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aDTfwQ3nirTB9pDCaFZmo
… codes it quotes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013aDTfwQ3nirTB9pDCaFZmo
chrisjleal
marked this pull request as ready for review
October 20, 2026 17:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User-facing change
Before: nothing in this repository let a builder test their own agent against prompt injection, or show what a scoped permission changes.
After:
examples/injection-testhides the five AgentDojo attack templates (verbatim, MIT notice kept) in a vendor note the agent reads. It runs the same task under a scoped and a broad permission, then grades every run from its signed receipt. The outcomes are resisted, refused, held or dispatched, alongside utility under attack. Two more cases frame the result: a clean run and an assume-breach run.npm run verifychecks a sealed run folder offline. It needs no account and catches an edited record.How: the agent is a hosted job on the builder's own account. The kit mints two short-lived grants, creates private notes, polls the jobs, reads each receipt's portable proof, verifies it with
@proofable/sdk, and seals the folder with SHA256SUMS. The attacker host is.invalid, so it never resolves. The grants are revoked when the run ends. Nothing is published on-chain.--sharemakes records link-only and is opt-in.The README shows a real first run, from 20 October 2026: 3 attacks, k=1. The model ignored every injected attack under both permissions. With the model taken over, the scoped permission refused the outbound call before it ran, and the broad key let it go. The README also says what a permission does not stop: misuse of a tool the task needs.
Changelog
## [Unreleased]in CHANGELOG.md.How you checked it
Live: the run reported in the README, on the hosted runtime. The assume-breach receipts were verified offline with the kit's
verify. Two tampered copies failed it: an edited record, and a forged signed content.The kit's task and assume-breach prompt builders reproduce the exact prompts used in the live jobs.
npm run validatepasses, including the public-surface gate.No keys, tokens, bearer secrets, or private proof content in this description or the diff.
Generated by Claude Code