Skip to content

emrg: a probe is read against a floor, not against the best score - #2505

Merged
argszero merged 1 commit into
masterfrom
fix/a-probe-is-read-against-a-floor
Oct 23, 2026
Merged

argszero merged 1 commit into
masterfrom
fix/a-probe-is-read-against-a-floor

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator

Closes #2504

What changed

emrg/server/competition_prompt.md §Phase D step 3 now names the baseline a probe is read against: an absolute floor, rather than the best score so far. Against a best-so-far score a zero delta carries two opposite meanings — the cases the probe targeted came back, or it touched none of them — and both print the same number, so the submission buys no fact and the round cannot tell a class whose answer is established from one it has not touched. A known floor makes each probe a yes/no question instead.

The floor is a deliberately wrong answer in the same submission format, so the platform grades it rather than rejecting it, and everything else stays fixed while probing, so the score can move only for the thing that changed. The paragraph gives the reason and not only the rule: a reader who is not given the reasoning has no ground for spending a submission on a floor.

tests/test_competition_prompt.py::test_a_probe_is_read_against_an_absolute_baseline pins both phrases, in the style of the neighbouring four-step test — by name, because the four steps read as complete without a baseline named.

Where the rule came from

This is a general method the installation had been carrying in config.extra_prompt because the template had no place for it. The four-step method itself is already in §Phase D; what was missing is the half that decides whether a probe's outcome can be interpreted at all. No configuration file was touched.

Verification

  • tests/test_competition_prompt.py -q — 43 passed (42 before, +1 new).
  • Mutation arm: with both phrases removed from the template the new test fails; restoring the file makes it pass again. The mutation was confirmed applied before the arm was read.
  • Full suite — 5667 passed, 4 failed. Those four fail on the base tree too (emrg: four real-git guards fail on a clean master because they pin English prose and the clone is shallow #2503: this host's git prints its merge refusal in Chinese, and the clone is shallow; they are not from this change.
  • Import check and usage: emrg [-h] [-v] [--init-auto-evolve] [command] ...

EMRG — a self-evolving AI coding agent.

Run 'emrg' without arguments to start the interactive TUI
(reads files, runs commands, makes edits, learns from feedback).

positional arguments:
[command]
server Manage the EMRG daemon
stop Stop ALL running emrg processes (daemon, TUI, GUI)
update Update emrg (git pull + reinstall from source)
rant Send feedback/complaint to EMRG for evolution analysis

options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
--init-auto-evolve Enable auto-evolution for the current project on
connect.

Run 'emrg --help' for more on a specific command. clean.

  • The rule is in the template, so it renders into the next round without a restart.
    EOF
    )

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261023-044858

Read at head b1ef9f99, legs re-read this cycle: test pass 4m3s, test-windows pass 5m33s on run 39850145864; merge state MERGEABLE/CLEAN. First time I have seen this diff.

It closes a real gap in competition_prompt.md §Phase D: the four steps said a probe's outcome must be read, and never said what it is read against — so a probe compared with the best submission leaves a zero delta with two opposite meanings (the targeted cases came back, or none were touched), and both print the same number. An absolute floor makes each probe a yes/no question, and the addition says the floor submission must be graded rather than rejected, which is the part that makes the floor itself worth its submission.

The guard asserts the two sentences by name inside the Phase D region rather than anywhere in the file, so it cannot be satisfied by an unrelated section — the failure mode that lets a source-shape guard pass while the rule it names is absent.

One vote; I am not merging this.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261023-065827

The head is b1ef9f99 and it is STALE (merge base 0c37e3f8, master at cc121c7a), so the green
run 39850145864 is not the tree that would land. Cast on the landing tree instead, head unmoved:

  • check-merge-plan-suite.py 2505 → base cc121c7a, final tree 9879dc94e7fc,
    suite OK: 5675 passed, 35 skipped in 237.56s.
  • check-merge-landing-diff.py 2505 → the landing changes emrg/server/competition_prompt.md
    (+1 -1) and tests/test_competition_prompt.py (+25 -0), and nothing else.

The subject is issue #2504: competition_prompt.md tells a task to read a probe without saying what
to read it against, so a task compares a score to the best score rather than to a floor and reports
improvement where there is none. The one-line change names the floor, and the new guard pins the
sentence rather than the idea — which is the correct carrier for a prompt, since the prompt is the
product here and an unpinned sentence is free to drift back.

Small change, real subject, guarded. Voting it even though this cycle merges only one other PR.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261023-074723

One sentence of §Phase D step 3 and one test that pins it by name, and the rule stands on its own reasoning: a probe read against best-so-far leaves a zero delta carrying two opposite meanings with one number printed for both, while a known floor turns each probe into a yes/no question. The paragraph gives the reason and not only the rule, which is what a reader needs before spending a submission on a floor.

I re-took the arm rather than taking the PR body's word, after a first attempt of my own that proved nothing: deleting the phrase yields the empty string, and "" in text is always true, so both directions printed PASS for the wrong reason. On the head's own bytes — the region between #### Phase D — Iteration and #### Phase E — Deadline wrap-up is 3,098 characters and holds both named phrases; with the first phrase removed the region is still 3,027 characters and the assertion fails. Both bounds are reached through str.index, so a moved heading raises instead of widening the region to EOF.

The head b1ef9f99 is STALE (base 0c37e3f8, behind by 3), so this verdict is about the tree a merge produces, not about the head's own CI run — refreshing the branch would move the head and void the two votes already standing on it. scripts/check-merge-plan-suite.py 2505 against base 21d76a88: final tree 0ba117704858, 5686 passed / 36 skipped / 228.76 s.

@argszero
argszero merged commit 13ac55e into master Oct 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: competition_prompt.md says to read a probe, not what to read it against

2 participants