Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/architecture.md§4 (the "one call" data-flow block) listed the gateway's key-selection stepas healthy first → random → sticky reuse, which puts a fresh random pick ahead of the sticky
reuse. The router does the opposite: a sticky hit wins over a random draw.
This PR rewrites that one parenthetical to healthy first → sticky reuse → otherwise random, so
the documented order matches the order the code executes. No code changes, no behaviour changes,
no other line touched.
What the code actually does
src/router.rs::pick():onand not in cooldown);(user_id, model)and, when that key is still inside the healthyset, returns it immediately;
healthy.choose(&mut rng).The early
return Some(*kid)sits abovelet mut rng = rand::thread_rng(), so a sticky hitshort-circuits the random draw rather than following it.
Carriers for the corrected order
Three independent readings agree, so this is not asserted on a single reading:
cannot be reversed without deleting that branch.
pick()'s own doc comment (Chinese in the source; translated here): "select key: stickyfirst (while still healthy), otherwise random from the healthy set." The function documents the
precedence that the §4 sentence denied.
router::tests::pick_prefers_sticky_key— marks a sticky key and then asserts that thevery next pick returns it (
assert_eq!(p1, 2)).Honest note on the strength of that test: it demonstrates sticky reuse deterministically, but it
separates "sticky" from "random" only probabilistically. With two candidate keys, an implementation
ordered random-then-sticky would return the other key on the first pick about half the time, so one
run in two would pass by luck. The body and the doc comment are what make the precedence a fact
rather than a probability; the test is what turns a green suite red when someone reorders the
branches.
Why the sentence is load-bearing
A reader uses §4 to predict how calls are spread across the keys behind one model. As written, two
calls from the same user to the same model appear to land on independently random keys. In fact
they reuse one key until it becomes unhealthy — which is precisely what to check when one key
starts rate-limiting while its siblings sit idle.
This defect is in what ships
git show v0.7.33:docs/architecture.md(line 45) still carries the inverted order, so the releasedartifact ships the wrong order. The correction matches code that has been unchanged since P0-B.
Scope
auth resolves the Bearer API key to a user; the balance pre-check is
available = gift + permanent, rejected with 402 when≤ 0(src/gateway.rs:266-277); settlecredits the sharer 90% and writes
transactions+usage_records+keys.used.router.rsrow in §3's module table andui/README.md:337both name sticky withoutasserting an order, so neither is touched.
docs/architecture.md: the onlyinclude_str!of a docs file isdocs/prototype/aitokenpool-console.html.Related Issue
None. The repository has no open issue for this, and a one-line documentation correction does not
warrant opening one.
Changes
Tests
cargo test— 579 passed, 0 failedcargo fmt --check— cleancargo clippy --all-targets— no warningsChecklist
docs/…)docs(architecture): …)