Repository navigation
Fail-closed fit, share, login, and engine key handling - #180
Merged
Merged
Conversation
peti12352
force-pushed
the
harden-fit-and-fail-closed
branch
from
November 18, 2026 20:24
1c229f7 to
7a913eb
Compare
Light engines stop paying SGLang's 1.8 GB overhead; retired pkg host migrates; pack SM unknown, verification origin, public share allowlist, and cmdline secrets are tightened. Share --allow is the client's IP (or goethe api allow); nginx always deny all.
peti12352
force-pushed
the
harden-fit-and-fail-closed
branch
from
November 18, 2026 20:24
7a913eb to
9f3dcb1
Compare
Drop the separate ternary Engine and per_gpu_budget_gb wrapper. Card runtime spelling stays ternary for learned rows; launch and overhead go through sglang. Equivalence tests cover argv, env gating, reserved flags, and digests.
Examples and the --allow row matched the old open-to-anyone behavior; they now match fail-closed share.
Top-level examples and the --allow clap line no longer teach opening without who may connect.
Reject everyone-CIDRs and the URL host as --allow; do not inherit goethe api allow for the public window; leaf cert is CA:FALSE; menu prints who may connect.
0.0.0.0/0 still needs that flag; rate limits stay on by default. Docs restore the URL/key/cert example comment.
…s warn. Reopening without --key no longer reprints the secret. Card LD_*/DYLD_* drops are named on stderr. models and models recommend both show per-GPU need with ctx and concurrency.
Keep both: their GGUF window KV, cuda-graph caps, ENGINE column, menu Load/Unload; our engine overhead_gb, fit ctx/concurrency labels, SGLang boot api-key, ternary fold.
Bare `expose public --yes` and raw `expose 0.0.0.0` refuse; cert SANs include interface IPs; models scan resolves folder names under GOETHE_HOME/models. isolated-home E2E traces and stage5 cover the regressions.
…l run packs. Also teach fake uv the wheel install path used after a plain-runtime fallback.
peti12352
force-pushed
the
harden-fit-and-fail-closed
branch
from
November 18, 2026 22:08
a44b87e to
57b6b02
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fit / overhead
Light engines (llama.cpp, goethe-engine, ninfer) were paying SGLang's 1.8 GB base overhead, so fit refused or looked too big.
Engineownsoverhead_gb(sglang / ternary 1.8, light 0.5);need()uses thatNeed = weights/tp + KV + window + state + overhead + load_peakKV_tokens = max(ctx, ceil(max_conc·ctx/4))0.95×(free−1.4)file_mb + file_mb/7 + 512)Model → engine
Unchanged install path; ternary launch is folded into SGLang:
card.runtime— kernel pack → goethe-engine; GGUF → llama.cpp; else registry / default sglang (or ternary / ninfer)engines::of_card→by_idby_id("ternary")launches as sglang;card.runtimestays"ternary"for learned rowsTERNARY_*env applies only when the card says ternary and the kernels are installedFail-closed
verification_uri_completemust share the platform host (wwwtwin ok)LD_*/DYLD_*and names them on stderr (no silent strip)GOETHE_ENGINE_API_KEY/LLAMA_API_KEYin environ; SGLang viagoethe_sglang_boot(not on/proc/cmdline)--api-keyuntil those binaries grow env/file supportsetup/updatestill drop retiredpkg.meshapplied.comfromgoethe.env(window still open)Public share
Naive user path:
--allowis who may connect (their phone/laptop public IP — ask them, or ifconfig.me on their device). This machine's IP is only the URL goethe prints.Opening with
--keystill prints the URL, the key, and the certificate to send. Reopening without--keykeeps the same keys/addresses but does not reprint the bearer (shows name + id;goethe apishows the secret).deny allgoethe api allow/ Tailscale CIDRs for the public window--allow0.0.0.0/0/::/0--dangerously-allow-all-inbound(not with--allow); CLI warnsCA:FALSEOnly: …)Expose / TLS / scan (isolated-home E2E)
--yesalone no longer binds every interface on a leftover allowlist; needs--allow <their-ip>/32 --yes(or interactivey). Rawexpose 0.0.0.0refused. Hidden--dangerously-allow-all-inboundfor world-open clients.curl --cacertto the bind address works; cert refreshes when SANs change.models scan <foldername>resolves underGOETHE_HOME/models, not CWD.expose public --yes→ bind every interface without client allowcurl --cacertfails)models scanfolder name from CWDinstall --name--yesnext to a display manager--yesGOETHE_HOME→ alt portgoethe runhas no-pSanitized summary:
docs/e2e-findings-pr180.md. Session logs with host paths and keys stay off-repo.Fit display (
models/models recommend)Both show the same shape so numbers are comparable:
goethe models:fits: X.X GB per GPU (ctx N, M concurrent)goethe models recommend:X.X GB/GPU · ctx N · M conc(per-GPU, not total×tp)Merged from engine-routing
Kept both: their GGUF window KV, cuda-graph caps, ENGINE column, menu Load/Unload; our engine
overhead_gb, fit ctx/concurrency labels, SGLang boot api-key, ternary fold.Cleanup
RESERVED_FLAGStable for SGLang extrasper_gpu_budget_gbremovedupdate::sha256_fileTests
No public bind, no GPU. Covers real failure modes:
TERNARY_*; missing kernels fail--dangerously-allow-all-inbound→0.0.0.0/0; flag +--allowerrors; reopen strips bearer; empty allowlist never admits the world; rate/conn embedded in nginx blockLD_*/DYLD_*listed incard_env_partsdrops; still absent from the process env--dangerously-allow-all-inboundstage5_configure_api.py: bare public--yes, raw0.0.0.0, and0.0.0.0/0refuse; named client allow succeedsmodels/; cert SAN includes interface IPsuvhandles the wheel install pathclippy -D warningscleanTest plan
cargo testconfigure / scan / nginx SAN / sglang / chooser / sharepython3 tests/stage5_configure_api.pyexpose public --yesrefuse + cert SAN includes interface IPs after gateway write