Repository navigation
🔑 feat: Recover Enrolled Worker Credentials On Startup - #341
Merged
Merged
Conversation
Contributor
Author
Contributor
Author
|
Independent review completed for 2682b1c: no findings. Local build, no-emit typecheck, formatting, and 179 focused worker tests passed without a storage fixture. Live Code API/Redis/TLS restart verification was not run. |
Collaborator
|
@codex review the latest head, final review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
danny-avila
approved these changes
Oct 21, 2026
ThabangEnv
added a commit
to ENVIRO365/code-interpreter
that referenced
this pull request
Oct 22, 2026
* ci: Fix Release Version Resolution for Untagged and Resumed Runs (LibreChat-AI#233) * ci: Fix Release Version Resolution for Untagged and Resumed Runs The release workflow resolved its version in one inline shell block under `set -euo pipefail`, where two paths could not succeed. Filtering tags through `grep` made a no-match fatal. On the ordinary untagged tip of `main`, `git tag --points-at HEAD | grep -E '^v[0-9]+...'` exits 1, and the step died before reaching its skip handling or `next-release-version.sh`, so a deployable commit could not obtain a release version (LibreChat-AI#228). Selecting stable tags now reads exit 1 as an empty answer while exit 2 and above still fail the release, which also lets the missing-previous-tag case report its own error. The rerun-resume path then rejected the tag it had itself chosen. With a stable tag already pointing at `HEAD` and no release published, the version comes from that tag, and the following existence check failed merely because the ref existed (LibreChat-AI#229). It now compares the tag's commit against the release commit, so only a tag on some other commit is a collision; `Create tag` already tolerates a tag that exists. The block moved into `.github/scripts/resolve-release-version.sh`, beside the `next-release-version.sh` it calls, so `tests/release-version-resolution.sh` can cover every path: automatic, resumed, skipped, dispatched, pushed-tag, and the runs that must be refused, each against a throwaway repository with a stubbed `gh`. * fix: harden release resolver execution --------- Co-authored-by: Lia <lia@librechat.ai> Co-authored-by: Danny Avila <danny@librechat.ai> * feat: Route GitHub App credentials per repository (LibreChat-AI#236) * feat: Route GitHub App credentials per repository * test: Make repository routing assertion deterministic * fix: Harden repository credential routing * fix: Bound shared GitHub credential refreshes * fix: Bind GitHub credentials to admitted workspaces * fix: Authenticate GitHub App bot identity lookup (LibreChat-AI#237) * feat: Advertise Workspace Command Timeout Ceiling (LibreChat-AI#238) * 🌳 feat: Provision Conversation-Scoped Code Worktrees (LibreChat-AI#239) * feat: provision conversation-scoped code worktrees * fix: isolate conversation checkout metadata * docs: clarify isolated conversation checkouts * fix: revalidate conversation checkout sources * fix: preserve synchronous legacy execution startup * fix: harden conversation worktree lifecycle * test: use canonical workspace isolation keys * fix: secure conversation worktree provisioning * fix: preserve isolated workspace lifecycle * fix: harden conversation worktree provisioning * fix: fence worktree setup and credential routing * fix: use kernel-backed provisioning locks * fix: load worktree locking only when provisioned * fix: retain conversation provisioning ownership through recovery * fix: reserve provisioning before launching checkout writers * fix: pin Git provisioning inputs and close instance admission gaps * fix: Close native scratch directory streams (LibreChat-AI#240) * feat(code): route trusted VM GitHub App tokens by checkout (LibreChat-AI#248) * feat(code): route trusted VM GitHub App tokens by checkout * fix(code): resolve checkout credentials within canonical root * fix(code): reject cross-root credential aliases * 🧹 fix: Await Job Cleanup and Measure Retained Resources (LibreChat-AI#249) * 🧹 fix: Await Job Cleanup and Measure Retained Resources * 📡 fix: Scrape Sandbox Runner Metrics in Helm --------- Co-authored-by: Lia <lia@librechat.ai> * feat: Allow Installation-Scoped GitHub Tokens for Trusted Workers (LibreChat-AI#263) * feat(code): allow installation-scoped GitHub tokens on trusted workers * fix(code): revalidate GitHub installation token grants --------- Co-authored-by: Lia <lia@librechat.ai> * ⏳ fix: Let BYOM Workspace Calls Queue Past Thirty Seconds (LibreChat-AI#264) * ⏳ fix: Let BYOM Workspace Calls Queue Past Thirty Seconds * fix: Recompute BYOM Reservation TTLs After Admission --------- Co-authored-by: Lia <lia@librechat.ai> * 🔐 feat: Recover Code API Machine Credentials After Outages (LibreChat-AI#260) * 🔐 feat: Recover Code API Machine Credentials After Outages * 🔐 fix: Protect Recovery Budgets and Persist Compose Enrollment * 🛡️ fix: Limit Untrusted Code API Recovery Traffic --------- Co-authored-by: Lia <lia@librechat.ai> * ⏳ fix: Honor Workspace Queue Budgets Per Request (LibreChat-AI#265) * fix: bound workspace queue admission per request * fix: classify pre-enqueue admission expiry and document queue budgets * fix: deduplicate files by destination before sandbox validation (LibreChat-AI#247) * fix: deduplicate files by destination before sandbox validation Some callers (e.g. LibreChat) send the same file more than once per exec request when a user re-uploads a file in the same conversation. The sandbox's validateExecuteFiles() rejects duplicate destinations, which causes every tool call in that conversation to fail with: Error from sandbox: [bad_request] files contains duplicate destination "filename.pdf" This surfaces to the end user as a confusing "sandbox is down" error when the sandbox itself is fine - it is just rejecting the duplicate file list. Fix: add deduplicateFilesByDestination() and call it in getJob() before validateExecuteFiles(). Keeps the first file for each destination, silently drops subsequent duplicates, and logs a warning. * fix: preserve validated destinations when deduplicating input files * fix: protect submitted code from uploaded file collisions --------- Co-authored-by: Lia <lia@librechat.ai> * 🛤️ feat: Schedule Linked Worktrees as Their Own Workspace Lanes (LibreChat-AI#270) * feat: Schedule Linked Worktrees as Their Own Workspace Lanes * fix: Refuse quarantined-parent lanes before enqueue, forward lane policy, reset lane fences - Check the parent checkout fence before the assignment is stored or queued. - Forward the linked worktree policy to the forked native sandbox. - Protect shared Git config, hooks and info even before they exist. - Reset a lane fence with --reset-workspace-worktree. * fix: Grant lanes an explicit Git write allowlist, hold checkouts on quarantined lanes, release stale lanes - A lane writes only shared objects, refs, ref logs, LFS storage and its own worktree metadata; the checkout HEAD, index, operation state, config and hooks stay read-only without per-path denies. - A checkout assignment waits on any quarantined lane beneath it. - Lane registrations are released when their worktree disappears and capped at 32, dropping command roots and credential routes. * fix: Refuse a checkout while a lane keeps a stuck fence; let lane probes read shared Git - Code API indexes the lane fences enqueued beneath each checkout. A checkout reaches enqueue only once no lane holds a slot, so an indexed fence that still exists is stuck and the checkout is refused, surviving worker restarts. - Replay probes of a lane may read its common Git directory (read-only). * fix: Guard linked-worktree Git grants and unresolved actions --------- Co-authored-by: Lia <lia@librechat.ai> * 🛡️ fix: Guard Linked-Worktree Lanes Against Accidental Git Pruning (LibreChat-AI#272) * feat: Schedule Linked Worktrees as Their Own Workspace Lanes * fix: Refuse quarantined-parent lanes before enqueue, forward lane policy, reset lane fences - Check the parent checkout fence before the assignment is stored or queued. - Forward the linked worktree policy to the forked native sandbox. - Protect shared Git config, hooks and info even before they exist. - Reset a lane fence with --reset-workspace-worktree. * fix: Grant lanes an explicit Git write allowlist, hold checkouts on quarantined lanes, release stale lanes - A lane writes only shared objects, refs, ref logs, LFS storage and its own worktree metadata; the checkout HEAD, index, operation state, config and hooks stay read-only without per-path denies. - A checkout assignment waits on any quarantined lane beneath it. - Lane registrations are released when their worktree disappears and capped at 32, dropping command roots and credential routes. * fix: Refuse a checkout while a lane keeps a stuck fence; let lane probes read shared Git - Code API indexes the lane fences enqueued beneath each checkout. A checkout reaches enqueue only once no lane holds a slot, so an indexed fence that still exists is stuck and the checkout is refused, surviving worker restarts. - Replay probes of a lane may read its common Git directory (read-only). * fix: Guard linked-worktree Git grants and unresolved actions * fix: Guard linked-worktree lanes against accidental Git pruning * fix: Refuse Git Aliases and Allow Pathspec Flags in Worktree Lanes * fix: Guard Worktree Git Against Indirect Pruning * fix: Refuse Git Repository Dispatchers in Worktree Lanes --------- Co-authored-by: Danny Avila <danny@librechat.ai> Co-authored-by: Lia <lia@librechat.ai> * 🔓 fix: Let Linked-Worktree Lanes Commit When the Checkout Lives in the Worker Home (LibreChat-AI#274) * fix: Grant lane Git reads entry by entry so write binds survive a read-denied home A read grant on the whole common Git directory masked the write binds beneath it whenever the checkout lived inside the worker home, which SRT re-binds under a tmpfs (writes first, then reads). Lanes could not create their own index.lock or ref locks. Grant each entry off the writable paths instead, and add a real-SRT regression test with a Linux CI job. * fix: Re-bind lane Git writes after the common read bind instead of granting entries Per-entry read grants left the common Git directory as a writable tmpfs placeholder inside the worker home (pack-refs could strand refs there) and pinned replaced files to stale inodes. Grant the whole directory read-only again and, on Linux, list each existing writable Git directory as a deeper read-deny so SRT re-binds its write mount after the ancestor read bind. * 🎯 feat: Diagnose Every Failing Workspace Edit and Negotiate Tolerant Matching (LibreChat-AI#271) * 🎯 feat: Diagnose Every Failing Workspace Edit and Negotiate Tolerant Matching A rejected edit batch now reports every failing edit by position: missing edits name the nearest candidate line and flag elided, line-numbered, whitespace-only or CRLF mismatches, and ambiguous edits give their match count and line numbers. Two negotiated edit features add tolerant matching (line-trimmed, indentation-flexible, whitespace-normalized) and replaceAll, with per-edit match reporting only for requests that opt in. * fix: Honor Edit Boundaries and Negotiated Capabilities * fix: Preserve Matched Line Endings in Tolerant Edits * fix: Reject Unrepresentable Tolerant Edit Boundaries * fix: Require Tolerant Edit Boundary Line Breaks * fix: Bound Edit Matching and Redact Edit-Only Diagnostics * fix: Stream Workspace Replace-All Results Within Memory Bounds * fix: Preserve Extra Blank Lines in Tolerant Edits * fix: Honor Full Edit Wrappers and Linear Candidate Budgets * fix: Retain Every Failed Edit Position in Bounded Diagnostics --------- Co-authored-by: Lia <lia@librechat.ai> * 🛡️ fix: Deny a Non-Lane Root's Own Git Metadata in the Native SRT Sandbox (LibreChat-AI#276) * 🛡️ fix: Deny a Non-Lane Root's Own Git Metadata in the Native SRT Sandbox A trusted-vm command in a native (non-lane) SRT workspace could write the registered root's own `.git/hooks/*` and `.git/config`, planting a hook or a `core.fsmonitor` / `filter.*` entry that then runs unsandboxed the next time the user — or trusted worker code — invokes Git in that checkout. The non-lane config relied on SRT's mandatory `.git/hooks` and `.git/config` denies, which on Linux are computed from the worker *process's* current directory (see `sandbox/linux-sandbox-utils.js`). The worker's cwd is its home (the systemd user unit sets no WorkingDirectory), never the registered root, so those denies landed elsewhere and the root's `.git` was writable. macOS was unaffected because it applies global `**/.git/hooks/**` and `**/.git/config` Seatbelt patterns. `NativeSrtWorkspaceCommandSandbox` now adds explicit `denyWrite` entries for the registered root's own Git metadata when `<root>/.git` is a directory (lanes are untouched — they already keep the whole common Git directory read-only): - `<root>/.git/hooks` and `<root>/.git/config` unconditionally; - `<root>/.git/config.worktree` and `<root>/.git/commondir` where they exist; - the same files for every submodule Git directory under `.git/modules/**` (including nested submodules) and every `.git/worktrees/*`. `commondir` and `config.worktree` are denied only where present: Git reads them strictly and SRT would otherwise mask an absent target with an empty `/dev/null` bind Git cannot parse, breaking ordinary commands. Denied files that exist are re-bound read-only, so `.git/config` stays readable (remotes keep working) while writes fail; a benign `git commit` in the workspace is unaffected. Tests: a live SRT test gated on `LIBRECHAT_CODE_LIVE_SRT_TESTS=1` runs the sandbox with `process.cwd()` outside the root and asserts hooks, config, an existing per-worktree config, a submodule config, and a linked-worktree commondir cannot be tampered while a normal commit still succeeds; wired into the Linux native-sandbox CI job. A fast unit test asserts the computed `denyWrite` set. `packages/code/README.md` documents the guarantee and the residual writable-`.git` vectors (whole-`.git` replacement, a fresh top-level commondir, a nested repo) that the personal-machine SRT trust model accepts. * 🔁 fix: Refresh Root Git Metadata Denies per Command and Fail Closed The root's Git metadata denies were a snapshot taken at sandbox initialization, but the worker is long-lived: a repository, submodule, or linked worktree created on the host afterwards stayed writable until restart. Ordinary root commands now recompute the set and pass it as a per-command `denyWrite` override (every other filesystem field is the session policy), the way SRT recomputes its own mandatory denies on every wrap. Native Windows keeps the initialization snapshot because srt-win rejects per-command allowRead/allowWrite. Inspecting the metadata now treats only ENOENT/ENOTDIR as absence; any other failure propagates, so an unreadable `.git` fails the command closed (COMMAND_UNAVAILABLE) instead of silently dropping a deny. Corrects the `config.worktree` comment: in a repository that already enables `worktreeConfig`, a missing per-worktree config stays creatable, because denying an absent file makes every Git command fail. The README lists it with the other writable-workspace residuals. * 🧷 test: Retain the Contention Lock Owner Through Garbage Collection (LibreChat-AI#283) Co-authored-by: Lia <lia@librechat.ai> * 🪟 feat: Preserve Command Output Heads and Tails (LibreChat-AI#275) * 🪟 feat: Preserve Command Output Heads and Tails * 🪟 fix: Retain Logical Command Output Suffixes --------- Co-authored-by: Lia <lia@librechat.ai> * ♻️ feat: Reuse Verified Environment Preparation (LibreChat-AI#286) * feat: Reuse Verified Environment Preparation * fix: Preserve Environment Setup Recovery Diagnostics * 🧰 feat: Share Explicit Environment Tool Resources (LibreChat-AI#287) * feat: Grant Explicit Shared Tool Cache Resources * fix: Reject Replaced Tool Stores Before Dispatch * test: Verify Protected Receipt Contents Across Native Platforms * 📦 feat: Restore Isolated Dependency Snapshots (LibreChat-AI#288) * feat: Restore Isolated Dependency Snapshots * fix: Reject Dependency Hardlink and Control Path Aliases * feat: Bound Managed Environment Storage (LibreChat-AI#289) * 🔬 feat: Begin the OpenShell Backend Evaluation Track (LibreChat-AI#282) Co-authored-by: Lia <lia@librechat.ai> * feat: Retire Stale Linked Worktrees Automatically (LibreChat-AI#299) * 🧹 feat: Retire Stale Linked Worktrees Automatically Agents create one linked worktree per task under <checkout>/.worktrees and nothing ever removed them. Each keeps its own dependencies and build output, so a trusted-VM checkout accumulated 231 worktrees and filled its disk to 100% three times, quarantining workspaces. A worker with linked worktree lanes now retires stale worktrees in the background: two minutes after start, every six hours, sooner while a backlog remains, and before managed setup would be deferred for low disk space. A worktree is retired only when it verifies as a linked worktree under .worktrees, no request uses its lane or checkout, it has been idle for seven days by both this worker's lane use and on-disk Git activity, it is clean, unlocked, outside any merge/rebase/cherry-pick/revert/bisect, unquarantined, and its work is published or merged: HEAD is contained in a remote-tracking ref, or, with no live upstream (detached, never pushed, or upstream deleted after merge), its commits are already in the remote's default branch by content (patch-equivalent per git cherry, or identical content at every path the branch changed). Squash-merged branches and abandoned review worktrees therefore qualify; only local Git is consulted. Removal is `git worktree remove` without --force followed by `git worktree prune`; branches are kept so `git worktree add` restores any worktree. Retirement is on by default. --no-worktree-retirement or LIBRECHAT_CODE_WORKTREE_RETIREMENT=false disables it, and --worktree-idle-days / LIBRECHAT_CODE_WORKTREE_IDLE_DAYS change the threshold. LIBRECHAT_CODE_LOG_LEVEL=debug logs every kept worktree with its reason. * fix: Fence Checkout Requests During Retirement and Never Cut Removal Short - Checkout requests wait for an in-progress retirement of any lane beneath them, since root commands can reach .worktrees/*. - Requests waiting on a retirement honor their own cancellation. - git worktree remove runs without a timeout; a killed removal could leave a half-deleted worktree that no later pass can verify. - Drop the repository-wide git worktree prune: remove already deletes the retired worktree's metadata, and prune would also expire other registered worktrees that are only temporarily unavailable. - Read every .worktrees entry each pass instead of the first 4096. * fix: Recheck Quarantine Under the Lane Reservation A checkout or lane request that finished between inspection and the lane reservation may have left an uncertain mutation quarantined. Re-read both quarantine markers inside the reservation, immediately before removal. * fix: Defer Retirement When the Checkout Was Used During Inspection A checkout request can write ignored files inside .worktrees/<name> that git status never reports, and finish before the lane reservation. Count checkout request starts and finishes, capture the count before inspecting each worktree, and keep the worktree for a later pass when it changed by the time the reservation is held. * test: Clone the Merge-Simulation Remote on main Regardless of init.defaultBranch On CI the bare remote's HEAD names master, so the hosting clone had an empty HEAD and `git merge --squash` refused to run. * fix: Report Missing Workspace Paths as NOT_FOUND and Create Missing Parents on Write (LibreChat-AI#301) * 🧭 fix: Report Missing Workspace Paths as NOT_FOUND and Create Missing Parents on Write A missing file, directory or search scope no longer reports INVALID_PATH. When every existing ancestor resolves to a directory inside the workspace, the worker returns NOT_FOUND ("Workspace path does not exist"); escapes, symlinks, files used as directories and other path-safety failures keep INVALID_PATH. write_file creates missing parent directories beneath the workspace or lane root, one verified level at a time, never through a symlink or an existing file, and removes the directories it created if the write then fails. Code API advertises the added code in registration and maps it to HTTP 422 like INVALID_PATH. A worker reports NOT_FOUND only to a Code API that advertised it, and resends a refused settlement with INVALID_PATH, so older Code API replicas keep accepting settlements. * fix: Anchor Parent Creation to a Held Root, Sync It, and Surface Cleanup Failure Parent directories are created only under a held root descriptor, where mkdirat cannot follow a swapped path out of the root. A pathname-only root creates nothing and reports the missing parent as NOT_FOUND. Each created directory is synced into its parent before the write is acknowledged, and a rejected write that cannot remove a directory it created now reports a possible mutation instead of an atomic failure. * fix: Recheck Ancestors Before NOT_FOUND and Sync Parent Cleanup A miss is reported as NOT_FOUND only when the walk reached it through real directories that still have the same identity afterwards; any symlink in the chain stays INVALID_PATH. Cleanup syncs each removal into its parent and treats any directory it could not durably remove as a possible mutation. * fix: Keep Created Parents as a Committed Side Effect Directories a write creates are no longer rolled back. Once one exists the write finishes despite cancellation, and any later failure is reported as a possible mutation. A raced EEXIST parent is synced too, and the NOT_FOUND ancestry recheck now includes the workspace root. * feat: Quote the Current Text Where a Missing Workspace Edit Belongs (LibreChat-AI#300) * 📍 feat: Quote the Current Text Where a Missing Workspace Edit Belongs A missing edit's diagnosis now ends with a bounded excerpt of the region old_text most likely meant: the alignment where most of its distinctive lines fall into place (found in one pass with bounded votes), else its unique first line or closest line. Each of at most 8 rows is `<line>|<mark><text>`, with a space for identical lines, `~` for whitespace-only differences and `!` for changed text; lines are shortened to 160 characters and the excerpt to 1,600. The excerpt is a final `; the current text at lines A-B (...) is "<json>"` hint, so hosts that parse the existing grammar drop it and keep every other hint. Excerpts are the first detail dropped to fit the 3,000-character message bound and never shorten another edit's reason. * 🧭 fix: Bound Every Alignment Offset and Anchor Excerpts Where old_text Starts - Count every inspected anchor offset against the region vote budget, and stop at the first offset that would start before the file (offsets ascend), so a repetitive old_text cannot escape the bound. - Start a first-line or closest-line excerpt at the line where old_text starts, not at its first non-blank line. - Escape U+2028/U+2029 in quoted text, which JSON.stringify leaves raw, so each batch edit stays on one line for hosts that read the message line by line. * 📏 fix: Bound Edit Excerpts by Their Encoded Error-Body Size Code API returns the diagnostic inside a JSON error body, and LibreChat reads that body through a 4,096-byte buffer, discarding a truncated one. A 3,000-character message with multibyte or escape-heavy excerpts could exceed it. Excerpts are now limited to 1,600 bytes and granted only while the whole message stays within 3,800 bytes, both measured as UTF-8 once JSON-encoded. * 📐 fix: Shorten Edit Diagnostics by Encoded Size as Well as Length Batch reasons that still exceed the bound after excerpts are declined are now shortened against both the 3,000-character limit and the 3,800-byte encoded budget, and a single edit's reason is shortened the same way instead of being cut by character count. Multibyte closest-line quotes can no longer push the Code API's error body past the 4,096 bytes hosts read. * feat: Persist Workspace Admission Across API Restarts (LibreChat-AI#302) * 🧾 feat: Persist Workspace Admission Across API Restarts * 🧾 fix: Fence Durable Admission Recovery Across Replicas * 🧾 fix: Fence Durable Capacity Acquisition Before Dispatch * 🧾 fix: Retain Durable Quarantine Outcomes Across Restarts * 🧾 fix: Preserve Durable Admission and Read Cancellation --------- Co-authored-by: Lia <lia@librechat.ai> * 📖 feat: Stream Bounded Workspace File Reads (LibreChat-AI#303) * 📖 feat: Stream Bounded Workspace File Reads * 🧹 fix: Settle Interrupted Reads Before Pending I/O Cleanup * ⏱️ fix: Track Workspace Read Interruption Through Cleanup * 🧪 test: Queue Descriptor I/O Consistently on Node 20 --------- Co-authored-by: Lia <lia@librechat.ai> * 🗂️ fix: Guard Lane Worktree Registration (LibreChat-AI#304) * 🗂️ fix: Guard Lane Worktree Registration * test: Cover Linked Lane Worktree Registration Denials --------- Co-authored-by: Lia <lia@librechat.ai> * 🛡️ fix: Harden Sandbox Startup and Internal Transports (LibreChat-AI#305) Co-authored-by: Lia <lia@librechat.ai> * 🐳 ci: Publish Code API Images to GHCR (LibreChat-AI#307) * 🐳 ci: Publish Code API Images to GHCR Build one linux/amd64 image per Compose build on pushes to main that touch an image input, tag it sha-<commit>, and move the main tags together once every image has built. Pull requests build without pushing. Each image keeps a registry build cache so the KVM sandbox runner does not recompile its runtime packages on every push. README documents the tags and a Compose override for pulling instead of building. * 🏷️ fix: Build Every Main Commit and Promote the Newest Complete One Give each main commit its own concurrency group so a queued run can no longer be replaced and lose its sha tags. The promote job now serializes, resolves the newest main commit that has every image, retries each retag, and finishes a partial move on the next run. Report each pushed digest in the job summary and document per-image digest pinning. Drop the disk cleanup step: a cold KVM runner build peaked at about 17 GB of builder state with over 95 GB free. * 🔒 fix: Write Commit Tags Once and Never Move main Backwards Skip building any image whose sha tag for the commit already exists, so a re-run that fails part-way cannot mix two builds under one commit. Promotion retries tag inspections and refuses to move an image's main to an older commit than it names, so a transient registry error can delay a promotion but not reverse one. Note that the macOS override keeps building its own sandbox target. * 🧯 fix: Fail Closed When the Registry Cannot Answer Read image revisions through one script that tells a missing tag (or one this token may not read, and so may not write) apart from a registry that does not answer. The write-once check and the promotion now fail on the latter instead of treating it as a missing tag, so neither can rebuild over an existing tag or move main backwards. Restore the disk cleanup on the KVM runner leg, since the hosted runner only guarantees 14 GB free. * fix: block sandbox descriptor passing into unix socket gc (LibreChat-AI#309) * feat: replace tool-call sockets with anonymous pipes (LibreChat-AI#310) * feat: replace job-visible tool-call sockets with anonymous pipes * test: Observe Broker Cancellation Through TCP Peer * fix: Align Bun 1.4.2 and Verify Packaged Pipe Runtime * chore: Drop Node 20 Support * fix: Bound Pipe Failure Paths and Package Launcher Runtime * 🧵 fix: Replace Runtime Socketpairs With Anonymous Pipes (LibreChat-AI#312) * fix: deny socketpairs with pipe-only Python runtime IPC * test: document and verify Bun socketpair spawn denial * fix: refresh mutable package-init images during runtime rollout * test: render runtime rollout chart without optional dependencies * fix: preserve offline package-init image policy * 🌿 feat: Report Lane Git Branch And Head On Command Results (LibreChat-AI#311) * feat: Report Lane Git Branch And Head On Command Results The worker reads the checked-out branch and HEAD of the lane a command ran in and attaches them to execute_command results as laneGit. Code API validates the value and forwards it only when CODEAPI_BRIDGE_LANE_GIT is enabled and the worker advertised lane_git. * fix: Address Review Findings on Lane Git Reporting Compare a retried settlement with the stored one before replica-local lane Git sanitization, and skip the Git probe until lane_git is negotiated at registration. * fix: Bound Lane Git Probe and Pass Flag Through Compose Cap the post-command Git probe to the budget the command left, reserving settlement time, and pass CODEAPI_BRIDGE_LANE_GIT to the api and service-worker compose services. * fix: Harden Lane Git Probe Bounds and Reported Values Bound root resolution with the probe limit, enable the probe from the worker's active registration, require HEAD to peel to a commit, and reject Unicode control and format characters in branch names. * fix: Trust Only Probed Lane Git and Use the Worker Deadline Assign the lane Git worker on the normal startup path, budget the probe from the worker's assignment deadline, and drop any laneGit supplied by the command executor. * fix: Reuse Settlement Reads and Preserve Branch Whitespace Fold lane Git sanitization into the settle path's existing reads instead of a separate pre-pass, and strip only Git's trailing newline from probe output. * fix: Report Null for Non-UTF-8 Branch Names Read the lane branch as bytes and decode it strictly, so a ref name that is not valid UTF-8 is reported as null instead of a lossy copy. * fix: Revalidate Source Git Metadata Before Probing Probe a source or conversation checkout only while its .git is its own plain directory, so a command that rewrites .git to point at another repository cannot make the host-side probe report that repository's branch and head. * fix: Reject Linked Refs Before Probing Source Git Walk refs/ and require every entry to be a plain file or directory before the host-side probe runs, so a ref symlinked to another checkout cannot be followed. * fix: Guard Linked Lane Refs and Report Full Branch Names Check a linked lane's metadata HEAD and the shared refs before probing, read the branch from the full symbolic ref instead of --short, and avoid Dirent.parentPath so the refs walk works on Node 20.11. * fix: Stop the Git Refs Walk When the Probe Aborts Pass the probe signal into the source and linked worktree metadata checks and check it between directory operations, so an abandoned walk does not keep running after the command result is returned. * fix: Report Lane Git Only From a Stable Branch Read Read the branch before and after the head and omit laneGit when it changed in between, so a concurrent checkout cannot pair one branch with another branch's commit. * fix: Own Object Storage and Pin the Head Before Probing Require a link-free refs tree and object store with no alternates or reftable before the host-side probe, ignore replacement refs, and read the head from the branch ref the first read named. * fix: Disable Transports and Lazy Fetch in the Lane Git Probe Run the probe with no allowed protocols and lazy fetch disabled, so a rewritten .git/config cannot make a missing object trigger a fetch or a transport command on the host. * fix: Relay Loopback Resolvers Into KVM Guests (LibreChat-AI#314) * 🧭 fix: Relay Loopback Resolvers Into KVM Guests The KVM launcher forwards the runner's /etc/resolv.conf into the libkrun guest. Under Docker Compose that names the embedded resolver 127.0.0.11, but the TSI guest kernel keeps loopback traffic on its own loopback device, so every service-name lookup in the guest failed. Generated-file uploads and session marker reads to egress_gateway were dropped with ConnectionRefused or FailedToOpenSocket, while connections by IP worked. When the runner has a loopback nameserver, relay it over UDP and TCP port 53 from the runner's own IPv4 address with socat and hand the guest that address instead. Further loopback nameservers are dropped, routable resolvers such as Kubernetes cluster DNS are forwarded unchanged, and startup fails if the relay cannot listen or the runner has no routable address. * 🧭 fix: Bound the Guest DNS Relay and Relay IPv6 Loopback Give both relays an idle timeout and cap concurrent TCP children. socat 1.8 fails every UDP-RECVFROM child given max-children, so the UDP relay relies on its short idle timeout. Treat a `::1` nameserver as loopback too, relaying it over IPv6 from the same runner address instead of forwarding an address the guest would only answer locally. * 🧭 fix: Recognize Every Loopback Resolver Spelling Classify nameservers as loopback by value rather than by the canonical `::1` string: zero-padded or compressed IPv6 loopback and IPv4-mapped 127.0.0.0/8 are relayed too, forwarding over IPv6 to the address exactly as written. Routable IPv6 resolvers stay untouched. * 🧭 fix: Parse Nameserver Addresses Before Loopback Detection Replace string matching with real parsing. IPv6 addresses are expanded to their eight groups (compressed zeros, dotted IPv4 tails), and IPv4 addresses are read in the inet_aton(3) forms glibc accepts in resolv.conf. Then ::1 and IPv4-mapped or plain 127.0.0.0/8 are relayed in any spelling. Mapped addresses are relayed over IPv4 to their canonical form. Spellings of the same resolver collapse into one relayed entry, and addresses glibc rejects are forwarded unchanged. * 🔒 Harden Runner Supervisors with Anonymous Pipes (LibreChat-AI#315) * Harden runner supervisors with anonymous pipes and inherited seccomp * Load hosted-app netfilter modules in native CI and retain diagnostics * Keep relay idle on buffered pipe EOF and assert owned descriptor cleanup * Preserve supervisor DNS resolution without descriptor-send syscalls * test: check parent-side nsjail startup failure (LibreChat-AI#318) * fix: resolve packaged pipe addon inside guests (LibreChat-AI#336) * 🧭 feat: Add Guided Worker Setup And Saved Profiles (LibreChat-AI#339) * 🧭 feat: Add Guided Worker Setup And Saved Profiles * 🔒 fix: Keep Pairing Codes Out Of Terminal History * 🔒 fix: Isolate Secret Prompt Editing State --------- Co-authored-by: Lia <lia@librechat.ai> * 🔑 feat: Recover Enrolled Worker Credentials On Startup (LibreChat-AI#341) Co-authored-by: Lia <lia@librechat.ai> * 🪪 feat: Add Key-Bound Browser Enrollment And Resumable Setup (LibreChat-AI#342) * 🪪 feat: Add Key-Bound Browser Enrollment And Resumable Setup * 🛡️ fix: Protect Setup State And Legacy Enrollment Fences * 🔐 fix: Remove Completed Setup Secrets And Bind Pending State To Identity * 🛡️ fix: Bind Setup Recovery State Across Startup Modes * 🗝️ fix: Protect The Entire Setup Journal Namespace --------- Co-authored-by: Lia <lia@librechat.ai> * 📑 feat: Add Standalone File Extraction Service (LibreChat-AI#306) * 📑 feat: Add Standalone File Extraction Service * fix: Restrict Worker Control and Preserve Nested Document Text * fix: Preserve Parser Policy Failures and Startup Restrictions * fix: Reject Unsupported DOCX Content and Validate OPC XML Parts * fix: Enforce Complete DOCX Content Structure * fix: Extract Merged DOCX Cells Once * fix: Reject Unmappable PDF Font Text * fix: Normalize Optional PDF Font Encodings --------- Co-authored-by: Lia <lia@librechat.ai> * feat(sandbox): close skill package gaps and expose /etc-backed config to jobs Add qpdf, ImageMagick, Carlito/Caladea fonts, pypdfium2 and react/react-dom/react-icons, which the docx/pdf/pptx/xlsx skills use. NsJail jobs mount /usr but not /etc, so Debian's /usr symlinks into /etc dangled: LibreOffice's registry main.xcd (soffice aborted with uno::RuntimeException), alternatives (convert/magick, libblas/liblapack) and OpenSSL certs. Resolve them at build time and bind /usr copies of /etc/fonts and /etc/ImageMagick-7 into the jail. * feat(sandbox): SANDBOX_ALLOW_UNIX_SOCKETS opt-in for LibreOffice Upstream denies AF_UNIX socket() and socketpair() in both the job seccomp policy and the inherited supervisor filter. LibreOffice's process launcher needs them (ERROR 4 forking process). The opt-in re-allows only those two; sendmsg/sendmmsg and io_uring stay denied, so SCM_RIGHTS descriptor passing (CVE-2026-80521) remains blocked. Default off keeps upstream behaviour. * ci(ecr): drop duplicated control-plane comment --------- Co-authored-by: lia-by-librechat[bot] <328778573+lia-by-librechat[bot]@users.noreply.github.com> Co-authored-by: Lia <lia@librechat.ai> Co-authored-by: Danny Avila <danny@librechat.ai> Co-authored-by: proseer-lars <lars@proseertech.com> Co-authored-by: Marco Beretta <81851188+berry-13@users.noreply.github.com> Co-authored-by: Dustin Healy <54083382+dustinhealy@users.noreply.github.com> Co-authored-by: ThabangLenonyana <lenonyanathabang@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Start slice 2 with automatic startup recovery. A worker stopped past its access-credential lifetime currently exits on refresh rejection even when Code API still holds its durable machine enrollment.
CREDENTIAL_INVALIDrefresh rejection invokes Code API's existing signed challenge/recovery flow.Independent of #339. This PR is the recovery portion of slice 2, not the
status/doctorcommands. Browser/OAuth approval from slice 1 is still outstanding.Verification
npm run buildandnpx --no-install tsc --noEmit: passed inpackages/code.Independent review
Reviewed head:
2682b1cc361ce94f65331ea9f73550bc816e85a2. Complete, no findings. Reviewer ran 47 recovery tests and 25 selected regressions with direct TypeScript execution. No live Code API/Redis/TLS restart test was performed.