Skip to content

Edge shipper: ship Claude Code / Codex usage from Mac + Linux to the hub - #4

Merged
ninedter merged 1 commit into
mainfrom
claude/edge-ship-agent
Sep 10, 2026
Merged

ninedter merged 1 commit into
mainfrom
claude/edge-ship-agent

Conversation

@ninedter

Copy link
Copy Markdown
Owner

Builds on the ingest contract from #2 (POST /api/ingest/v1, merged into main): this is the client half. Machines that aren't the hub previously had no way in — hooks could only reach a tracker on the same box, and the hub could only see remote activity by mounting that machine's ~/.claude / ~/.codex. This adds a small always-on agent that runs where the work happens.

Claude Code hook ──POST 127.0.0.1:3799──▶ edge ──spool (JSONL, survives reboot)
                                            │
                                            └──POST /api/ingest/v1 (Bearer)──▶ hub
                                               batches ≤ 500 · retry w/ backoff

Hooks stay in Claude Code's critical path, so they can't wait on a tailnet that might be down. The edge answers on loopback in ~1 ms after the event is on disk, and takes over delivery from there.

What's here

  • edge/ — zero-dependency Node package (needs Node ≥ 20, no npm install): config, loopback intake, JSONL spool, batching flusher, redacting logger, and an llm-edge start|status|drain|send|config CLI. Everything path-portable (os.homedir() / path.join), config and spool under ~/.config/llm-usage-tracker-edge/ on both platforms. Secrets come from env or a config file — no Keychain-only path.
  • hooks/edge-hook.sh — bridges the existing hook to the edge by setting MONITOR_URL; the intake accepts POST /api/monitor/events verbatim, so no hook logic is duplicated. hooks_edge block added to hooks/claude-hooks-config.json. The local hub+watchers path is untouched for anyone running both on one box.
  • Docs — edge/README.md has copy-paste install for macOS and Linux, launchd and systemd user units, the full config table, CLI reference and troubleshooting. The root README gains a "Shipping from another machine (the edge)" section.

Decisions worth reviewing

  • Idempotency: every event carries a source_id — the caller's if it sends one, otherwise a SHA-256 over the event's content and its millisecond timestamp. Retries and replayed batches are counted as duplicates by the hub, never inserted twice.
  • Failure handling: 401/403 stops the flush and keeps the backlog (the token is wrong; hammering the hub won't help). 503 (hub started without INGEST_TOKEN) and 5xx/network errors retry with exponential backoff + ±20% jitter. Only 400 drops its batch — an unshippable batch would otherwise wedge every later event behind it.
  • Secrets: INGEST_TOKEN only ever appears in an Authorization header. Every log line goes through a redactor that strips the literal token plus anything bearer- or token-shaped; status / config / GET /status report ***set*** or ***missing***. A test asserts the token cannot reach a log sink even on the auth-failure path.
  • Spool is JSONL, not SQLite: no native module to rebuild per platform, and a machine that's been offline for a week can be debugged with grep. Capped at 100k events, oldest-first eviction, compaction batched so a long offline stretch doesn't go quadratic.
  • Intake is loopback-only and unauthenticated by design: anything that can reach 127.0.0.1:3799 can already run the hook.

Scope

macOS + Linux for v1. Windows is a one-line roadmap note in edge/README.md — the code is path-portable, but no service installer is provided. No mTLS (Tailscale is the transport security, matching #2). No hub schema/API changes.

Tests

  • npm run test:edge — 85 tests in edge/ (node --test, nothing to install): queue idempotency and durability across restart, oldest-first eviction, batch chunking at the hub's 500 cap, auth header present, exact hub body shape, offline retry, backoff bounds, config precedence, CLI, and log redaction.
  • npm test at root — 160 tests (was 155), including a new src/app/api/__tests__/edge-contract.test.ts that posts an edge-built batch through the real ingest route: accepted as-is, label lands on the session row, a replay comes back as duplicates, a full 500-event batch is accepted, and a batch without the bearer token is still 401.
  • npm run lint, npx tsc --noEmit and npm run build clean.
  • Manual end-to-end: MONITOR_URL=http://127.0.0.1:3799 bash hooks/test-hook.sh with a stub hub — all seven hook types exited 0 in ~55 ms each, arrived as one correctly shaped batch with the right bearer header, queue drained to zero, no token anywhere in the logs.

grok-bot-box deploy checklist (docs only — nothing deployed here)

  1. Hub (henrys-mac-mini) is up with INGEST_TOKEN set and reachable on the tailnet: curl -sf http://henrys-mac-mini:3789/api/health.
  2. On grok-bot-box: clone the repo, write ~/.config/llm-usage-tracker-edge/.env with TRACKER_URL=http://henrys-mac-mini:3789, MACHINE_ID=grok-bot-box, MACHINE_LABEL=Grok Bot box, then append INGEST_TOKEN via read -rs so it never hits shell history; chmod 600.
  3. node edge/bin/llm-edge.js send --dry-run — check the batch shape, then llm-edge start.
  4. Install the systemd user unit from edge/README.md, systemctl --user enable --now llm-edge, loginctl enable-linger $USER.
  5. Add the hooks_edge block to ~/.claude/settings.json, then MONITOR_URL=http://127.0.0.1:3799 bash hooks/test-hook.sh.
  6. llm-edge status → queued: 0, shipped_total climbing; the machine appears in GET /api/machines on the hub.
  7. Repeat on henrymacbook-pro with MACHINE_ID=henrymacbook-pro (launchd plist instead of systemd).

🤖 Generated with Claude Code

Machines that aren't the hub had no way in: hooks could only reach a tracker
listening on this box, and the hub could only see a remote machine's activity
by mounting its ~/.claude / ~/.codex. This adds the missing half of the
multi-machine story — a small always-on agent that runs where the work happens.

Claude Code hooks POST to a loopback intake, which appends to a JSONL spool
before answering, so the hook keeps its "always exit 0, fast" contract and
nothing is lost while the tailnet is down. A flusher forwards batches of at
most 500 events (the hub's MAX_BATCH_SIZE) to /api/ingest/v1 with the shared
bearer token, backing off exponentially when the hub is unreachable.

- edge/: zero-dependency Node package (config, intake, queue, shipper, agent,
  redacting logger) plus an `llm-edge start|status|drain|send|config` CLI.
  Paths go through os.homedir()/path.join and config lives under
  ~/.config/llm-usage-tracker-edge on both macOS and Linux — v1 platforms are
  macOS and this Linux box; Windows is a roadmap note only.
- Every event carries a source_id: the caller's, or a SHA-256 of the event's
  content and millisecond timestamp. Retries and replayed batches are dropped
  by the hub's idempotency index rather than duplicated.
- 401/403 stops the flush and keeps the backlog; 503 and 5xx/network errors
  retry; only a 400 drops its batch, since it would otherwise wedge every
  later event behind it.
- The token exists only in an Authorization header. Every log line passes
  through a redactor, and status/config report ***set*** or ***missing***.
- hooks/edge-hook.sh bridges the existing hook to the edge by setting
  MONITOR_URL, and the intake accepts POST /api/monitor/events verbatim — so
  the local hub+watchers path is untouched for people running both on one box.
- Tests: 85 in edge/ (node --test, nothing to install) covering queue
  idempotency and durability, batch chunking, the auth header and exact body
  shape, offline retry, backoff bounds and log redaction; plus a vitest
  contract test that posts an edge-built batch through the real ingest route.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ninedter
ninedter force-pushed the claude/edge-ship-agent branch from e9561ad to b550ab7 Compare September 10, 2026 14:22
@ninedter
ninedter merged commit 8d7b4f9 into main Sep 10, 2026
1 check passed
@ninedter
ninedter deleted the claude/edge-ship-agent branch September 10, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants