Skip to content

Commit e8f780a

Browse files
AslemammadAslemammad
committed
feat: add registry proxy and sumdb prototype
1 parent 1d1f450 commit e8f780a

29 files changed

+2507
-0
lines changed

‎REGISTRY_PROXY_PLAN.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# Registry Proxy + SumDB Status
2+
3+
## Summary
4+
5+
This file now reflects the current implemented state.
6+
7+
What exists today:
8+
9+
- `registry.npmx.dev` is an npm-compatible proxy surface
10+
- `sum.npmx.dev` is a minimal transparency log for immutable tarball assertions
11+
- sumdb logs only tarball records
12+
- every logged record includes:
13+
- `keyId`
14+
- `name`
15+
- `version`
16+
- `type`
17+
- `digest`
18+
- `size`
19+
- `url`
20+
- `integrity`
21+
- `signature`
22+
- `keyId` is the actual package-signing key from npm-style `dist.signatures`
23+
- the proxy verifies the upstream package signature before ingest
24+
- sumdb verifies the same logged signature again using trusted public keys loaded at startup
25+
- sumdb does not know registry base URLs, labels, or proxy-specific routes
26+
- the proxy no longer exposes custom routes like `/registries`
27+
- package-pattern routing has been removed; the proxy fetches from the first configured source registry
28+
29+
## Implemented Behavior
30+
31+
### 1. Registry catalog and fetch source
32+
33+
The checked-in registry catalog in [config/registries.ts](/Users/mohammadbagherabiyat/oss/npmx.dev/config/registries.ts) is now just an ordered list of source registries:
34+
35+
- `label`
36+
- `registryBaseUrl`
37+
38+
Current behavior:
39+
40+
- source-registry keys are fetched from each registry’s `/-/npm/v1/keys`
41+
- the full catalog is used to collect trusted public keys for startup/bootstrap
42+
- the proxy uses the first configured registry as its fetch source
43+
- there are no per-package or per-scope routing rules anymore
44+
45+
### 2. Proxy behavior
46+
47+
Implemented in [apps/registry-proxy/src/server.ts](/Users/mohammadbagherabiyat/oss/npmx.dev/apps/registry-proxy/src/server.ts):
48+
49+
- supports npm-compatible packument routes
50+
- supports npm-compatible tarball passthrough routes
51+
- exposes `GET /-/npm/v1/keys`
52+
- preserves source-registry tarball URLs in packuments so lockfiles do not point at the proxy
53+
- caches packuments on disk
54+
- verifies upstream `dist.signatures` before ingesting a tarball into sumdb
55+
- sends the minimal tarball record directly to sumdb
56+
57+
Not present anymore:
58+
59+
- `/registries`
60+
- proxy-side registry registration behavior
61+
- package-pattern routing logic
62+
- witness/envelope signing between proxy and sumdb
63+
64+
### 3. SumDB behavior
65+
66+
Implemented in [apps/sumdb/src/server.ts](/Users/mohammadbagherabiyat/oss/npmx.dev/apps/sumdb/src/server.ts) and [apps/sumdb/src/store.ts](/Users/mohammadbagherabiyat/oss/npmx.dev/apps/sumdb/src/store.ts):
67+
68+
- accepts only tarball ingest records
69+
- requires `integrity`
70+
- verifies the logged `signature` against the trusted responsible public key for `keyId`
71+
- rejects unknown key IDs
72+
- stores only the minimal logged record in the Merkle tree
73+
- serves lookup, checkpoint, inclusion proof, consistency proof, and tile endpoints
74+
- signs checkpoints with its own npm-style P-256 keypair
75+
76+
Startup model:
77+
78+
- trusted responsible public keys are fetched outside the sumdb core
79+
- the sumdb CLI bootstraps them from the checked-in registry catalog before creating the server
80+
- the server/store itself remains registry-URL-agnostic
81+
82+
### 4. Logged record and canonical leaf
83+
84+
Implemented in [apps/registry-core/src/protocol.ts](/Users/mohammadbagherabiyat/oss/npmx.dev/apps/registry-core/src/protocol.ts):
85+
86+
Logged record fields:
87+
88+
- `keyId`
89+
- `name`
90+
- `version`
91+
- `type`
92+
- `digest`
93+
- `size`
94+
- `url`
95+
- `integrity`
96+
- `signature`
97+
98+
The Merkle leaf is built from exactly those fields and nothing else.
99+
100+
Why packuments are not logged:
101+
102+
- they are mutable registry metadata, not immutable versioned artifacts
103+
- they do not have one stable package-version integrity value like tarballs do
104+
- dropping them keeps sumdb closer to a Go-sumdb-style artifact log
105+
106+
## Public Interfaces
107+
108+
### Ingest API
109+
110+
`POST /ingest` currently accepts:
111+
112+
- `keyId`
113+
- `name`
114+
- `version`
115+
- `type`
116+
- `digest`
117+
- `size`
118+
- `url`
119+
- `integrity`
120+
- `signature`
121+
122+
Validation:
123+
124+
- `type` must be `tarball`
125+
- `integrity` must be present
126+
- `signature` must verify for `${name}@${version}:${integrity}`
127+
128+
### Proxy API surface
129+
130+
Currently exposed:
131+
132+
- `GET /<package>`
133+
- `GET /@scope/<package>`
134+
- tarball passthrough routes
135+
- `GET /-/npm/v1/keys`
136+
137+
Currently not exposed:
138+
139+
- `/registries`
140+
- proxy-specific registry inspection endpoints
141+
142+
### Sumdb API surface
143+
144+
Currently exposed:
145+
146+
- `GET /lookup/:keyId/:packageName/:version`
147+
- `GET /latest-checkpoint`
148+
- `GET /checkpoint/:treeSize`
149+
- `GET /proof/inclusion/:leafIndex`
150+
- `GET /proof/consistency/:from/:to`
151+
- `GET /tile/...`
152+
- `POST /ingest`
153+
154+
## Verification And Tests
155+
156+
Verified today:
157+
158+
- minimal-leaf and Merkle tests pass
159+
- end-to-end proxy + sumdb install flow passes with real installs
160+
- lockfile tarball URLs point to the source registry, not the proxy
161+
- proxy no longer exposes `/registries`
162+
- logged records include the upstream package signature
163+
- sumdb verifies the logged signature using startup-loaded trusted public keys
164+
165+
Main test entrypoints:
166+
167+
- `node --test --experimental-strip-types apps/registry-core/src/*.test.ts`
168+
- `node --test --experimental-strip-types apps/demo/src/integration.test.ts`
169+
170+
## Assumptions
171+
172+
- trust is per published signing key, not per registry hostname
173+
- the registry catalog is the source of truth for source-registry order and external key fetching
174+
- the first configured registry is the active fetch source for the proxy
175+
- duplicate tarball ingests are deduplicated by canonical leaf

‎apps/demo/package.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"name": "@npmx/registry-demo",
3+
"version": "0.0.0",
4+
"private": true,
5+
"type": "module",
6+
"scripts": {
7+
"keys": "node --experimental-strip-types src/generate-keys.ts",
8+
"run": "node --experimental-strip-types src/run-demo.ts",
9+
"test": "node --test --experimental-strip-types src/integration.test.ts",
10+
"verify": "node --experimental-strip-types src/verify.ts"
11+
}
12+
}

‎apps/demo/src/generate-keys.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import { generateRegistryKeyPair } from '../../registry-core/src/index.ts'
2+
3+
const sumDb = generateRegistryKeyPair()
4+
const registry = generateRegistryKeyPair()
5+
6+
console.log('# SumDB')
7+
console.log(`SUMDB_PUBLIC_KEY<<'EOF'\n${sumDb.publicKeyPem}EOF`)
8+
console.log(`SUMDB_PRIVATE_KEY<<'EOF'\n${sumDb.privateKeyPem}EOF`)
9+
console.log(`SUMDB_KEY_ID=${sumDb.keyId}`)
10+
console.log('')
11+
console.log('# Registry proxy')
12+
console.log(`REGISTRY_PUBLIC_KEY<<'EOF'\n${registry.publicKeyPem}EOF`)
13+
console.log(`REGISTRY_PRIVATE_KEY<<'EOF'\n${registry.privateKeyPem}EOF`)
14+
console.log(`REGISTRY_KEY_ID=${registry.keyId}`)

0 commit comments

Comments
 (0)