Skip to content

Commit e7ea93c

Browse files
Add inline-script cache key hash utility (PEP 723 PR 1/16) (#1634)
> Part of #1602 (PEP 723 inline script env support). Design doc: #1601. ### Roadmap context This is **PR 1 of 16** in the PEP 723 inline-script roadmap. The full plan lives in #1602. | Phase | PR | Status | |---|---|---| | **Phase 1: Foundation** | **PR 1: cache key hash utility** | **this PR** | | | PR 2: cache layout + `meta.json` sidecar | _in progress_ | | | PR 3: `requires-python` to interpreter selection | _in progress_ | | **Phase 2: Manager** | PR 4: `InlineScriptEnvManager` skeleton | merged (#1610) | | | PR 5: `create()` happy path | not started (needs 1, 2, 3, 4) | | | PR 6: `create()` uv-install fallback | not started (needs 3, 5) | | | PR 7: persistence with `get`, `set`, and Memento | _in progress_ (needs 4) | | | PR 8: activation-time discovery | not started (needs 2, 4, 7) | | **Phase 3: Routing** | PR 9: route PEP 723 scripts to the inline manager | not started (needs 4, 7) | | | PR 10: per-script project registration | not started (needs 9) | | **Phase 3.5: Cross-repo** | PRs 17-19: Pylance and vscode-python integration | not started | | **Phase 4: UX** | PR 11: picker item; PR 12: bulk command | not started | | **Phase 5: Lifecycle and polish** | PR 13: clear cache; PR 14: TTL; PR 15: telemetry; PR 16: status bar | not started | PRs 1-3 are independent Phase 1 utilities that PR 5 will compose when it implements the environment creation path. ### Why this PR The cache key decides whether a PEP 723 script reuses an existing environment or builds a fresh one. Under the dependency-keyed design from Q4 of #1601, scripts using the same dependencies and interpreter should share an environment. Trivial metadata edits such as changing package-name casing, separator style, whitespace, dependency order, or extras order should not fragment the cache. This PR isolates that logic as pure functions with no filesystem access, global storage dependency, or extension activation behavior. The on-disk cache layout remains separate in PR 2. ### What this PR does Adds `src/common/inlineScriptCacheKey.ts` with four exports: 1. **`normalizeDependency(dep)`** Canonicalizes common variants of the same requirement: - Applies PEP 503 normalization to project names and extras: lowercase and collapse runs of `.`, `_`, and `-` to a single `-`. - Sorts and deduplicates extras after normalization. - Removes whitespace around version comparison operators. - Preserves meaningful version and marker content. - Drops empty dependency entries when keys are computed. Examples: - `Django` and `django` normalize identically. - `Flask-Login`, `flask_login`, and `flask.login` normalize identically. - `requests[Socks,security]` becomes `requests[security,socks]`. - `requests < 3` becomes `requests<3`. This is intentionally not a complete PEP 508 parser. Inputs it cannot meaningfully parse remain deterministic without attempting semantic normalization. 2. **`normalizeInterpreterPath(interpreterPath)`** Uses the existing `normalizePath` helper so Windows path casing and separator differences do not fragment the cache. POSIX path casing remains unchanged. 3. **`computeCacheKey({ dependencies, interpreterPath })`** - Normalizes, deduplicates, and sorts dependencies. - Normalizes the interpreter path. - Builds a versioned, labelled payload. - Hashes the payload with SHA-256. - Returns the first 16 lowercase hexadecimal characters. The resulting key is deterministic and safe to use as a directory name under the cache root introduced by PR 2. 4. **`CACHE_KEY_HEX_LENGTH`** Exposes the key length so callers and tests do not duplicate the value `16`. ### Caller contract `interpreterPath` must be absolute, trimmed, and already resolved through symlinks, for example with `fs.realpath()`. The utility deliberately performs no I/O. Two different path strings that resolve to the same executable will therefore produce different keys unless the caller canonicalizes them first. The creation flow in PR 5 will own that resolution. ### Cache behavior The script path and raw `requires-python` value are intentionally not included directly in the key. - Two scripts with equivalent dependencies and the same selected interpreter share an environment. - Moving or renaming a script does not invalidate its environment. - Adding, removing, or repinning a dependency produces a new key. - Selecting a different interpreter produces a new key. - A later PR re-verifies `requires-python` on cache hits in case the constraint changes without changing the selected interpreter. This supports the design's two outcomes: reuse an unchanged cache entry or build a fresh environment. There is no in-place dependency synchronization path. ### Tests Adds `src/test/common/inlineScriptCacheKey.unit.test.ts` with 37 unit tests covering: - PEP 503 project-name normalization. - Extras normalization, sorting, deduplication, and whitespace handling. - Version comparison operators and multi-clause specifiers. - Empty and duplicate dependency entries. - Dependency ordering and deterministic output. - Inputs that should and should not change the key. - Windows and POSIX interpreter-path behavior. - Fixed-length lowercase hexadecimal output. - Filesystem-safe output. - Pinned caller-contract behavior such as trailing whitespace in interpreter paths. Platform behavior is stubbed through `platformUtils`, keeping the suite independent of the host operating system. ### User impact **Zero.** Nothing in the extension calls these helpers yet. This PR adds no settings, commands, UI, logging, filesystem writes, or activation behavior. The utility is wired into environment creation in PR 5. ### Design context This implements the cache-key portion of Q4 in #1601: a pipx-style cache keyed by normalized dependencies and the selected interpreter. PEP 503 normalization for project names and extras was added in response to review feedback on the design PR. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent b16150a commit e7ea93c

2 files changed

Lines changed: 415 additions & 0 deletions

File tree

‎src/common/inlineScriptCacheKey.ts‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
// Copyright (c) Microsoft Corporation. All rights reserved.
2+
// Licensed under the MIT License.
3+
4+
import { createHash } from 'crypto';
5+
import { normalizePackageName } from '../managers/builtin/utils';
6+
import { normalizePath } from './utils/pathUtils';
7+
8+
/** Length, in hex chars, of the cache key returned by {@link computeCacheKey}. 16 = 64 bits of SHA-256; fixed-length and filesystem-safe. */
9+
export const CACHE_KEY_HEX_LENGTH = 16;
10+
11+
/**
12+
* Inputs that participate in the cache key.
13+
*
14+
* **Caller contract**: `interpreterPath` must be an absolute path that the
15+
* caller has already resolved through symlinks (e.g. via `fs.realpath()`).
16+
* This function does no I/O. Two string-distinct paths that point to the
17+
* same physical executable will produce two different cache keys.
18+
*/
19+
export interface CacheKeyInputs {
20+
readonly dependencies: readonly string[];
21+
readonly interpreterPath: string;
22+
}
23+
24+
function normalizeExtras(inner: string): string {
25+
const items = inner
26+
.split(',')
27+
.map((e) => e.trim())
28+
.filter((e) => e.length > 0)
29+
.map((e) => normalizePackageName(e));
30+
const deduped = Array.from(new Set(items)).sort();
31+
return deduped.length > 0 ? `[${deduped.join(',')}]` : '';
32+
}
33+
34+
/**
35+
* Canonicalize a PEP 723 dependency entry so common variants of the same
36+
* requirement produce identical strings. Not a full PEP 508 parser — only
37+
* folds the superficial edits users are likely to make:
38+
*
39+
* "Requests" → "requests"
40+
* "Flask-Login" → "flask-login"
41+
* "requests <3" → "requests<3"
42+
* "requests <3, !=2.0" → "requests<3,!=2.0"
43+
* "Requests[Security,Socks]" → "requests[security,socks]"
44+
* "requests[socks,security]" → "requests[security,socks]"
45+
* "requests[security,Security]" → "requests[security]"
46+
*
47+
* Extras (per PEP 503) are lowercased, separator-folded ([._-]+ → -),
48+
* deduplicated, and sorted alphabetically — same canonical form pip and
49+
* uv use for the project name, applied to each extra.
50+
*/
51+
export function normalizeDependency(dep: string): string {
52+
const trimmed = dep.trim();
53+
if (trimmed.length === 0) {
54+
return '';
55+
}
56+
57+
const nameMatch = trimmed.match(/^[A-Za-z0-9_.-]+/);
58+
if (!nameMatch) {
59+
// URL/VCS spec or other malformed entry — return trimmed so the
60+
// hash stays deterministic without pretending to parse it.
61+
return trimmed;
62+
}
63+
const name = normalizePackageName(nameMatch[0]);
64+
let rest = trimmed.slice(nameMatch[0].length);
65+
66+
let extras = '';
67+
const extrasMatch = rest.match(/^\s*\[([^\]]*)\]/);
68+
if (extrasMatch) {
69+
extras = normalizeExtras(extrasMatch[1]);
70+
rest = rest.slice(extrasMatch[0].length);
71+
}
72+
73+
const compactedRest = rest
74+
.replace(/\s+/g, ' ')
75+
.replace(/\s*([<>=!~]=?)\s*/g, '$1')
76+
.trim();
77+
78+
return `${name}${extras}${compactedRest}`;
79+
}
80+
81+
export function normalizeInterpreterPath(interpreterPath: string): string {
82+
return normalizePath(interpreterPath);
83+
}
84+
85+
/**
86+
* Compute the deterministic cache key for an inline-script env. The
87+
* payload uses a versioned, labelled-line shape — any future hashed input
88+
* MUST extend this shape rather than appending a new separator, or
89+
* existing hashes break silently.
90+
*/
91+
export function computeCacheKey(inputs: CacheKeyInputs): string {
92+
const normalizedDeps = Array.from(
93+
new Set(inputs.dependencies.map((d) => normalizeDependency(d)).filter((d) => d.length > 0)),
94+
).sort();
95+
const normalizedInterpreter = normalizeInterpreterPath(inputs.interpreterPath);
96+
97+
const payload = ['v1', `interpreter=${normalizedInterpreter}`, `deps=${normalizedDeps.join('\n ')}`].join('\n');
98+
99+
return createHash('sha256').update(payload, 'utf8').digest('hex').slice(0, CACHE_KEY_HEX_LENGTH);
100+
}

0 commit comments

Comments
 (0)