Skip to content

Implement fs-extra as a built-in package; hoist lazily defined error classes - #1504

Merged
ctate merged 14 commits into
opt/npm-polyfill-shimsfrom
opt/npm-fs-extra-shim
Oct 20, 2026
Merged

ctate merged 14 commits into
opt/npm-polyfill-shimsfrom
opt/npm-fs-extra-shim

Conversation

@cramforce

@cramforce cramforce commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1458 (opt/npm-polyfill-shims, which now includes #1487). Review the stack below it first.

Summary

fs-extra 8.x–11.x is built in. fs-extra builds its export object at load time. It copies Object.keys(graceful-fs) onto exports, wraps the asynchronous methods with universalify, and spreads its submodules into module.exports. The static CommonJS model can't follow that, so programs that read its members through the default import failed with "a binding form with no lowering". Reading mkdtemp, readFile, writeFile, stat, existsSync, promises or createReadStream from import fs from 'fs-extra' hit this.

Like undici in #1425, the package-shim registry now maps fs-extra to a TypeScript implementation (packages/compiler/shims/fs-extra) built on node:fs:

  • Wrapped methods. They follow universalify's rule: a callback as the last argument takes the callback form, any other call returns a promise. Both forms are implemented on fs.promises, using only the call shapes that compile natively. The methods are access, appendFile, writeFile, readFile, chmod, chown, lchown, copyFile, link, lstat, stat, mkdir, mkdtemp, readdir, readlink, realpath, rename, rm, rmdir, symlink, truncate, unlink and utimes.
  • fs's other members are re-exported, including promises, the streams, the *Sync functions and constants.
  • Extras: copy/copySync (with fs-extra's checks and messages), move/moveSync, remove/removeSync, emptyDir, ensureDir/mkdirs/mkdirp and their sync forms, ensureFile, pathExists, outputFile, readJson/readJsonSync, writeJson/writeJsonSync, outputJson/outputJsonSync, and the readJSON-style aliases. JSON follows jsonfile's formatting, BOM stripping, throws: false and file-prefixed error messages.
  • Types. The shim's declarations stand in for @types/fs-extra. The DefinitelyTyped twin's index.d.ts reads as a re-export of the shim, including its default export. A namespace import (import * as fs from 'fs-extra') resolves to the shim as well.
  • What isn't implemented is not declared, so using it is a type error at build time. This covers ensureLink, ensureSymlink and their aliases, gracefulify, the descriptor-based promise forms, the callback forms of the methods that answer stats, listings or paths, Buffer and URL paths, and option shapes fs.promises does not compile natively.
  • Not reproduced: graceful-fs's EMFILE retry queue. On Node 24 it only matters when the process runs out of file descriptors. The dependencies page lists this with the other differences.

Error classes defined in a lazy initializer. The hoisting from #1487 now also accepts a class whose base is a standard global constructor the bundle never rebinds, such as class extends Error; reading that base runs no code. This is the shape of a bundled package's error class, and a program class can now extend it, with new and instanceof working against both classes. The hoisting also changed in two other ways:

  • The initializer may be the last declarator of a shared var statement (var A, B, init_x = __esm(…)).
  • References in esbuild's 0 && (module.exports = { … }) annotation, which never runs, no longer block it.

Bundled copies of shimmed packages. A bundle that embeds fs-extra (build-utils does) compiled fs-extra's own JavaScript, which the static model can't follow. Now a __commonJS chunk whose key names a shimmed package's main file at a supported version (…/.pnpm/fs-extra@9.0.0/node_modules/fs-extra/lib/index.js) requires the shim instead:

  • The require goes through a forwarder declaration placed beside the bundle. A package's JavaScript can't load TypeScript sources directly, so the checker follows the forwarder to the shim, and module resolution answers the shim for its path.
  • The package's other chunks (graceful-fs, universalify, jsonfile) are no longer reached.
  • A call through such a require of a TypeScript module now lowers when the export never reads this, because require() answers the module's namespace.

The registry gained mainFile for each shim to support this.

A guard over every shim. packages/compiler/test/package-shim-guard.test.ts compiles each registered shim module (fs-extra, undici v5, undici v7) under both type worlds: scriptc's bundled Node declarations and @types/node. It fails on any lowering diagnostic or runtime fence located in the shim files, whether a program reaches it or not. Shim internals only suppress TypeScript type errors. A lowering refusal in shim code a program reaches is a build error located in the shim; one in unreached shim code would fail a later program, and the guard catches it now. I checked that the guard fails on a broken shim version.

A fix to the class hoisting from #1487. In minified output, an initializer that ends with its class lost the opener of its next run-once piece. Its empty insertion shared a position with the body's closing edit, so the initializer then called a piece that did not exist. The fix is on #1458 (Keep a hoisted class's next piece when the initializer ends with it), with a unit test, and is merged here. Here it also covers the shared-var form that the whitespace-minified lazy-barrel case uses.

Evidence

  • Corpus program fs-extra-package/ runs the real fs-extra 9.0.0 under Node; the package, its dependencies and @types/fs-extra are vendored and unmodified (MIT). It exercises:
    • the default, namespace and named imports;
    • promise and callback forms;
    • fs.promises and a read stream;
    • mkdirs, copy (including its error messages), move, emptyDir, remove, the JSON helpers and outputFile.
      The binary's output is byte-identical to Node's.
  • package-shims.test.ts has a new test: an installed CommonJS package's require('fs-extra') compiles the shim and matches Node running the real package.
  • npm-static case 2589-bundled-error-subclass uses real esbuild output. A program class extends a bundled error class defined in a lazy initializer, and new and instanceof are checked against both classes. Before this change it failed with "extending classes not declared in the program".
  • Unit tests cover the hoisting over global bases, past the export annotation, from a shared var statement, and in minified text that ends with the class.
  • npm-static case 2590-bundled-fs-extra uses real esbuild output that embeds fs-extra 9.0.0, with its user module lazily required as in build-utils. A unit test checks the forwarder, and that an unsupported fs-extra version keeps its chunk.
  • package-shim-guard.test.ts covers all three shim modules in both type worlds.
  • Zero-cost gate for the bundled-shim and guard commits, against Compile polyfill packages that are Node builtins as the builtins #1458's tip: PASS. All 2502 programs are byte-identical; with 3 timing runs, bench geomean is +0.6% and self +1.5%. An earlier single-run timing pass flagged bench at +5%, but every binary was identical and the per-program deltas swung ±30% both ways, so that was noise.
  • Zero-cost gate against Hoist lazily initialized ES module classes; follow on-demand workspace imports #1487's tip: PASS. All 2502 programs (bench, self, corpus) are byte-identical; bench geomean is −0.5% and self +0.6%.
  • The self-hosted compiler builds its IR cleanly. self-hosting-native-driver.test.ts passes on the final candidate (d118fcb), in 1022 s.
  • pnpm lint and the formatting of the touched files pass, as do test:ts7 --baselines-only, the frontend unit tests, package-shims.test.ts, the npm-static suite (127) and the npm suite. The npm suite's one failure, node24-url-authority, comes from the sandbox's Node 24.15 versus the pinned 24.22.

🤖 Generated with Claude Code

cramforce and others added 6 commits October 19, 2026 22:44
fs-extra builds its exports at load time (computed keys over graceful-fs,
universalify wrappers), which the static CommonJS model cannot follow, so
programs reading its members through the default import failed. The
package-shim registry now maps fs-extra 8.x-11.x to a TypeScript
implementation over node:fs: fs's members re-exported, the wrapped methods
following universalify's callback-or-promise rule over fs.promises, and
the extras (copy, move, remove, emptyDir, mkdirs, ensureFile, pathExists,
outputFile, the JSON helpers and their aliases) with fs-extra's messages.
Its declarations stand in for @types/fs-extra, including the default
import, and a namespace import resolves to the shim. What it does not
implement is not declared, so using it is a type error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bundled package's error class defined inside an __esm initializer
(`NowBuildError = class extends Error { … }`) now hoists to a declaration
like the classes over builtin module members, so a program class can
extend it and `new` and `instanceof` work against both. Reading a standard
global constructor the bundle never rebinds runs no code. References in
esbuild's `0 && (module.exports = { … })` annotation for Node's CommonJS
lexer never run and no longer block the move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Date-only utimes, literal mkdir recursion and promise-backed readFile
callbacks: the forms that compile natively with either type world.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build-utils declares its error module's bindings and initializer in one
statement (`var NowBuildError, mapTypoToSuggestion, init_errors = __esm(…)`).
The split pieces follow the statement, so the initializer only needs to be
its last declarator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 20, 2026 3:36pm UTC

@vercel-security-reviewer

Copy link
Copy Markdown

Security review details

Comment thread packages/compiler/shims/fs-extra/core/api.ts
Comment thread packages/compiler/shims/fs-extra/core/api.ts Outdated
Comment thread packages/compiler/shims/fs-extra/core/api.ts Outdated
Comment thread packages/compiler/shims/fs-extra/core/api.ts
cramforce and others added 3 commits October 20, 2026 01:20
In minified output the class closes right where the initializer body
does (`…Counter=class{…}}})`), so the empty insertion that opens the next
run-once piece shared its position with the body's closing edit and was
dropped: the initializer then called a piece that did not exist. The
opener now rides on the class's own closing brace.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A bundle that embeds fs-extra (build-utils does) compiled the package's
own JavaScript, whose computed export table and graceful-fs internals the
static model cannot follow. A __commonJS chunk whose key names a shimmed
package's main file at a supported version now requires the shim
instead, through a forwarder declaration beside the bundle: the checker
follows it to the shim (a package's JavaScript cannot load TypeScript
sources directly), and module resolution answers the shim for its path.
The package's other chunks are no longer reached. Calls through such a
require of a TypeScript module lower when the export never reads `this`
(require() answers the module's namespace).

package-shim-guard.test.ts compiles every registered shim module under
both type worlds and fails on any lowering diagnostic or fence inside the
shims, reached or not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	packages/compiler/src/frontend/npm-static-bundled-esm.test.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fs-extra shim

- writeFile and outputFile with byte data and a mode open the file with
  that mode, and writeJsonSync and outputJsonSync pass the mode.
- copySync and moveSync compare source and destination inodes and walk
  the destination's parents, as copy and move do.
- The resolver parity sweep skips the project-import comparison for
  shimmed packages.
- The JSON reader writes the byte order mark as an escape.
…ource

copy and copySync now follow fs-extra's onLink: only a destination that
is itself a symlink is replaced; over a regular file or directory the
symlink call fails with EEXIST and the destination is kept, whatever
overwrite says. The fs-extra corpus program covers both forms.
@ctate
ctate merged commit d1c516a into opt/npm-polyfill-shims Oct 20, 2026
64 checks passed
@ctate
ctate deleted the opt/npm-fs-extra-shim branch October 20, 2026 15:41

This branch was successfully deployed

1 active deployment
Preview — 313a9840 Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants