feat: generate package docs using `@deno/doc` by devdumpling · Pull Request #135 · npmx-dev/npmx.dev

devdumpling · 2026-01-26T16:48:14Z

Overview

This PR is one of two variants adding autogenerated documentation pages, accessible via a nice little link and routing to /docs/{package}/v/{version}.

If a package provides a homepage in its manifest, the link will default to that. Otherwise, we attempt to generate docs via deno docs using a WASM binary around @deno/docs. More on this below.

Note: in the other variant, we use a microservice instead of WASM. This has tradeoffs. See this PR.

Problem

Okay, so npmx.dev currently shows all kinds of fun package data (metadata, READMEs, file contents, etc). But it lacks a feature I love from JSR/crates/other modern package browsers... autogenerated docs off type definitions.

Generating these kind of docs is challenging for npm packages because:

npm pkgs don't ship in a standard doc format
TS types may be inline, in .d.ts, @types/*, jsdocs, or any combination
extracting and parsing that requires complex ASTs and compute overhead

Solution

Luckily, there's tons of prior art here. In particular, Deno and the JSR folks basically already solved this problem with deno doc.

Note: we should make sure they're cool with us using this if we end up going this direction... since we're sort of competing in a sense.

Architecture

Uses @deno/doc (official Deno doc generator package) compiled to WASM, running directly in the Node.js/Vercel serverless env. No subprocess shelling or microservice required.

It's a bit more complex than the microservice under the hood though, and has some thorns. It works like this:

Fetches the types URL from esh.sh with a x-typescript-types header
Loads the TS definitions using custom fetch-based loader
Parses them with @deno/doc WASM to extract DocNode[]
Render as HTML with syntax highlighting

Parts of this are the same as the microservice one, just calling it out here.

Arch Diagram

Implementation

This was quite a ride for me, as much of it is transforming the format from deno docs --json, but there's some key points.

Most of the heavy lifting is done in /server/utils/docs/ and is broken down into a few submodules:

index | entry point, calls generateDocsWithDeno(pkg, version)
client | WASM integration
processing | flattens namespaces, merge overloads, build symbol lookup, etc
render | generate HTML with Shiki syntax highlighting server side to not bloat the JS payload
text | escape HTML, parse JSDocs links, render markdown
format | format typescript signatures
types | internal types

WASM integration

The WASM integration required a Node.js compatability patch since @deno/doc is designed for Deno. This is a bit concerning since we'd have to maintain this or upstream it to the deno repo (possible, but not sure what they'd think). The issues are:

It has .d.ts exports in JS, which node won't natively resolve (see export * from './types.d.ts' lines in it)
WASM loader doesn't support Node.js fs, it tries to use the deno native one, so we needed to monkeypatch it to use node:fs and node:url for file:// URLs

I managed the patch with pnpm and is auto-applied on install.

Frontend implementation

The frontend implementation simply fetches from the api endpoint (/api/registry/docs/{pkg}/v/{version}) and renders the pre-generated HTML in v-html. I'm still learning vue/nuxt, so let me know if I'm doing anything dumb here.

The UI could probably be improved / iterated on. I left that out of scope for this POC / initial implementation.

A note on esm.sh

I needed to use esm.sh here instead of our existing calls to jsDeliver. Why?

In my testing, I couldn't get jsDeliver to resolve @types/* packages for untyped packages properly and kept hitting module resolution issues with varying .js|.ts extension 😢. I'm betting there's a way to make it work, but esm.sh worked on the first try, so I went with that. It also serves the typescript types natively with a x-typescript-types header which I use here.

If we're concerned about having two underlying providers here (jsDeliver and esm.sh) I can take this back to the drawing board / iterate.

Note on bundle size

This does add some WASM weight to our bundle. It should load once and then cache and only impact server-side, but worth calling out as downside.

Alternative approaches

I personally prefer this to the microservice as it works without more infra, but the caveats in the WASM bindings are worth exploring fixes / alternatives to.

As in the microservice approach, here are the alternatives:

A microservice variant -- run deno doc --json out of a microservice
Build our own parsing / AST solution natively, likely in Rust (I tried doing this with TypeDoc and it was clumsy and slow). We don't currently have any Rust stuff in the codebase so big decision there. Might be better solve long term.

Option 2 may be interesting, but also comes with the steep cost of diverging from the incredible work the JSR team already put into this. If we want to build out other rust stuff here though, might make sense. We could also theoretically do more interesting stuff than is possible with just the json data we get from deno doc --json.

closes #134

vercel · 2026-01-26T16:48:18Z

@devdumpling is attempting to deploy a commit to the danielroe Team on Vercel.

A member of the Team first needs to authorize it.

vercel · 2026-01-26T17:24:35Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
npmx-dev-docs	Ready	Preview, Comment	Jan 27, 2026 10:19pm
npmx.dev	Ready	Preview, Comment	Jan 27, 2026 10:19pm

danielroe · 2026-01-26T17:25:52Z

deployed, if you want to test ✅

devdumpling · 2026-01-26T17:26:17Z

deployed, if you want to test ✅

Thanks! Just finished writing up the description in the microservice variant. #134

devdumpling · 2026-01-26T18:26:45Z

i hate this but... it does work

although i realize now this isn't going to work in serverless envs 😭

back to the drawing board with this part

devdumpling · 2026-01-26T18:29:27Z

+    swr: true,
+    getKey: event => {
+      const pkg = getRouterParam(event, 'pkg') ?? ''
+      return `docs:v5:${pkg}`


was rolling multiple versions of this locally during rapid prototyping -- needs to be dropped back before moving from poc

devdumpling · 2026-01-26T18:30:44Z

      })

+      // Shiki doesn't encode > in text content (e.g., arrow functions)
+      html = escapeRawGt(html)


this also fixes a lot of random html validation warnings we're getting

devdumpling · 2026-01-26T18:31:53Z

not sure if this is better defined / housed elsewhere (i didn't see anything for it at first glance flipping through the repo)

devdumpling · 2026-01-27T11:56:00Z

Still trying to debug this, but tough as whatever is failing seems to be specific to the vercel env. I'm guessing I need to configure something for nitro / vercel to make it happy w wasm.

Test urls are here:

I ran the vercel build locally and peeked through the build artifacts. I can see the wasm in the vercel function bundle. It's 4.7mb at the correct path. The patch is there as well and module importing / doc gen works in it. It's under the 25mb vercel limit too so...?

Only things I can think of without having access to the logs are:

cold start time is too high for this
function timeout default is too low for this (maybe it's taking longer than 10s or something?)
memory? i can't imagine this would hit a memory limit but maybe

Or that there's just something completely different wrong I'm missing here (likely).

devdumpling · 2026-01-27T15:00:06Z

+ * Fenced code blocks in descriptions use a subtle left-border style.
+ *
+ * Design rationale: We use two visual styles for code examples:
+ * 1. Boxed style (bg + border + padding) - for formal @example JSDoc tags
+ *    and function signatures. These are intentional, structured sections.
+ * 2. Left-border style (blockquote-like) - for inline code in descriptions.
+ *    These are illustrative/casual and shouldn't compete with the signature.
+ */


LMK if anyone has questions on this. My rationale was that the examples should feel more prominent. For a comparison look at the svelte docs page vs the react one.

React one uses a lot of @example while svelte has inline @description with casual examples.

devdumpling · 2026-01-27T15:01:05Z

-              <a
-                :href="homepageUrl"
-                target="_blank"
-                rel="noopener noreferrer"
-                class="link-subtle font-mono text-sm inline-flex items-center gap-1.5"
-              >
-                <span class="i-carbon-link w-4 h-4" aria-hidden="true" />
-                {{ t('package.links.homepage') }}
-              </a>
-            </li>
+


I dropped the homepage url here in favor of the docs link, but as I'm looking back I wonder if maybe we just add both instead?

devdumpling · 2026-01-27T15:01:46Z

-      "code": "code"
+      "code": "code",
+      "docs": "docs",
+      "socket": "socket"


addresses this on main

devdumpling · 2026-01-27T15:04:59Z

+export function formatType(type?: TsType): string {
+  if (!type) return ''
+
+  // Strip ANSI codes from repr (deno doc may include terminal colors since it's built for that)


this is hacky

devdumpling · 2026-01-27T15:07:10Z

+  "resolutions": {
+    "vite": "^8.0.0-beta.10"
+  },
+  "pnpm": {
+    "overrides": {
+      "sharp": "0.34.5"
+    },
+    "packageExtensions": {
+      "@nuxt/scripts": {
+        "dependencies": {
+          "estree-walker": "^3.0.3"
+        }
+      }
+    },


Did these get dropped since I started working on this? Taking a look now...

ah i see 14ccc37

…-attempt

- Add synchronous highlightCodeSync() function to shiki.ts - Update readme.ts to use shared function instead of duplicating logic - Reduces code duplication between readme and docs rendering

devdumpling mentioned this pull request Jan 26, 2026

[poc] feat: docs (microservice variant) #134

Closed

vercel Bot deployed to Preview January 26, 2026 17:25 View deployment

devdumpling commented Jan 26, 2026

View reviewed changes

Comment thread app/pages/[...package].vue Outdated

devdumpling commented Jan 26, 2026

View reviewed changes

devdumpling force-pushed the dwells-feat-docs-wasm-attempt branch 3 times, most recently from 87ee5b2 to 2987c0c Compare January 27, 2026 11:47

vercel Bot deployed to Preview January 27, 2026 11:52 View deployment

devdumpling force-pushed the dwells-feat-docs-wasm-attempt branch 5 times, most recently from 4fcc2ee to 3d0121c Compare January 27, 2026 13:09

vercel Bot deployed to Preview January 27, 2026 13:16 View deployment

devdumpling added 8 commits January 27, 2026 09:36

feat: prototyping phase for doc gen

92ddf1e

cleanup old prototype

d87bfcd

fix: polish and cleanup post prototype

3d1221d

version selector

5830860

test: add basic e2e and unit tests

582fd36

fix: some validation issues with markdown rendering and parsing

6aaf681

chore: some planning, remove old typedoc ref

24017c0

fix: type issue with DocsResponse default status

6d7a091

devdumpling commented Jan 27, 2026

View reviewed changes

fix: update manifest with upstream

e277a2d

vercel Bot deployed to Preview January 27, 2026 15:16 View deployment

fix: docs link and homepage coexist

ba0ec18

vercel Bot deployed to Preview January 27, 2026 15:30 View deployment

chore: move docs api version back to v1 for caching

c06b1b2

vercel Bot deployed to Preview January 27, 2026 15:49 View deployment

fix: handle invalid urls and errors generating docs

0299f87

danielroe changed the title ~~feat: docs (wasm variant)~~ feat: generate package docs using @deno/doc Jan 27, 2026

fix: some a11y issues

58dbc53

vercel Bot deployed to Preview January 27, 2026 16:20 View deployment

danielroe added 2 commits January 27, 2026 17:31

Merge origin/main into dwells-feat-docs-wasm-attempt

11bcd36

Merge remote-tracking branch 'origin/main' into dwells-feat-docs-wasm…

48887f1

…-attempt

vercel Bot deployed to Preview – npmx.dev January 27, 2026 22:04 View deployment

vercel Bot deployed to Preview – npmx-dev-docs January 27, 2026 22:04 View deployment

danielroe added 3 commits January 27, 2026 22:07

Merge remote-tracking branch 'origin/main' into dwells-feat-docs-wasm…

ed424e1

…-attempt

fix: remove socket link that was incorrectly re-added during merge

d7aacaa

refactor: extract highlightCodeSync for shared code highlighting

458ba1a

- Add synchronous highlightCodeSync() function to shiki.ts - Update readme.ts to use shared function instead of duplicating logic - Reduces code duplication between readme and docs rendering

vercel Bot deployed to Preview – npmx.dev January 27, 2026 22:13 View deployment

vercel Bot deployed to Preview – npmx-dev-docs January 27, 2026 22:14 View deployment

chore: remove reexport

722b3bd

danielroe approved these changes Jan 27, 2026

View reviewed changes

vercel Bot deployed to Preview – npmx.dev January 27, 2026 22:18 View deployment

vercel Bot deployed to Preview – npmx-dev-docs January 27, 2026 22:19 View deployment

danielroe merged commit 8aae4f3 into npmx-dev:main Jan 27, 2026
9 checks passed

vinnymac pushed a commit to vinnymac/npmx.dev that referenced this pull request Jan 28, 2026

feat: generate package docs using @deno/doc (npmx-dev#135)

4c8ce58

Uh oh!

Conversation

devdumpling commented Jan 26, 2026 • edited by danielroe Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Problem

Solution

Architecture

Arch Diagram

Implementation

Alternative approaches

Uh oh!

vercel Bot commented Jan 26, 2026

Uh oh!

vercel Bot commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danielroe commented Jan 26, 2026

Uh oh!

devdumpling commented Jan 26, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devdumpling commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devdumpling commented Jan 26, 2026 •

edited by danielroe

Loading

vercel Bot commented Jan 26, 2026 •

edited

Loading

devdumpling commented Jan 27, 2026 •

edited

Loading