Skip to content

[poc] feat: docs (microservice variant)#134

Closed
devdumpling wants to merge 10 commits intonpmx-dev:mainfrom
devdumpling:dwells-feat-docs-microservice
Closed

[poc] feat: docs (microservice variant)#134
devdumpling wants to merge 10 commits intonpmx-dev:mainfrom
devdumpling:dwells-feat-docs-microservice

Conversation

@devdumpling
Copy link
Copy Markdown
Contributor

@devdumpling devdumpling commented Jan 26, 2026

Overview

This PR is one of two variants adding autogenerated documentation pages, accessible via a nice little link and routing to /docs/{package}/v/{version}.

If a package provides a homepage in its manifest, the link will default to that. Otherwise, we attempt to generate docs via deno docs in a microservice. More on this below.

Note: in the other variant, we use WASM instead of a microservice. This has tradeoffs. See this PR.

Problem

Okay, so npmx.dev currently shows all kinds of fun package data (metadata, READMEs, file contents, etc). But it lacks a feature I love from JSR/crates/other modern package browsers... autogenerated docs off type definitions.

Generating these kind of docs is challenging for npm packages because:

  • npm pkgs don't ship in a standard doc format
  • TS types may be inline, in .d.ts, @types/*, jsdocs, or any combination
  • extracting and parsing that requires complex ASTs and compute overhead

Solution

Luckily, there's tons of prior art here. In particular, Deno and the JSR folks basically already solved this problem with deno doc.

Note: we should make sure they're cool with us using this if we end up going this direction... since we're sort of competing in a sense.

Architecture

In development, this variant has a version that will shell out deno doc --json in a subprocess. This makes it easier for local development and you can more easily test with a local installation of deno. This is what I used to build out most of this.

In production, the idea is to call a Deno microservice hosted on Vercel (since vercel node.js/edge runtimes don't have Deno). For this we'd need to use the vercel community runtime (which I tried to configure here but don't have access to the vercel project, so)

see: https://github.com/vercel-community/deno

Arch Diagram

Screenshot 2026-01-26 at 12 03 15 PM

Implementation

This was quite a ride for me, as much of it is transforming the format from deno docs --json, but there's some key points.

Most of the heavy lifting is done in /server/utils/docs/ and is broken down into a few submodules:

  • index | entry point, calls generateDocsWithDeno(pkg, version)
  • client | deno integration
  • processing | flattens namespaces, merge overloads, build symbol lookup, etc
  • render | generate HTML with Shiki syntax highlighting server side to not bloat the JS payload
  • text | escape HTML, parse JSDocs links, render markdown
  • format | format typescript signatures
  • types | internal types

The microservice is pretty straightforward. I'm open to opinions here. Just threw up the easiest deno implementation I could think of. Big plus for how easy it was honestly. I haven't really built a deno micro before this!

The frontend implementation simply fetches from the api endpoint (/api/registry/docs/{pkg}/v/{version}) and renders the pre-generated HTML in v-html. I'm still learning vue/nuxt, so let me know if I'm doing anything dumb here.

The UI could probably be improved / iterated on. I left that out of scope for this POC / initial implementation.

A note on esm.sh

I needed to use esm.sh here instead of our existing calls to jsDeliver. Why?

In my testing, I couldn't get jsDeliver to resolve @types/* packages for untyped packages properly and kept hitting module resolution issues with varying .js|.ts extension 😢. I'm betting there's a way to make it work, but esm.sh worked on the first try, so I went with that. It also serves the typescript types natively with a x-typescript-types header which I use here.

If we're concerned about having two underlying providers here (jsDeliver and esm.sh) I can take this back to the drawing board / iterate.

config

There's some necessary config here. Can change naming here. I don't have access to our vercel env, but we'd basically need something like the following:

DOCS_API_URL -> npmx.dev vercel config -> this is the microservice URL (default in my setup is https://docs-api.npmx.dev/api/generate)
DOCS_API_SECRET -> npmx.dev vercel config -> bearer token for auth
API_SECRET -> docs-api vercel config -> expected bearer token

Alternative approaches

So in truth I don't love this approach. I mean, it's nice in a sense that it's a decoupled service. But as a rule I hate standing up infrastructure if there's a more maintainable way around it.

Hence there are two other approaches we ought to consider:

  1. A WASM variant -- instead of a microservice, run @deno/doc WASM directly in Node.js server side
  2. Build our own parsing / AST solution natively, likely in Rust (I tried doing this with TypeDoc and it was clumsy and slow). We don't currently have any Rust stuff in the codebase so big decision there. Might be better solve long term.

Option 2 may be interesting, but also comes with the steep cost of diverging from the incredible work the JSR team already put into this. If we want to build out other rust stuff here though, might make sense. We could also theoretically do more interesting stuff than is possible with just the json data we get from deno doc --json.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jan 26, 2026

@devdumpling is attempting to deploy a commit to the danielroe Team on Vercel.

A member of the Team first needs to authorize it.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jan 26, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
npmx.dev Ready Ready Preview, Comment Jan 26, 2026 5:25pm

Request Review

@danielroe
Copy link
Copy Markdown
Member

deployed, if you want to test ✅

Comment thread docs-api/vercel.json
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haven't ever used the community deno runtime, so TBD how well this works

happy to try it on my private vercel first and let you know, but based on their docs should be as simple as a config file like this

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happened to be looking around, I'm assuming this is the package from @vercel-community/deno (I don't do Vercel much)? The one time I had to actually use Vercel it was with this package, which is unmaintained and it was a pretty awful experience. That project (which I no longer work on) now uses @lowlighter/vercel-deno. Simon (lowlighter) himself is pretty AFK, so I don't know how much of an improvement it is (again, haven't worked on it recently), but it might be worth considering.

@devdumpling devdumpling force-pushed the dwells-feat-docs-microservice branch from e1c2557 to e30e128 Compare January 27, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants