Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ jobs:
run: pnpm build:test

- name: ♿ Accessibility audit (Lighthouse - ${{ matrix.mode }} mode)
run: ./scripts/lighthouse-a11y.sh
run: pnpm test:a11y:prebuilt
env:
LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
LIGHTHOUSE_COLOR_MODE: ${{ matrix.mode }}
Expand Down
46 changes: 42 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ This focus helps guide our project decisions as a community and what we choose t
- [Testing](#testing)
- [Unit tests](#unit-tests)
- [Component accessibility tests](#component-accessibility-tests)
- [Lighthouse accessibility tests](#lighthouse-accessibility-tests)
- [End to end tests](#end-to-end-tests)
- [Test fixtures (mocking external APIs)](#test-fixtures-mocking-external-apis)
- [Submitting changes](#submitting-changes)
Expand Down Expand Up @@ -111,6 +112,7 @@ pnpm test # Run all Vitest tests
pnpm test:unit # Unit tests only
pnpm test:nuxt # Nuxt component tests
pnpm test:browser # Playwright E2E tests
pnpm test:a11y # Lighthouse accessibility audits
```

### Project structure
Expand Down Expand Up @@ -598,6 +600,40 @@ A coverage test in `test/unit/a11y-component-coverage.spec.ts` ensures all compo
> [!IMPORTANT]
> Just because axe-core doesn't find any obvious issues, it does not mean a component is accessible. Please do additional checks and use best practices.

### Lighthouse accessibility tests

In addition to component-level axe audits, the project runs full-page accessibility audits using [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci). These test the rendered pages in both light and dark mode against Lighthouse's accessibility category, requiring a perfect score.

#### How it works

1. The project is built in test mode (`pnpm build:test`), which activates server-side fixture mocking
2. Lighthouse CI starts a preview server and audits three URLs: `/`, `/search?q=nuxt`, and `/package/nuxt`
3. A Puppeteer setup script (`lighthouse-setup.cjs`) runs before each audit to set the color mode and intercept client-side API requests using the same fixtures as the E2E tests

#### Running locally

```bash
# Build + run both light and dark audits
pnpm test:a11y

# Or against an existing test build
pnpm test:a11y:prebuilt

# Or run a single color mode manually
pnpm build:test
LIGHTHOUSE_COLOR_MODE=dark ./scripts/lighthouse-a11y.sh
```

This requires Chrome or Chromium to be installed. The script will auto-detect common installation paths. Results are printed to the terminal and saved in `.lighthouseci/`.

#### Configuration

| File | Purpose |
| ---------------------------- | --------------------------------------------------------- |
| `.lighthouserc.cjs` | Lighthouse CI config (URLs, assertions, Chrome path) |
| `lighthouse-setup.cjs` | Puppeteer script for color mode + client-side API mocking |
| `scripts/lighthouse-a11y.sh` | Shell wrapper that runs the audit for a given color mode |

### End to end tests

Write end-to-end tests using Playwright:
Expand All @@ -619,10 +655,12 @@ E2E tests use a fixture system to mock external API requests, ensuring tests are
- Serves pre-recorded fixture data from `test/fixtures/`
- Enabled via `NUXT_TEST_FIXTURES=true` or Nuxt test mode

**Client-side mocking** (`test/e2e/test-utils.ts`):
**Client-side mocking** (`test/fixtures/mock-routes.cjs`):

- Uses Playwright's route interception to mock browser requests
- All test files import from `./test-utils` instead of `@nuxt/test-utils/playwright`
- Shared URL matching and response generation logic used by both Playwright E2E tests and Lighthouse CI
- Playwright tests (`test/e2e/test-utils.ts`) use this via `page.route()` interception
- Lighthouse tests (`lighthouse-setup.cjs`) use this via Puppeteer request interception
- All E2E test files import from `./test-utils` instead of `@nuxt/test-utils/playwright`
- Throws a clear error if an unmocked external request is detected

#### Fixture files
Expand Down Expand Up @@ -670,7 +708,7 @@ URL: https://registry.npmjs.org/some-package
You need to either:

1. Add a fixture file for that package/endpoint
2. Update the mock handlers in `test/e2e/test-utils.ts` (client) or `modules/runtime/server/cache.ts` (server)
2. Update the mock handlers in `test/fixtures/mock-routes.cjs` (client) or `modules/runtime/server/cache.ts` (server)

## Submitting changes

Expand Down
69 changes: 68 additions & 1 deletion lighthouse-setup.cjs
Original file line number Diff line number Diff line change
@@ -1,13 +1,28 @@
/**
* Lighthouse CI puppeteer setup script.
* Sets the color mode (light/dark) before running accessibility audits.
*
* Sets the color mode (light/dark) before running accessibility audits
* and intercepts client-side API requests using the same fixture data
* as the Playwright E2E tests.
*
* The color mode is determined by the LIGHTHOUSE_COLOR_MODE environment variable.
* If not set, defaults to 'dark'.
*
* Request interception uses CDP (Chrome DevTools Protocol) Fetch domain
* at the browser level, which avoids conflicts with Lighthouse's own
* Puppeteer-level request interception.
*/

const mockRoutes = require('./test/fixtures/mock-routes.cjs')

module.exports = async function setup(browser, { url }) {
const colorMode = process.env.LIGHTHOUSE_COLOR_MODE || 'dark'

// Set up browser-level request interception via CDP Fetch domain.
// This operates below Puppeteer's request interception layer so it
// doesn't conflict with Lighthouse's own setRequestInterception usage.
await setupCdpRequestInterception(browser)

const page = await browser.newPage()

// Set localStorage before navigating so @nuxtjs/color-mode picks it up
Expand All @@ -21,3 +36,55 @@ module.exports = async function setup(browser, { url }) {
// Close the page - Lighthouse will open its own with localStorage already set
await page.close()
}

/**
* Set up request interception using CDP's Fetch domain on the browser's
* default context. This intercepts requests at a lower level than Puppeteer's
* page.setRequestInterception(), avoiding "Request is already handled!" errors
* when Lighthouse sets up its own interception.
*
* @param {import('puppeteer').Browser} browser
*/
async function setupCdpRequestInterception(browser) {
// Build URL pattern list for CDP Fetch.enable from our route definitions
const cdpPatterns = mockRoutes.routes.map(route => ({
urlPattern: route.pattern.replace('/**', '/*'),
requestStage: 'Request',
}))
Comment on lines +50 to +53
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

find . -name "mock-routes*" -o -name "lighthouse-setup*" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 115


🏁 Script executed:

git ls-files | grep -E "(mock-routes|lighthouse-setup)" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 111


🏁 Script executed:

git ls-files | grep -E "\.cjs$" | head -20

Repository: npmx-dev/npmx.dev

Length of output: 129


🏁 Script executed:

cat -n lighthouse-setup.cjs | sed -n '40,65p'

Repository: npmx-dev/npmx.dev

Length of output: 1238


🏁 Script executed:

cat -n test/fixtures/mock-routes.cjs

Repository: npmx-dev/npmx.dev

Length of output: 15791


🏁 Script executed:

grep -r "downloads/point" test/ --include="*.ts" --include="*.js" --include="*.cjs"

Repository: npmx-dev/npmx.dev

Length of output: 471


🏁 Script executed:

grep -r "v1/packages" test/ --include="*.ts" --include="*.js" --include="*.cjs"

Repository: npmx-dev/npmx.dev

Length of output: 43


🏁 Script executed:

grep -r "lighthouse-setup" test/ .github/ --include="*.ts" --include="*.js" --include="*.cjs" --include="*.json" --include="*.yaml" --include="*.yml"

Repository: npmx-dev/npmx.dev

Length of output: 148


🏁 Script executed:

cat -n test/nuxt/composables/use-npm-registry.spec.ts | sed -n '1,50p'

Repository: npmx-dev/npmx.dev

Length of output: 1954


🏁 Script executed:

rg "https://api.npmjs.org/downloads" test/ -A 2 -B 2

Repository: npmx-dev/npmx.dev

Length of output: 1423


🏁 Script executed:

cat -n lighthouse-setup.cjs | sed -n '1,100p'

Repository: npmx-dev/npmx.dev

Length of output: 3971


🏁 Script executed:

rg "Fetch\.enable" lighthouse-setup.cjs -A 5 -B 5

Repository: npmx-dev/npmx.dev

Length of output: 734


The pattern conversion from /** to /* will break CDP interception for multi-segment paths.

CDP Fetch patterns use glob syntax where * matches a single path segment and /** matches recursively. Converting patterns like https://api.npmjs.org/** to https://api.npmjs.org/* prevents CDP from pausing requests to endpoints with multiple path segments (e.g., https://api.npmjs.org/downloads/point/last-week/vue). Requests that don't match CDP's patterns won't pause, so mockRoutes.matchRoute() is never called and the mock response is never served.

Keep the patterns unchanged or use /** syntax if that's supported by CDP's urlPattern implementation.


// Listen for new targets so we can attach CDP interception to each page
browser.on('targetcreated', async target => {
if (target.type() !== 'page') return

try {
const cdp = await target.createCDPSession()

cdp.on('Fetch.requestPaused', async event => {
const requestUrl = event.request.url
const result = mockRoutes.matchRoute(requestUrl)

if (result) {
const body = Buffer.from(result.response.body).toString('base64')
await cdp.send('Fetch.fulfillRequest', {
requestId: event.requestId,
responseCode: result.response.status,
responseHeaders: [
{ name: 'Content-Type', value: result.response.contentType },
{ name: 'Access-Control-Allow-Origin', value: '*' },
],
body,
})
} else {
await cdp.send('Fetch.continueRequest', {
requestId: event.requestId,
})
}
})
Comment on lines +62 to +82
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add error handling within the Fetch.requestPaused handler.

The cdp.send() calls inside the event handler are not wrapped in a try/catch. If the target closes whilst a request is paused (before fulfillRequest or continueRequest completes), this will throw an unhandled promise rejection.

🛡️ Proposed fix to add error handling
       cdp.on('Fetch.requestPaused', async event => {
+        try {
           const requestUrl = event.request.url
           const result = mockRoutes.matchRoute(requestUrl)
 
           if (result) {
             const body = Buffer.from(result.response.body).toString('base64')
             await cdp.send('Fetch.fulfillRequest', {
               requestId: event.requestId,
               responseCode: result.response.status,
               responseHeaders: [
                 { name: 'Content-Type', value: result.response.contentType },
                 { name: 'Access-Control-Allow-Origin', value: '*' },
               ],
               body,
             })
           } else {
             await cdp.send('Fetch.continueRequest', {
               requestId: event.requestId,
             })
           }
+        } catch {
+          // Target may have closed mid-request; safe to ignore.
+        }
       })


await cdp.send('Fetch.enable', { patterns: cdpPatterns })
} catch {
// Target may have been closed before we could attach.
// This is expected for transient targets like service workers.
}
})
}
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
"generate:fixtures": "node scripts/generate-fixtures.ts",
"generate:lexicons": "lex build --lexicons lexicons --out shared/types/lexicons --clear",
"test": "vite test",
"test:a11y": "pnpm build:test && pnpm test:a11y:prebuilt",
"test:a11y:prebuilt": "LIGHTHOUSE_COLOR_MODE=dark ./scripts/lighthouse-a11y.sh && LIGHTHOUSE_COLOR_MODE=light ./scripts/lighthouse-a11y.sh",
"test:browser": "pnpm build:test && pnpm test:browser:prebuilt",
"test:browser:prebuilt": "playwright test",
"test:browser:ui": "pnpm build:test && pnpm test:browser:prebuilt --ui",
Expand Down
Loading
Loading