fix(crawl): surface API errors and failed pages from crawl status

[RaphaelFakhri]

Summary

firecrawl_check_crawl_status, and the polling inside firecrawl_crawl, read job state through the SDK's HTTP layer directly. That path has two defects:

• A non-2xx answer reaches the caller as the raw transport error. Checking an unknown or expired crawl ID returns Tool 'firecrawl_check_crawl_status' execution failed: Request failed with status code 404 instead of the API's Job not found.
• While following next links, a page that answers success: false stops the loop without an error. The call returns the pages read so far with status: "completed" and next: null, so a partial crawl looks finished.

Changes

• Add getCoreJson, which runs a GET through the SDK's HTTP layer and raises a non-2xx answer as CoreHttpError carrying the API's error message, HTTP status and agent hints. Because the status is preserved, the existing 401 credential recovery still applies.
• Use it for the first status request and for every pagination request.
• Throw when the first response has success: false, and when a later page has success: false, instead of breaking out of the loop.
• Add tests/crawl-status.test.mjs with a fake API covering a two-page crawl, an unknown job, and a page that fails mid-pagination.
• Add a changelog entry.

Test plan

pnpm run build
node --test tests/crawl-status.test.mjs

Before the change: 3 tests, 1 pass, 2 fail (reports the API error for an unknown crawl, fails when a result page cannot be read). After the change: 3 tests, 3 pass. npx eslint src/index.ts and npx tsc --noEmit report no problems.

The two-page test passes both before and after the change, so it guards against regressions in the join behavior.

---

Summary by cubic

Makes crawl status checks surface the API's real error messages and fail loudly when a paginated result can't be read.

Previously, firecrawl_check_crawl_status (and polling inside firecrawl_crawl) returned raw transport errors like Request failed with status code 404 for unknown jobs, and silently returned partial results as a completed crawl when a follow-up page failed. Now both cases fail with the API's own error message while preserving the HTTP status so the existing 401 credential recovery still applies.

• Throws CoreHttpError carrying the API's error message, status, and agent hints for non-2xx responses and for 2xx replies with success: false.
• Raises an error when the initial response or any paginated page returns success: false.
• Adds tests/crawl-status.test.mjs with a fake API covering a two-page crawl, an unknown job, a mid-pagination failure, and a first-response failure.

^{Written for commit 1e9a61b. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 3 files

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[RaphaelFakhri]

Thanks for the review. Three of the four findings are fixed in bc5f44c; one is not changed.

• Hints on success: false (fixed): both the first-response and pagination-page failures now throw CoreHttpError with readAgentHints(...), so the hints reach the error text and the action log. The test asserts the hint appears in the tool result.
• First-response success: false test (fixed): a new fixture answers the initial /v2/crawl/:id request with HTTP 200 and { success: false, error }, and the test asserts the error and hints.
• RPC harness (fixed): pending requests now reject with the exit code and signal when the server process exits, and each request clears its timer when it settles, matching StdioMcpClient in mcp-smoke.test.mjs.
• payload.success !== true (not changed): the check is !payload.success, which already rejects false, undefined and null. The API returns a boolean here, and the previous code used the same truthiness check, so a stricter comparison would not change behavior for real responses.

[cubic-dev-ai]

All reported issues were addressed across 2 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Auto-approved: Crawl status polling now surfaces the API's own error messages and fails loudly when a result page cannot be read, instead of returning partial results as completed; new tests cover unknown jobs and mid-pagination failures.

View guided diff | Re-trigger cubic</sub>

[RaphaelFakhri]

Rebased onto current main as 95ef631. src/index.ts moved to src/server.ts, so the change now lives in getCrawlStatusWithOrigin there, with no change in behavior. pnpm test passes (171 tests), and the 3 failure-path tests in tests/crawl-status.test.mjs fail without the fix.

[RaphaelFakhri]

Rebased onto v3.30.2. Since agent hints were removed in #530, the three CoreHttpError throws no longer pass a hints argument, and the hint assertions are gone from the test. The rest of the change is the same as the approved version. The full suite passes (173 tests).