[erikengervall]

Adds firecrawl batch-scrape, a command for the /v2/batch/scrape API, and teaches the bundled skills when to use it and how to extract JSON fields from a page.

Today firecrawl scrape url1 url2 ... runs concurrent single scrapes that keep markdown only. batch-scrape runs one async job that keeps every requested format, supports JSON extraction, webhooks and cancellation, and can be checked later by job ID.

Command

• URLs come from arguments and/or --urls-file (one per line; blank lines and # comments ignored). URLs are normalised and de-duplicated.
• A job ID instead of URLs checks status; --status and --cancel work like crawl.
• Scrape options: --format, --only-main-content, --wait-for, --max-age, and --scrape-options/--scrape-options-file as the base that explicit flags override. --schema, --schema-file and --prompt add the json format; --format json without either fails locally.
• Job options: --wait, --progress, --poll-interval, --timeout, --webhook, --max-concurrency, --ignore-invalid-urls, -o, --pretty. Without --wait it prints the job ID and a status hint, matching crawl.
• Uses the SDK's startBatchScrape, batchScrape, getBatchScrapeStatus and cancelBatchScrape with integration: 'cli'.

Skills and docs

• firecrawl-scrape: new "Structured JSON from a page" (scrape --format json --schema) and "Many known URLs" (batch-scrape) sections.
• firecrawl: a command-table row and a pointer for batch scraping.
• README: a batch-scrape section after crawl.
• Version 1.26.10.

Tests

• 19 unit tests for URL collection, option building, job-ID routing, wait and no-wait, progress polling and cancel; cli-aliases covers the job-status case against the built CLI.
• pnpm run type-check, pnpm run format:check and pnpm test pass.
• Live runs against the API: a two-URL wait with markdown,links, a no-wait start then status by job ID, --schema and --prompt extraction, --urls-file then --cancel, and the two local error paths. Webhook delivery and a live --timeout expiry were not exercised.

---

Summary by cubic

Adds firecrawl batch-scrape, a command for the /v2/batch/scrape API, and teaches the bundled skills when to use it and how to extract JSON fields from a page. Unlike multi-URL scrape, which keeps markdown only, batch-scrape runs one async job that keeps every requested format, supports JSON extraction, webhooks, and cancellation, and can be checked later by job ID.

• URLs come from arguments and/or --urls-file (one per line, blank lines and # comments ignored), normalised and de-duplicated.
• A job ID instead of URLs checks status; --status and --cancel work like crawl.
• --schema, --schema-file, or --prompt add the json format and keep existing --scrape-options formats; a json format without any of them fails locally.
• Uses the SDK's startBatchScrape, batchScrape, getBatchScrapeStatus, and cancelBatchScrape with integration: 'cli'; version bumped to 1.26.18.
• Skills, README, and tests updated with batch scrape usage and JSON extraction guidance.

^{Written for commit 9806545. Summary will update on new commits.}

[cubic-dev-ai]

All reported issues were addressed across 9 files

<sub>Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 4 files (changes from recent commits).

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

<sub>Shadow auto-approve: would auto-approve. Adds a new batch-scrape command (option merging, job status/cancel, docs, skills) with unit tests; no existing command behavior changes, and the dismissed option-merging findings are covered by the current validation and documented explicit-flag precedence.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>