Skip to content

feat(cli): let wrappers set the integration tag with FIRECRAWL_INTEGRATION - #336

Merged
erikengervall merged 31 commits into
mainfrom
capy/firecrawl-integration-env
Oct 20, 2026
Merged

erikengervall merged 31 commits into
mainfrom
capy/firecrawl-integration-env

Conversation

@erikengervall

@erikengervall erikengervall commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Every command hardcodes integration: 'cli', so tools that wrap the CLI always show up as cli. The CLI now reads FIRECRAWL_INTEGRATION, the same variable firecrawl-mcp reads, and sends it as the X-Integration request header. The body (and query) integration stays cli.

  • New src/utils/integration.ts: integrationHeaders() returns { 'X-Integration': <trimmed value> } when the variable is set. There is no underscore rule and no list of names in the CLI: the API attributes a recognized name and silently ignores anything else, so the tag can never make a request fail. The only values dropped client side are ones that cannot be sent as a header at all (control or non-ASCII characters), which would otherwise make fetch throw.
  • The header goes on every API request the CLI makes:
    • SDK client requests (scrape, map, crawl, agent, alexandria, browser, and the research, developer and gov GETs that go through the client's http), via withIntegrationHeader() in getClient();
    • the keyless path (keylessRequest, keylessGet);
    • the raw fetch calls: feedback, search-feedback, interact, parse, monitor, credit-usage, status, doctor, list and terms.
  • The SDK (4.50.0) has no option for custom default headers, so withIntegrationHeader() adds it to the SDK client's axios defaults. The CLI already reaches into the same http member for research and developer search. The smallest proper fix is a headers?: Record<string, string> option on FirecrawlClientOptions, merged into the axios defaults in HttpClient; once an SDK release has it, the CLI can pass the header through that option instead.
  • origin and the keyless X-Origin header are unchanged.
  • README documents the variable next to the telemetry setting.
  • vitest.config.mjs sets FIRECRAWL_INTEGRATION to empty for test runs, so a value exported in the shell doesn't change request assertions.
  • Version bump to 1.26.15.

The API side: a valid X-Integration header must beat a body or query value of cli on every endpoint the CLI calls. That rule lands in firecrawl/firecrawl#5260; until it ships, requests keep being attributed to cli, and nothing fails either way.

Testing

  • pnpm type-check, pnpm format:check, pnpm build pass; pnpm test: 768 passed, with and without FIRECRAWL_INTEGRATION=hermes exported.
  • New tests in src/__tests__/utils/integration.test.ts run against a local HTTP server: rtk, hermes and _rtk go out as X-Integration on SDK client POST and GET requests while the body keeps integration: cli; a junk value (not a valid tag!) is sent and the request succeeds; a value that cannot be a header is dropped and the request still succeeds. feedback.test.ts checks the header on feedback and search feedback.
  • The built CLI against the production API: firecrawl scrape https://example.com exits 0 with FIRECRAWL_INTEGRATION set to hermes, _rtk and not a valid tag!.

Merge order

Merge after #337 (merged, 1.26.14) and once firecrawl/firecrawl#5260 is deployed, so the README matches the API. Merging publishes 1.26.15.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 21 files

Shadow auto-approve: would auto-approve with 4 open P3 issues. Adds an opt-in FIRECRAWL_INTEGRATION env var to tag CLI API requests (defaulting to cli) across commands, with tests and README docs; bounded because only _-prefixed values are honored and default behavior is unchanged.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread README.md Outdated
Comment thread src/commands/feedback.ts Outdated
Comment thread src/commands/search-feedback.ts Outdated
Comment thread README.md Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds an opt-in FIRECRAWL_INTEGRATION env var so CLI wrappers can tag API requests, defaulting to cli and honoring only underscore-prefixed values; focused tests and README docs cover the behavior.

View guided diff | Turn on auto-fix | Re-trigger cubic

@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@capy-ai[bot] I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 28 files

Confidence score: 4/5

  • src/commands/setup.ts still prints skip notices and the summary when quiet is set, so init can lose its compact output for env-backed keys. Gate those messages on quiet.
  • package.json bumps the exact-pinned firecrawl SDK from 4.44.0 to 4.49.1 without explaining the reason. Document why the integration needs this version or split the bump into its own PR.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="package.json">

<violation number="1" location="package.json:88">
P3: This bumps the exact-pinned `firecrawl` SDK from 4.44.0 to 4.49.1, which the PR description never mentions. Either document why the new integration tag needs this SDK version, or split the bump into its own PR so it can be reviewed and rolled back separately.</violation>
</file>

<file name="src/commands/setup.ts">

<violation number="1" location="src/commands/setup.ts:777">
P2: The skip notices and the summary block print even when `quiet` is set. `init` calls `installMcp` with `quiet: true` and `agent: 'all'` when the key is env-backed, so the compact init output gets the full summary. Gate these logs on `!options.quiet`, and keep the `console.error` failure output.</violation>
</file>

Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/setup.ts Outdated
Comment thread package.json Outdated
Comment thread src/__tests__/commands/setup.test.ts Outdated
@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@capy-ai[bot] I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 28 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds opt-in FIRECRAWL_INTEGRATION for CLI requests; default stays cli and _-prefixed values are trimmed and validated, with tests. SDK/setup changes in the diff are the stacked #326/#337 changes named in the merge order.

View guided diff | Turn on auto-fix | Re-trigger cubic

@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@capy-ai[bot] I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 28 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds opt-in FIRECRAWL_INTEGRATION tagging for CLI requests; default stays cli, only _-prefixed values pass through, with tests and docs. SDK/setup/init changes are the stacked #326/#337 changes scheduled to merge first.

View guided diff | Turn on auto-fix | Re-trigger cubic

@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@capy-ai[bot] I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 28 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds opt-in FIRECRAWL_INTEGRATION tagging for CLI requests; default stays cli and only _-prefixed values pass, with tests/docs. SDK/setup/init changes are the stacked #326/#337 work merging first.

View guided diff | Turn on auto-fix | Re-trigger cubic

@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review

@capy-ai[bot] I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 28 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Adds opt-in FIRECRAWL_INTEGRATION tagging across CLI commands; default stays cli and only _-prefixed values pass, matching the API schema, with tests and docs. Delta since prior review is only the firecrawl SDK bump, which belongs to stacked #337.

View guided diff | Turn on auto-fix | Re-trigger cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Opt-in FIRECRAWL_INTEGRATION replaces the hardcoded 'cli' tag across commands; default stays cli and only _-prefixed values are sent, with tests and docs. Delta since prior approval is just a README clarification.

View guided diff | Turn on auto-fix | Re-trigger cubic

…dk-4-49-1

# Conflicts:
#	package.json
#	pnpm-lock.yaml
…egration-env

# Conflicts:
#	package.json
#	src/__tests__/commands/feedback.test.ts
…every API request

The body integration stays cli. The CLI keeps no list of names: the API
ignores a value it does not recognize, so the tag can never fail a request.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 27 files (changes from recent commits).

Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread README.md Outdated
Comment thread src/commands/doctor.ts
Comment thread src/utils/client.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread README.md Outdated
Comment thread README.md Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 1 file (changes from recent commits).

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Shadow auto-approve: would auto-approve. Opt-in FIRECRAWL_INTEGRATION adds X-Integration to all CLI API requests while body/query stay cli; default is unchanged, values are sanitized, and tests plus README cover the feature, making it bounded and clearly beneficial.

View guided diff | Turn on auto-fix | Re-trigger cubic

@erikengervall
erikengervall merged commit 778ec49 into main Oct 20, 2026
8 checks passed
@erikengervall
erikengervall deleted the capy/firecrawl-integration-env branch October 20, 2026 04:20
erikengervall added a commit that referenced this pull request Oct 20, 2026
Squashed onto main 82229b1. The branch's merged-in work from #326, #336, #337, #338 and #339 is already on main; this commit holds only the Amp, Grok Build and Zed setup mcp change and the bump to 1.26.17.
erikengervall added a commit that referenced this pull request Oct 20, 2026
Squashed onto main 82229b1. The branch's merged-in work from #326, #336, #337, #338 and #339 is already on main; this commit holds only the Amp, Grok Build and Zed setup mcp change and the bump to 1.26.17.

Co-authored-by: erikengervall <5392475+erikengervall@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant