Skip to content

feat(imports): collect Snyk Code and Open Source findings locally - #2168

Open
mldangelo-oai wants to merge 22 commits into
mdangelo/codex/semgrep-platform-collectionfrom
mdangelo/codex/snyk-platform-local
Open

mldangelo-oai wants to merge 22 commits into
mdangelo/codex/semgrep-platform-collectionfrom
mdangelo/codex/snyk-platform-local

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Collect Snyk Open Source and Code findings for a project and import them into the local workbench. This is stacked on #2167, which provides the HTTP transport. Cloud publication follows in #2085.

Changes

  • Add the opt-in codex-security import snyk PROJECT_ID --snyk-org ORG_ID --source-repository-url URL collector. It uses SNYK_TOKEN, defaults to Open Source, and accepts --snyk-product code. The optional --snyk-api-origin selects a regional endpoint.
  • Read the configured regional API with the project and organization in scope. Native report subsets retain each issue and its generated occurrence ID. The ordinary normalized import also accepts caller-defined IDs.
  • Keep issue identity across repeat imports and validate the source repository against the checkout. SDK callers can identify a known private GitLab host with sourceRepositories[].platform = "gitlab" when clone aliases require an independently verified platform.
  • Document the local workflow and sample commands for the scanner import guide.
  • Carry forward feat(imports): collect scoped Semgrep Platform findings #2167. Marked Semgrep Platform reports require the original collection scope when retained findings carry repository, ref, or product evidence, including historical supply-chain findings. Move unrelated annotations that use reserved fields to another key or an ordinary normalized batch.

Testing

Scope follow-up at a6303b59

  • All 358 tests in 15 Semgrep, Snyk, OpenGrep, and Black Duck suites passed, with 2,299 assertions. These cover retained collection scope, historical supply-chain findings, selected files, ordinary scanner inputs, replay, and repository binding. The SDK build, CI build, bundled plugin, types, formatting, and whitespace checks passed.

Earlier validation at c90f2ee1

  • At c90f2ee1, 430 tests passed with 2,669 assertions and one Windows-only skip on Linux. This includes 94 hosted Snyk tests, 91 local import tests, 241 adjacent scanner tests and four comparisons using original-format captures, gzip and real SQLite. SDK types, formatting and all five ordered portable checks passed.
  • Replaying Code and Open Source records retained the source and SQLite IDs. The tests cover selected issue IDs, equivalent native JSON, archived exports, pagination and private Git host mappings. Replaying a Code row with contradictory evidence about its project's source no longer assigns it a different local file. A directory from the Projects API no longer becomes a typed package manifest.
  • Structured JSON errors retain the original vendor message, including tabs and newlines. The human-readable display keeps its existing terminal rendering. The vendor API tests used synthetic responses. Live Snyk qualification is outstanding.

Risk and rollout

The collector requires read access to the selected project. Local storage uses the existing opt-in and source key. A local or Cloud destination must match before evidence can be stored.

At c90f2ee1, the automated review gate reported three P2 findings and one P3 advisory and was blocked. All three independent reviews of the scope update from c90f2ee1 to a6303b59 completed without findings, and the verifier cleared the update. A native issue can omit optional integration details. Replay preserves its original evidence and rejects a conflicting verified target. A malformed ambient Git configuration can affect two Git setup calls in the test fixture. The production collector and the checks above use valid Git configurations.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. No blocking findings in this owner-level merge-safety review of the Snyk collector, saved-report compatibility, or local import integration.

Validation: 27 focused tests, nine portable self-tests, built SDK paginated collection/offline-readback checks for both products, compatibility checks, types, formatting, builds, and the 844-entry package contract passed. An independent review of token origins and response scope found no additional issues.

Local workbench persistence and Git-binding integration remain unverified because of existing fixture-environment constraints; current-head hosted SDK tests were still running. No live Snyk qualification, full-suite pass, or local Windows/macOS validation is claimed.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No blocking findings in this owner-level merge-safety re-review of the Snyk collector, saved-report compatibility, and local import integration. The projected-row and regional-token changes preserve the existing workflow.

Validation: 31 focused tests, nine portable self-tests, built SDK collection and compatibility checks, builds, types, formatting, and the 844-entry package contract passed. Full local persistence, current native Windows behavior, and live Snyk qualification remain unverified.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No blocking findings in this owner-level merge-safety review of the Snyk changes. Hosted subsets retain their native issue IDs, while ordinary reader-defined findings remain supported without hosted context.

Validation: 32 focused tests, nine portable self-tests, built SDK collection/replay and source-mapping checks, builds, types, formatting, and the package contract passed. Exact-tree Linux CI also passed 165 Snyk tests. Local persistence remains environment-limited; Windows/macOS and the full CI matrix were not fully verified. No live Snyk qualification was performed.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No blocking findings in this owner-level merge-safety re-review of Snyk collection, retained native evidence, and local import integration. The replay changes preserve issue identity and distinguish ordinary SDK-authored findings from hosted subsets.

Validation: 120 focused tests, nine portable self-tests, built SDK checks for both products, builds, types, formatting, and the package contract passed. Exact-tree Linux CI passed 7,622 tests, including 167 Snyk tests; six inspected Windows shards also passed. Local persistence remains environment-limited, and the full Windows/macOS matrix and live Snyk service were not independently qualified.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No material findings in this merge-safety review. I checked the complete change, including native scope and identity checks, partial SDK projections, target reconciliation, and the local import boundary.

Validation: 120 focused tests, nine portable self-tests, built collection and projection checks, builds, types, formatting, and the package contract passed. Exact-tree Linux CI passed all six shards, including 168 Snyk-file tests; the available Windows shards also passed. Local SQLite execution and live Snyk qualification remain unverified, and the full Windows workflow was still running.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No material findings in this owner-level merge-safety review of the complete change, including selected issue identity, archived v1 evidence, scope reconciliation, and the local import boundary.

Validation: 104 focused tests, nine portable self-tests, built SDK collection/projection checks, builds, types, formatting, and the package contract passed. Exact-tree CI also passed 169 Snyk tests on Linux and 68 inspected Windows cases. Local persistence and installed-package smoke remain environment-limited; live Snyk behavior and the full Windows/macOS matrix were not independently qualified.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. This owner-level merge-safety pass found no new Snyk-specific issue, but the remote-discovery fixes acknowledged in #2166 are still needed by this import path. A checkout whose literal default fetch URL matches the Snyk repository can be rejected when another named remote exists, because the default URL is omitted. Git 2.55 URL resets also need to reach the intended legacy-metadata check. Please carry those base fixes into this branch before merging. These conclusions follow static review; no runtime reproduction was performed.

The collector intentionally remains read-only and local, with native issue identity and evidence retained. Validation: 28 focused tests, nine portable self-tests, 12 built-SDK functional groups, builds, types and formatting passed. Matching-tree CI passed all eight installed-package jobs; available artifacts had no failures, with one Windows shard still outstanding. Local persistence and live Snyk qualification remain unverified.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. No blocking finding in this owner-level merge-safety pass of the Snyk change. The two remote-discovery issues from the prior review are resolved. Collection intentionally remains read-only and local, preserves native issue identity, and now retains original vendor diagnostics in structured output. Separate Semgrep scope feedback remains tracked on #2167.

Validation at this head: 29 focused tests, eight portable self-tests, eight built-SDK functional groups, builds, types and formatting passed. Matching-tree CI has passed the package build and five installed-package jobs; the full matrix is still running. Local persistence and live Snyk qualification remain unverified.

@mldangelo-oai
mldangelo-oai marked this pull request as ready for review October 31, 2026 20:12
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T11:13:01.779824Z a6303b5 New commits
🔒 Security Review ✅ Completed 2026-11-01T11:14:18.964920Z a6303b5 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants