Repository navigation
feat(ssp-templates): discovery worker and periodic job - #525
Merged
gusfcarvalho merged 2 commits intoOct 22, 2026
Merged
Conversation
|
Important Review skippedBot user detected. To trigger a single review, invoke the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Oct 19, 2026
This was referenced Oct 19, 2026
Apply SSP templates to the latest evidence: create one SSP per group (deterministic id, rendered title/description/props, the template's profiles attached), record it as an instance, and learn the group's scope. A periodic River job (CCF_SSP_TEMPLATE_DISCOVERY_SCHEDULE) runs every enabled template; CCF_SSP_TEMPLATE_STALE_AFTER configures stale reporting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ccf-lisa
Bot
force-pushed
the
feat/ssp-templates-4-discovery
branch
from
October 19, 2026 14:53
daa9052 to
cf52115
Compare
gusfcarvalho
pushed a commit
that referenced
this pull request
Oct 22, 2026
## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B1 of 5** (B1–B5: #522, #523, #524, #525, #526), stacked on the SSP scope stack (#516–#520, top `feat/ssp-scope-5-auto-filters`). ## What it adds - Models `SSPTemplate` and `SSPTemplateInstance` (`internal/service/relational/ssp_templates.go`), in both AutoMigrate lists and both drop lists. Their foreign keys (instance → template `ON DELETE CASCADE`, instance → SSP `ON DELETE SET NULL`) are added by `EnsureSSPConstraints`, which is `EnsureSSPScopeConstraints` renamed, with two keys appended. AutoMigrate here does not enforce gorm FK tags. - Package `internal/service/relational/ssptemplates`: - `Build`: validation and normalization. Keys are trimmed, lower-cased and sorted (1–5). Scope-label groups are normalized the same way (≤10 groups of 1–5 labels). Keys and labels follow `sspscope.Normalize`'s key rules, so anything a template accepts can be stored as a scope entry. `min-streams` must be ≥ 1. The title template is required, and the title, description and prop templates must parse. The labelfilter must be valid. - `Service`: list/get/create/update/delete. Profile ids must exist. Keys cannot change on update (400). - Admin routes `/api/admin/ssp-templates` (list, create, get, update, delete), behind `JWTMiddleware` + `pep.Authorize(authz.ResourceAdmin, authz.ActionManage)`, like `/admin/subject-templates`. JSON is kebab-case. Swagger is regenerated. - **Contract addition from review**: template responses also return `ssp-count` and `last-run-at`. - `ssp-count` is the number of instances whose SSP still exists. - `last-run-at` is a new nullable `last_run_at` column. It stays `null` until B4's discovery job sets it, and Update preserves it. Not in this PR: preview (B3), SSP creation (B2/B4), discovery, run, instances routes (B4). ## How it was tested - Unit: `ssptemplates/validate_test.go`. It covers normalization, defaults, and every rejection rule. - Integration (`internal/api/handler/templates/ssp_template_integration_test.go`) runs under the embedded Cedar PDP: - an admin creates, lists, gets, updates and deletes templates; - changing keys is rejected; - a contributor gets 403 on every route; - 400/404 cases; - the FK behaviour and `ssp-count`: deleting an SSP nulls its instance's `ssp_id` and drops it from `ssp-count`, and deleting a template drops its instances and keeps its SSPs. - Gate on this branch: - `make swag`, `make lint` (only the existing SA1019 in `internal/api/metrics.go`), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/api/handler`, `internal/api/handler/templates`, `internal/service`, `internal/service/relational`, `internal/service/relational/sspscope`; - `make check-diff`. Changed lines excluding generated swagger: 1000. ## Deviations from design - The instance model has exactly the design's fields. The `skipped-entries` count and sample reported by the instances route are added in B4, with the code that writes them. - Duplicate scope-label groups (after normalization) are rejected with 400, not merged silently. Duplicate profile ids are de-duplicated. - An empty filter (`{}` / `{"scope":{}}`) matches everything, so it is stored as no filter. - The `enabled` column has no database default (the design has `default:true`). gorm writes a column default over a false `Enabled` on create, so a disabled template was stored as enabled. `Build` always sets `Enabled`, with nil meaning true. - The list route is paginated (`svc.ListResponse`, `page`/`limit`) like subject templates. - The response timestamps are kebab-case (`created-at`/`updated-at`), unlike subject templates' camelCase ones. ## Open questions for reviewers - Templates are only checked to parse. Should save also reject templates that reference a field that is not one of the keys (keys are lower-cased, so `{{ ._APP }}` would render empty)? The design asks only for parse errors. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
gusfcarvalho
pushed a commit
that referenced
this pull request
Oct 22, 2026
## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B5 of 5**. It is the second half of the design's B4, split to keep each PR under 1000 lines. Base: B4 (#525, `feat/ssp-templates-4-discovery`). Below it: B3 (#524), B2 (#523), B1 (#522) and the SSP scope stack (#516–#520). ## What it adds All three routes sit behind the existing `/api/admin/ssp-templates` group: JWT + `admin:manage`. - `POST /api/admin/ssp-templates/{id}/run` queues a discovery run of the template (`ssp_template_discovery` with the template id as args) and answers **202**. - 404 for an unknown template, **409** for a disabled one. - 503 when no job queue is wired, and 500 when the worker is disabled. - Runs are unique by args among waiting or running jobs, so a finished run never blocks the next one. - `worker.Service.EnqueueSSPTemplateDiscovery` implements the new `templates.SSPTemplateRunEnqueuer`, wired through `APIServices.SSPTemplateRunEnqueuer` in `cmd/run.go`. - `GET /api/admin/ssp-templates/{id}/ssps` is paginated and newest first. Each item has: - `key-values` and **`key-hash`** (a contract addition from review: the forget route's id); - `ssp-id` (null once the SSP was deleted) and the SSP's current `title`; - `created-at` and `last-seen-at`; - `stale`: the SSP exists and was last seen more than `CCF_SSP_TEMPLATE_STALE_AFTER` ago (from B4's config); - `last-error`, `skipped-entries` and `skipped-sample`. - `DELETE /api/admin/ssp-templates/{id}/ssps/{keyHash}` forgets an instance whose SSP was deleted, so the next run creates it again. It answers 409 while the SSP exists and 404 for an unknown instance. Swagger is regenerated. ## How it was tested - Integration under the embedded Cedar PDP (`ssp_template_instances_integration_test.go`): - run: queued once, disabled 409, unknown 404, contributor 403, no job queue 503; - list: key values, key hash, title, stale/fresh/deleted, last error, skipped entries; contributor 403; unknown template 404; - forget: 409 while the SSP exists, 404 unknown, contributor 403, 204 for a deleted SSP, and the list shrinks afterwards. - Unit: enqueueing with the worker disabled is an error. - Gate on this branch: - `make swag`, `make lint` (only the existing SA1019), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/api/handler`, `internal/api/handler/templates`, `ssptemplates`, `internal/service/worker`; - `make check-diff`. Changed lines excluding generated swagger: 395. ## Deviations from design - This PR exists only because B4 was split. - The instance list is paginated, and adds `title`, `key-hash` (contract addition) and `skipped-sample` to the design's fields. - A run requested while the worker is disabled answers 500 instead of a 202 for a run that would never happen. ## Open questions for reviewers - Should a run requested while the worker is disabled answer 503 rather than 500? Telling that case apart from an enqueue failure needs an error exported from the worker package. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Place in the stack
SSP templates (feature 2 of
ssp-templates.md), PR B4 of 5. Base: B3 (#524,feat/ssp-templates-3-planner). Below it: B2 (#523), B1 (#522) and the SSP scope stack (#516–#520). The design's B4 is split in two to stay under 1000 lines. This PR is discovery. B5 (feat/ssp-templates-5-routes) adds the run route, the instances list (stale,key-hash,skipped-entries) and the forget route.What it adds
ssptemplates.Discoveryruns the B3 planner against the latest evidence and applies it:New group: creates the SSP through B2's
sspservicein a transaction holding the template's advisory lock, records the instance, and learns the scope (sspscope.Service.Learn, origintemplate).uuid.NewSHA1(namespace, templateID || keyHash).metadata.titleis the rendered title andmetadata.versionis1.0.ssp-template=<template id>.system-nameis the title, the description is rendered (or falls back to the title), and status isunder-development.Profiles are attached after that transaction commits. A failed attach is recorded in
last_errorandprofiles_pending, and the next run retries it.Existing group: refreshes
last_seen_atand learns new scope entries. Learn already skips exceptions and existing entries.Deleted SSP (instance
ssp_idnull): left alone, never recreated.At most
MaxNewSSPsPerRun(100) new SSPs per template per run; the next run creates the rest.Learned entries the scope cannot hold (from the planner, and
LearnResult.Skipped) go into new instance columnsskipped_entriesandskipped_sample(up to 5).A failing group gets
last_errorand the run's error; the other groups still apply.last_run_atis set when the run finishes. This is the contract addition from review: B1 exposes it aslast-run-at, null until this job sets it.ssp_template_discoveryon theschedulerqueue. Empty args run every enabled template, loading the evidence once; args withtemplate_idrun one template, for B5's run route.ProfileControlResolverthatcmd/run.goalready injects (the oscal handler's control resolution), adapted tosspservice.ControlResolver.CCF_SSP_TEMPLATE_DISCOVERY_SCHEDULE(default hourly,0 37 * * * *) andCCF_SSP_TEMPLATE_STALE_AFTER(default168h, used by B5), ininternal/configand.env.example.How it was tested
ssptemplates/discovery_integration_test.go, testcontainers):templateand the template id;last_run_atis set;last_seen_atrefreshed);make swag(no doc change),make lint(only the existing SA1019),make test,go vet -tags integration ./...;internal/service/worker,ssptemplates,internal/service,internal/service/relational,internal/config;make check-diff.Changed lines excluding generated swagger: 762.
Deviations from design
profiles_pending: retry an attach only while one failed, so a profile a user removed is never re-attached;skipped_entries/skipped_sample: the design's "skipped count and a sample".ssp-templateprop uses the existingrelational.CCFOSCALNamespace(https://compliance-framework.github.io/ns/oscal), not…/ns.plan.InvalidEntryis now an alias ofrelational.SkippedScopeEntry.Open questions for reviewers
min-streamsis not refreshed, so it can be reported stale even though some evidence remains. Shouldmin-streamsapply only to creating SSPs?ssp-templateprop namespace: the existing…/ns/oscalconstant, or the design's…/ns?🤖 Generated with Claude Code