feat(ssp-templates): SSP template model and admin CRUD routes by ccf-lisa[bot] · Pull Request #522 · compliance-framework/api

ccf-lisa · 2026-10-19T13:50:02Z

Place in the stack

SSP templates (feature 2 of ssp-templates.md), PR B1 of 5 (B1–B5: #522, #523, #524, #525, #526), stacked on the SSP scope stack (#516–#520, top feat/ssp-scope-5-auto-filters).

What it adds

Models SSPTemplate and SSPTemplateInstance (internal/service/relational/ssp_templates.go), in both AutoMigrate lists and both drop lists. Their foreign keys (instance → template ON DELETE CASCADE, instance → SSP ON DELETE SET NULL) are added by EnsureSSPConstraints, which is EnsureSSPScopeConstraints renamed, with two keys appended. AutoMigrate here does not enforce gorm FK tags.
Package internal/service/relational/ssptemplates:
- Build: validation and normalization. Keys are trimmed, lower-cased and sorted (1–5). Scope-label groups are normalized the same way (≤10 groups of 1–5 labels). Keys and labels follow sspscope.Normalize's key rules, so anything a template accepts can be stored as a scope entry. min-streams must be ≥ 1. The title template is required, and the title, description and prop templates must parse. The labelfilter must be valid.
- Service: list/get/create/update/delete. Profile ids must exist. Keys cannot change on update (400).
Admin routes /api/admin/ssp-templates (list, create, get, update, delete), behind JWTMiddleware + pep.Authorize(authz.ResourceAdmin, authz.ActionManage), like /admin/subject-templates. JSON is kebab-case. Swagger is regenerated.
Contract addition from review: template responses also return ssp-count and last-run-at.
- ssp-count is the number of instances whose SSP still exists.
- last-run-at is a new nullable last_run_at column. It stays null until B4's discovery job sets it, and Update preserves it.

Not in this PR: preview (B3), SSP creation (B2/B4), discovery, run, instances routes (B4).

How it was tested

Unit: ssptemplates/validate_test.go. It covers normalization, defaults, and every rejection rule.
Integration (internal/api/handler/templates/ssp_template_integration_test.go) runs under the embedded Cedar PDP:
- an admin creates, lists, gets, updates and deletes templates;
- changing keys is rejected;
- a contributor gets 403 on every route;
- 400/404 cases;
- the FK behaviour and ssp-count: deleting an SSP nulls its instance's ssp_id and drops it from ssp-count, and deleting a template drops its instances and keeps its SSPs.
Gate on this branch:
- make swag, make lint (only the existing SA1019 in internal/api/metrics.go), make test, go vet -tags integration ./...;
- integration suites of internal/api/handler, internal/api/handler/templates, internal/service, internal/service/relational, internal/service/relational/sspscope;
- make check-diff.

Changed lines excluding generated swagger: 1000.

Deviations from design

The instance model has exactly the design's fields. The skipped-entries count and sample reported by the instances route are added in B4, with the code that writes them.
Duplicate scope-label groups (after normalization) are rejected with 400, not merged silently. Duplicate profile ids are de-duplicated.
An empty filter ({} / {"scope":{}}) matches everything, so it is stored as no filter.
The enabled column has no database default (the design has default:true). gorm writes a column default over a false Enabled on create, so a disabled template was stored as enabled. Build always sets Enabled, with nil meaning true.
The list route is paginated (svc.ListResponse, page/limit) like subject templates.
The response timestamps are kebab-case (created-at/updated-at), unlike subject templates' camelCase ones.

Open questions for reviewers

Templates are only checked to parse. Should save also reject templates that reference a field that is not one of the keys (keys are lower-cased, so {{ ._APP }} would render empty)? The design asks only for parse errors.

🤖 Generated with Claude Code

Add SSPTemplate and SSPTemplateInstance with their foreign keys, template validation, and /api/admin/ssp-templates list/create/get/update/delete behind admin:manage. Keys cannot change after creation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-10-19T13:50:13Z

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: dadb5061-733a-4142-aab9-e48d092bd95f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Contract addition from review: template responses carry the number of SSPs the template created that still exist, and the time of its last completed discovery run (a new nullable last_run_at column, set by discovery in a later PR). The template instance foreign keys move into EnsureSSPConstraints (renamed from EnsureSSPScopeConstraints). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The enabled column's default made gorm write true over a false Enabled on create. Drop the column default; Build always sets Enabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

gusfcarvalho

ccf-review: APPROVE

no findings.

Stack (gh stack 521): #516 → #517 → #518 → #519 → #520 → #522 → #523 → #524 → #525 → #526

…vice (#523) ## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B2 of 5**. Base: B1 (#522, `feat/ssp-templates-1-model`), on top of the SSP scope stack (#516–#520). ## What it adds A pure refactor, with no behaviour change. SSP creation and profile attachment move out of `SystemSecurityPlanHandler.Create` and `AddProfile` into a new `internal/service/relational/sspservice.Service`. B4's discovery needs the same code to create SSPs inside its own transaction. - `Create(db, oscalSSP)` unmarshals the SSP, stamps last-modified and the OSCAL version, and stores it. Callers validate the SSP first; the handler still runs `validateSSPInput`. - `AttachProfile(db, sspID, profileID)` takes the same steps as before: 1. load the SSP and the profile; 2. insert the binding with `ON CONFLICT DO NOTHING`; 3. resolve the profile's controls; 4. ensure a control implementation exists; 5. create the missing implemented requirements, matched case-insensitively and stored with canonical casing. The writes run in `db.Transaction`, which becomes a savepoint when `db` is already a transaction. It returns `ErrSSPNotFound`, `ErrProfileNotFound`, `ErrProfileAlreadyBound` or `ErrNoResolvedControls`, and the handler maps them to the same status codes and messages as before (404/404/409/400). - Profile control resolution stays in the oscal handler: the cache, the pivot table and the full-resolution fallback. It is injected as a `ControlResolver` func, because services cannot import `internal/api`. ## How it was tested - Existing SSP handler suites pass unchanged: the whole `internal/api/handler/oscal` integration package, including `system_security_plans*` and `system_security_plan_profiles_integration_test`. - New `sspservice/service_integration_test.go` covers: - attachment creates only the missing requirements, case-insensitively; - the not-found, already-bound and no-controls errors, with no binding left behind; - both calls take part in a caller's transaction, so a rollback leaves nothing. - Gate on this branch: - `make swag` (no doc change), `make lint` (only the existing SA1019 in `internal/api/metrics.go`), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/api/handler/oscal` and `sspservice`; - `make check-diff`. Changed lines excluding generated swagger: 385. ## Deviations from design None. The control resolver is injected rather than moved into the service; see the `lisa-design` note. B4 wires the same resolver into the discovery worker. ## Open questions for reviewers - Should profile control resolution (`getControlIDsForProfile`, `extractControlIDsFromProfile`, the process-wide cache) move out of the oscal handler package into a service? This PR injects it to stay a pure refactor. B4 passes the handler's resolver to the worker from where both are built. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B3 of 5**. Base: B2 (#523, `feat/ssp-templates-2-ssp-service`). Below it: B1 (#522) and the SSP scope stack (#516–#520). ## What it adds - `ssptemplates.BuildPlan(tmpl, streams, instances, entries, exceptions)` is the pure planner that preview and B4's discovery job share. It: - keeps the latest streams that match the template's filter and carry every key with exactly one non-empty value; the others count as `missing-keys`; - groups the kept streams by key values, and lists groups under `min-streams` as `below-threshold`; - reports groups whose key values fail `sspscope.Normalize`, or whose title fails to render or renders empty, under `invalid`; - plans the entries each remaining group would learn: the key entry (all keys ANDed), plus every combination of each scope-label group's values seen on at least `min-streams` of the group's streams. Multi-valued labels expand to every combination. - drops entries already in the SSP's scope, moves excepted ones to `skipped-by-exception`, and moves entries a scope cannot hold to `invalid-entries`; - keeps the evidence's value casing. `LatestEvidenceStream` gains `RawLabels`: the trimmed original values, index for index with the folded `Labels`, which are unchanged for other callers. The filter, key checks and grouping still use the folded labels, so `Payments` and `payments` are one group with one `KeyHash`. Titles, key values and learned entries come from the group's first stream by UUID; scope-label combinations count case-insensitively. - marks a group `exists` when an instance records it. An instance whose SSP was deleted (`ssp-id` null) gets nothing to learn. - `POST /api/admin/ssp-templates/preview` takes a template, saved or not, with an optional `id` so a saved template's instances mark existing SSPs. It loads the latest evidence and the instances' scope hashes, runs the planner and writes nothing. It sits behind the group's JWT + `admin:manage`. Swagger is regenerated. ## How it was tested - Unit (`plan_test.go`): - grouping by several keys; - missing, empty and multi-valued keys; - the filter; - min-streams for groups and for scope-label combinations; - scope-label groups with multi-valued labels; - existing and deleted instances, known entries and exceptions; - invalid key values and entries; - an empty rendered title; - de-duplication of a key repeated in a scope-label group; - mixed-case evidence: grouping is case-insensitive, and titles, key values and learned entries keep their casing. - Integration under the embedded Cedar PDP (`TestPreview`): - an unsaved template, and a saved one whose instance and exception change the result (mixed-case evidence; the title keeps its casing); - nothing is written; - unknown id 404, invalid template 400, contributor 403. - Gate on this branch: - `make swag`, `make lint` (only the existing SA1019), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/api/handler/templates`, `ssptemplates`, `sspscope`, `internal/service/worker` and `internal/service/relational/evidence`; - `make check-diff`. Changed lines excluding generated swagger: 662. ## Deviations from design - The function is named `BuildPlan`, because `Plan` is its result type. Its arguments are `(tmpl, streams, instances, entries, exceptions)`. - The response adds `key-hash`, `ssp-id` and `invalid-entries` to each planned SSP. - `missing-keys` also counts streams whose key is multi-valued, since they cannot be grouped. - `below-threshold` lists groups only. Scope-label combinations under `min-streams` are dropped without being listed. - A group whose title renders empty is `invalid`, so no SSP with an empty title is created. ## Open questions for reviewers - Should `below-threshold` also list scope-label combinations under `min-streams`? The design's wording says "groups or entries", but its example shows groups only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B4 of 5**. Base: B3 (#524, `feat/ssp-templates-3-planner`). Below it: B2 (#523), B1 (#522) and the SSP scope stack (#516–#520). The design's B4 is split in two to stay under 1000 lines. This PR is discovery. B5 (`feat/ssp-templates-5-routes`) adds the run route, the instances list (stale, `key-hash`, `skipped-entries`) and the forget route. ## What it adds - `ssptemplates.Discovery` runs the B3 planner against the latest evidence and applies it: - **New group:** creates the SSP through B2's `sspservice` in a transaction holding the template's advisory lock, records the instance, and learns the scope (`sspscope.Service.Learn`, origin `template`). - The SSP id is `uuid.NewSHA1(namespace, templateID || keyHash)`. - `metadata.title` is the rendered title and `metadata.version` is `1.0`. - Props are the rendered props plus `ssp-template=<template id>`. - `system-name` is the title, the description is rendered (or falls back to the title), and status is `under-development`. Profiles are attached after that transaction commits. A failed attach is recorded in `last_error` and `profiles_pending`, and the next run retries it. - **Existing group:** refreshes `last_seen_at` and learns new scope entries. Learn already skips exceptions and existing entries. - **Deleted SSP** (instance `ssp_id` null): left alone, never recreated. - At most `MaxNewSSPsPerRun` (100) new SSPs per template per run; the next run creates the rest. - Learned entries the scope cannot hold (from the planner, and `LearnResult.Skipped`) go into new instance columns `skipped_entries` and `skipped_sample` (up to 5). - A failing group gets `last_error` and the run's error; the other groups still apply. `last_run_at` is set when the run finishes. This is the contract addition from review: B1 exposes it as `last-run-at`, null until this job sets it. - River worker and periodic job `ssp_template_discovery` on the `scheduler` queue. Empty args run every enabled template, loading the evidence once; args with `template_id` run one template, for B5's run route. - The worker reuses the `ProfileControlResolver` that `cmd/run.go` already injects (the oscal handler's control resolution), adapted to `sspservice.ControlResolver`. - Config: `CCF_SSP_TEMPLATE_DISCOVERY_SCHEDULE` (default hourly, `0 37 * * * *`) and `CCF_SSP_TEMPLATE_STALE_AFTER` (default `168h`, used by B5), in `internal/config` and `.env.example`. ## How it was tested - Integration (`ssptemplates/discovery_integration_test.go`, testcontainers): - the SSP is created with the evidence's casing, the deterministic id, title, description, props, the template's profile and its implemented requirements; scope entries have origin `template` and the template id; `last_run_at` is set; - runs are idempotent (one SSP, `last_seen_at` refreshed); - later runs learn new values (evidence in other casing joins the same SSP) and respect exceptions; - a deleted SSP is never recreated; - the 100-per-run cap: 101 groups give 100, then 1; - a failed profile attach is recorded and then retried successfully; - disabled templates are skipped; - invalid learned entries are counted and sampled. - Unit: the config loader, the schedule fallback, and the resolver adapter's case-insensitive de-duplication. - Gate on this branch: - `make swag` (no doc change), `make lint` (only the existing SA1019), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/service/worker`, `ssptemplates`, `internal/service`, `internal/service/relational`, `internal/config`; - `make check-diff`. Changed lines excluding generated swagger: 762. ## Deviations from design - The design's B4 is split into B4 (discovery) and B5 (routes). - New instance columns, not in the design's model: - `profiles_pending`: retry an attach only while one failed, so a profile a user removed is never re-attached; - `skipped_entries` / `skipped_sample`: the design's "skipped count and a sample". - The `ssp-template` prop uses the existing `relational.CCFOSCALNamespace` (`https://compliance-framework.github.io/ns/oscal`), not `…/ns`. - Rendered prop values that come out empty are dropped, because OSCAL props need a value. - `plan.InvalidEntry` is now an alias of `relational.SkippedScopeEntry`. ## Open questions for reviewers - An existing SSP whose evidence drops under `min-streams` is not refreshed, so it can be reported stale even though some evidence remains. Should `min-streams` apply only to creating SSPs? - The `ssp-template` prop namespace: the existing `…/ns/oscal` constant, or the design's `…/ns`? 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

## Place in the stack SSP templates (feature 2 of `ssp-templates.md`), PR **B5 of 5**. It is the second half of the design's B4, split to keep each PR under 1000 lines. Base: B4 (#525, `feat/ssp-templates-4-discovery`). Below it: B3 (#524), B2 (#523), B1 (#522) and the SSP scope stack (#516–#520). ## What it adds All three routes sit behind the existing `/api/admin/ssp-templates` group: JWT + `admin:manage`. - `POST /api/admin/ssp-templates/{id}/run` queues a discovery run of the template (`ssp_template_discovery` with the template id as args) and answers **202**. - 404 for an unknown template, **409** for a disabled one. - 503 when no job queue is wired, and 500 when the worker is disabled. - Runs are unique by args among waiting or running jobs, so a finished run never blocks the next one. - `worker.Service.EnqueueSSPTemplateDiscovery` implements the new `templates.SSPTemplateRunEnqueuer`, wired through `APIServices.SSPTemplateRunEnqueuer` in `cmd/run.go`. - `GET /api/admin/ssp-templates/{id}/ssps` is paginated and newest first. Each item has: - `key-values` and **`key-hash`** (a contract addition from review: the forget route's id); - `ssp-id` (null once the SSP was deleted) and the SSP's current `title`; - `created-at` and `last-seen-at`; - `stale`: the SSP exists and was last seen more than `CCF_SSP_TEMPLATE_STALE_AFTER` ago (from B4's config); - `last-error`, `skipped-entries` and `skipped-sample`. - `DELETE /api/admin/ssp-templates/{id}/ssps/{keyHash}` forgets an instance whose SSP was deleted, so the next run creates it again. It answers 409 while the SSP exists and 404 for an unknown instance. Swagger is regenerated. ## How it was tested - Integration under the embedded Cedar PDP (`ssp_template_instances_integration_test.go`): - run: queued once, disabled 409, unknown 404, contributor 403, no job queue 503; - list: key values, key hash, title, stale/fresh/deleted, last error, skipped entries; contributor 403; unknown template 404; - forget: 409 while the SSP exists, 404 unknown, contributor 403, 204 for a deleted SSP, and the list shrinks afterwards. - Unit: enqueueing with the worker disabled is an error. - Gate on this branch: - `make swag`, `make lint` (only the existing SA1019), `make test`, `go vet -tags integration ./...`; - integration suites of `internal/api/handler`, `internal/api/handler/templates`, `ssptemplates`, `internal/service/worker`; - `make check-diff`. Changed lines excluding generated swagger: 395. ## Deviations from design - This PR exists only because B4 was split. - The instance list is paginated, and adds `title`, `key-hash` (contract addition) and `skipped-sample` to the design's fields. - A run requested while the worker is disabled answers 500 instead of a 202 for a run that would never happen. ## Open questions for reviewers - Should a run requested while the worker is disabled answer 503 rather than 500? Telling that case apart from an enqueue failure needs an error exported from the worker package. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: ccf-lisa[bot] <286799724+ccf-lisa[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

ccf-lisa Bot and others added 3 commits October 19, 2026 10:45

fix(ssp-templates): lock the template row while updating it

06e3782

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs(ssp-templates): regenerate swagger

7c65c77

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This was referenced Oct 19, 2026

refactor(ssp): extract SSP creation and profile attachment into a service #523

Merged

feat(ssp-templates): discovery planner and preview route #524

Merged

fix(ssp-templates): store disabled templates as disabled

3c2a199

The enabled column's default made gorm write true over a false Enabled on create. Drop the column default; Build always sets Enabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This was referenced Oct 19, 2026

feat(ssp-templates): discovery worker and periodic job #525

Merged

feat(ssp-templates): run, created SSPs and forget routes #526

Merged

ccf-lisa Bot added this pull request to stack #521 October 19, 2026 14:37

gusfcarvalho approved these changes Oct 19, 2026

View reviewed changes

gusfcarvalho merged commit b647d23 into main Oct 22, 2026
24 checks passed

gusfcarvalho deleted the feat/ssp-templates-1-model branch October 22, 2026 08:44

ccf-release-bot Bot mentioned this pull request Oct 22, 2026

chore(main): release 0.22.0 #512

Open

Navigation Menu

Repository navigation

feat(ssp-templates): SSP template model and admin CRUD routes - #522

feat(ssp-templates): SSP template model and admin CRUD routes#522
gusfcarvalho merged 5 commits into
feat/ssp-scope-5-auto-filtersfrom
feat/ssp-templates-1-model

ccf-lisa Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading

Review skipped

Uh oh!

gusfcarvalho left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ccf-lisa Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Place in the stack

What it adds

How it was tested

Deviations from design

Open questions for reviewers

Uh oh!

coderabbitai Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

gusfcarvalho left a comment

Choose a reason for hiding this comment

ccf-review: APPROVE

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ccf-lisa Bot commented Oct 19, 2026 •

edited

Loading

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading