Skip to content

ci(coverage): measure test coverage in CI, report-only (#1278 item 4)#1322

Merged
jonfroehlich merged 2 commits into
masterfrom
1278-coverage-in-ci
Jun 17, 2026
Merged

ci(coverage): measure test coverage in CI, report-only (#1278 item 4)#1322
jonfroehlich merged 2 commits into
masterfrom
1278-coverage-in-ci

Conversation

@jonfroehlich

Copy link
Copy Markdown
Member

Implements item 4 of the testing roadmap (#1278): measure test coverage in CI. Report-only — no gate, per the issue ("Report in CI, don't gate on a number yet — use it to target backfill").

What changed

  • .coveragerc (new) — measures the website app only (vendored forks + third-party out of scope), branch coverage on, omits tests/ and the disabled per-env migrations/.
  • requirements-dev.txt — add coverage==7.14.1 (test-only; not in the prod image). CI's test job installs it directly so it doesn't drag in Playwright; the pin is kept in sync with this file.
  • .github/workflows/test.yml — the test job now runs coverage run manage.py test … (propagates the suite's exit code, so a test failure still goes red ✗) then coverage report to the log and a markdown table to the run Summary.
  • .gitignore — ignore .coverage / htmlcov/ / coverage.xml (written into the bind-mounted repo root).
  • CLAUDE.md / CONTRIBUTING.md / README.md — document the coverage step and how to run it locally; refresh CONTRIBUTING's CI paragraph (coverage was listed as future roadmap) and add a "Test coverage" subsection.

DEPLOYMENT.md intentionally untouched: its testing mention is a pre-deploy pass/fail checklist, and coverage is report-only (not a deploy gate).

Why report-only

A coverage number is a tool for deciding what to backfill next (#1278 item 5), not a quality bar to enforce on day one. Gating now would just block PRs on unrelated untested code. The number rides along on every run so we can watch it climb as backfill lands.

Verification (locally, in the container)

  • Full suite under coverage: 201 tests OK, TOTAL 59% (branch coverage). (This branch is off master; with test(fixtures): factory_boy factories + delegate make_* helpers (#1272) #1319's factories it's 211 tests.)
  • Confirmed coverage run propagates a non-zero exit (wrapped a sys.exit(7) → step would fail) — so test failures still surface as red, the report step is skipped on failure.
  • coverage report --format=markdown renders; .coverage artifact is gitignored; website/tests/* correctly omitted from the report.
  • Workflow YAML validated (parses; step order correct).

Note for local dev

coverage is in requirements-dev.txt, so install it locally with pip install -r requirements-dev.txt. CI installs it directly in the test job.

🤖 Generated with Claude Code

jonfroehlich and others added 2 commits June 17, 2026 13:53
Wrap the CI `test` job's suite in `coverage run` and publish a coverage
table to the run Summary. Report-only — no --fail-under gate; the number
is a signal for targeting backfill (#1278 item 5), not a pass/fail bar.

- .coveragerc: measure the `website` app only (vendored forks + 3rd-party
  out of scope), branch coverage on, omit tests/ and the disabled
  per-env migrations/.
- requirements-dev.txt: add coverage==7.14.1 (test-only; not in the prod
  image). CI's `test` job installs it directly so it doesn't drag in
  Playwright; the pin is kept in sync with this file.
- workflow: `coverage run manage.py test ...` (propagates the suite's exit
  code, so failures still go red) followed by `coverage report` to the log
  and a markdown table to $GITHUB_STEP_SUMMARY.
- .gitignore: ignore .coverage / htmlcov/ / coverage.xml (written into the
  bind-mounted repo root).
- CLAUDE.md: document the coverage step + how to run it locally.

Baseline locally: 59% of the website app (branch coverage), 201 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…m 4)

- CONTRIBUTING.md: refresh the now-stale CI paragraph (coverage was listed
  as future roadmap; it's now implemented) and add a "Test coverage"
  subsection — report-only, how to run it locally, .coveragerc scope. Add
  the Continuous integration + Test coverage entries to the TOC.
- README.md: list .coveragerc in the Documentation/config table.

DEPLOYMENT.md intentionally left unchanged: its testing mention is a
pre-deploy pass/fail checklist, and coverage is report-only (not a deploy
gate), so adding it there would misrepresent it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jonfroehlich jonfroehlich merged commit 2536f93 into master Jun 17, 2026
2 checks passed
@jonfroehlich jonfroehlich deleted the 1278-coverage-in-ci branch June 17, 2026 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant