ci(coverage): measure test coverage in CI, report-only (#1278 item 4)#1322
Merged
Conversation
Wrap the CI `test` job's suite in `coverage run` and publish a coverage table to the run Summary. Report-only — no --fail-under gate; the number is a signal for targeting backfill (#1278 item 5), not a pass/fail bar. - .coveragerc: measure the `website` app only (vendored forks + 3rd-party out of scope), branch coverage on, omit tests/ and the disabled per-env migrations/. - requirements-dev.txt: add coverage==7.14.1 (test-only; not in the prod image). CI's `test` job installs it directly so it doesn't drag in Playwright; the pin is kept in sync with this file. - workflow: `coverage run manage.py test ...` (propagates the suite's exit code, so failures still go red) followed by `coverage report` to the log and a markdown table to $GITHUB_STEP_SUMMARY. - .gitignore: ignore .coverage / htmlcov/ / coverage.xml (written into the bind-mounted repo root). - CLAUDE.md: document the coverage step + how to run it locally. Baseline locally: 59% of the website app (branch coverage), 201 tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…m 4) - CONTRIBUTING.md: refresh the now-stale CI paragraph (coverage was listed as future roadmap; it's now implemented) and add a "Test coverage" subsection — report-only, how to run it locally, .coveragerc scope. Add the Continuous integration + Test coverage entries to the TOC. - README.md: list .coveragerc in the Documentation/config table. DEPLOYMENT.md intentionally left unchanged: its testing mention is a pre-deploy pass/fail checklist, and coverage is report-only (not a deploy gate), so adding it there would misrepresent it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements item 4 of the testing roadmap (#1278): measure test coverage in CI. Report-only — no gate, per the issue ("Report in CI, don't gate on a number yet — use it to target backfill").
What changed
.coveragerc(new) — measures thewebsiteapp only (vendored forks + third-party out of scope), branch coverage on, omitstests/and the disabled per-envmigrations/.requirements-dev.txt— addcoverage==7.14.1(test-only; not in the prod image). CI'stestjob installs it directly so it doesn't drag in Playwright; the pin is kept in sync with this file..github/workflows/test.yml— thetestjob now runscoverage run manage.py test …(propagates the suite's exit code, so a test failure still goes red ✗) thencoverage reportto the log and a markdown table to the run Summary..gitignore— ignore.coverage/htmlcov//coverage.xml(written into the bind-mounted repo root).CLAUDE.md/CONTRIBUTING.md/README.md— document the coverage step and how to run it locally; refresh CONTRIBUTING's CI paragraph (coverage was listed as future roadmap) and add a "Test coverage" subsection.DEPLOYMENT.mdintentionally untouched: its testing mention is a pre-deploy pass/fail checklist, and coverage is report-only (not a deploy gate).Why report-only
A coverage number is a tool for deciding what to backfill next (#1278 item 5), not a quality bar to enforce on day one. Gating now would just block PRs on unrelated untested code. The number rides along on every run so we can watch it climb as backfill lands.
Verification (locally, in the container)
master; with test(fixtures): factory_boy factories + delegate make_* helpers (#1272) #1319's factories it's 211 tests.)coverage runpropagates a non-zero exit (wrapped asys.exit(7)→ step would fail) — so test failures still surface as red, the report step is skipped on failure.coverage report --format=markdownrenders;.coverageartifact is gitignored;website/tests/*correctly omitted from the report.Note for local dev
coverageis inrequirements-dev.txt, so install it locally withpip install -r requirements-dev.txt. CI installs it directly in thetestjob.🤖 Generated with Claude Code