Skip to content

fix(coverage): exclude convergence tests from map build; harden refresh bot commit#1464

Open
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:fix-coverage-refresh-convergence-and-hook
Open

fix(coverage): exclude convergence tests from map build; harden refresh bot commit#1464
sbryngelson wants to merge 1 commit into
MFlowCode:masterfrom
sbryngelson:fix-coverage-refresh-convergence-and-hook

Conversation

@sbryngelson
Copy link
Copy Markdown
Member

What

A manual run of the weekly coverage-refresh workflow (workflow_dispatch) surfaced two automation bugs. The SLURM gcov collector itself ran correctly in CI (618 tests → 574 mapped), but:

1. The map builder ran convergence tests and they all failed (32/618).
--build-coverage-map passed every case to the direct-invocation collector, including kind=convergence cases. Convergence tests are order-of-accuracy checks driven by convergence.py, which fills in grid resolution and patch geometry per refinement level at runtime — their base params are skeletons (m=n=p=0, no patch_icpp%geometry) that cannot run standalone. So all 32 errored in pre_process (geometry must be set / nGlobal < min_cells).

  • Map impact: none harmful. Failed tests are omitted → absent from the map → select_tests conservatively always-runs them (rung 5). That is the correct behavior for convergence checks anyway. This change just makes it explicit and removes 32 spurious failure warnings.

2. The commit-back ran the pre-commit hook and aborted before pushing.
git commit triggered the repo hook (./mfc.sh precheck/spelling) against the clean: false working tree (polluted with leftover SLURM coverage artifacts), which failed → commit aborted → the git push never ran. The bot commit stages only the binary map, so it should bypass source-code prechecks: git commit --no-verify.

Changes

  • toolchain/mfc/test/test.py: exclude kind=='convergence' cases from the coverage-map build; log how many are excluded (always-run by design).
  • .github/workflows/coverage-refresh.yml: git commit --no-verify for the map-refresh bot commit.

Verification

  • {'golden': 586, 'convergence': 32} — filter excludes exactly the 32 tests that failed in CI.
  • ./mfc.sh precheck + ./mfc.sh lint (273 unit tests) pass locally; spell check passes locally, confirming the CI hook failure was a runner-tree artifact, not a master regression.

Still unverified

The CACHE_PUSH_TOKEN branch-protection bypass never executed (the commit aborted first). The next refresh run after this merges will exercise the push for the first time.

…bot commit

The coverage-map builder ran every test directly, including convergence tests (kind=convergence). Those are order-of-accuracy checks driven by convergence.py, which fills grid resolution and patch geometry per refinement level at runtime; their base params are skeletons (m=n=p=0, no geometry) that cannot run standalone, so all 32 failed pre_process during collection. Exclude them from the map: absent entries are conservatively always-run by select_tests (rung 5), the desired behavior for convergence checks anyway.

Separately, the refresh workflow's commit-back ran the repo pre-commit hook (./mfc.sh precheck/spelling) against the clean:false working tree (polluted with leftover SLURM coverage artifacts), aborting the commit before the push ever ran. The bot commit stages only the binary map; use git commit --no-verify.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two issues in the weekly coverage-refresh automation: (1) convergence tests (which require the convergence.py driver) were incorrectly being passed to the direct-invocation coverage map builder and failing, and (2) the bot’s commit step could be aborted by repo pre-commit hooks when the runner working tree contains non-source artifacts.

Changes:

  • Exclude kind == "convergence" cases from the coverage-map build path and emit a warning indicating how many were excluded.
  • Bypass repo commit hooks for the bot commit by using git commit --no-verify in the coverage-refresh workflow.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
toolchain/mfc/test/test.py Filters out convergence cases during --build-coverage-map so only directly runnable cases are mapped.
.github/workflows/coverage-refresh.yml Uses git commit --no-verify to prevent runner-tree artifacts from aborting the automation commit.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.31%. Comparing base (574e53d) to head (b84862a).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1464   +/-   ##
=======================================
  Coverage   61.31%   61.31%           
=======================================
  Files          72       72           
  Lines       19771    19771           
  Branches     2852     2852           
=======================================
  Hits        12123    12123           
  Misses       5699     5699           
  Partials     1949     1949           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants