Repository navigation
fix(ci): fail tests-ok when workspace discovery fails - #19
Draft
MakerFlyFly wants to merge 4 commits into
Draft
MakerFlyFly wants to merge 4 commits into
MakerFlyFly wants to merge 4 commits into
Conversation
Require successful member discovery before accepting a successful or skipped pytest matrix. Exercise the workflow's actual Bash assertion across all job results and preserve empty-selection behavior. Generated-by: Codex (GPT-6)
Require the discovery output to be [] before accepting a skipped matrix. Probe Bash before executing gate tests, check failure diagnostics, and accept equivalent literal dependency declarations without new dependencies. Align the workflow and branch-protection comments with the selection rules. Generated-by: Codex (GPT-6)
A Bash launcher can start without forwarding the environment needed by the tests-ok gate. Probe the real BATCHES input path and reject a shell that drops it, while keeping unavailable-launcher skips explicit. Replace implementation-detail assertions with a real-shell environment loss regression and remove synthetic bindings already checked against the workflow. Gate logic and workflow configuration are unchanged. Generated-by: Codex (GPT-6)
Keep return-code inspection in the try-else branch, where the probe is known to have completed. This makes assignment explicit for CodeQL without relying on its model of pytest.skip raising an exception. Launcher failures still skip the gate tests with the same diagnostics. The workflow gate, modelled outcomes and environment contract are unchanged. Generated-by: Codex (GPT-6)
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests-okmay pass. A skipped pytest matrix additionally requires discovery output to be exactly[], supplied through the step environment.BATCHESenvironment input so a launcher that starts but drops the gate input is skipped explicitly.Type of change
tools/dev/regression tests)Fixed revision and scope
Current head:
a301d0df4b4b2d524285a87ef4e83bc55c14f0be.Base: upstream
deb35662.The original two commits remain intact. Two test-only commits were appended after reviewed
2c0beae4: the environment-probe revision and a minimal control-flow clarification required by the first real CodeQL result. Relative to2c0beae4, one test file changes by +15/-19 lines, retaining mode 100644. Workflow logic, selector, dependencies and the required check name are unchanged by these follow-ups. The complete PR still includes the original workflow gate and.asf.yamlcomment changes.Latest review disposition
[ "${BATCHES:-}" = "[]" ]withBATCHES=[]in the child environment. The real-shell regression removes that variable and requires an explicit skip. Restoring the old start-only probe makes exactly this regression fail. No global environment or WSL configuration is changed. An actual Linux WSL distribution was not tested.3987c7ad, the analysis workflow succeeded but its independent aggregation check failed with “Potentially uninitialized local variable” atprobe.returncode. Move that check into the try-else branch, where assignment is explicit, without relying on CodeQL's model ofpytest.skip(). Existing exception/skip behavior remains covered. The aggregation check on current head now succeeds with “No new alerts” and zero annotations. No finding was dismissed and no CodeQL configuration was weakened.Original review disposition
members=success, pytest=skippedwith nonempty or missing batches. Require canonical[]for a skipped matrix. The proposed cancellation timing remains unverified; GitHub's cancellation reference does not establish that exact outcome.bashresolved to the unusable WSL launcher. The fixture now probes the real arguments, with a five-second timeout, and skips missing, failed, or timed-out launchers.always()were falsely rejected. A bounded standard-library field reader now accepts those forms; its docstring states the unsupported YAML features. Incorrect dependencies, conditions, and literal/misdirected BATCHES bindings still fail..asf.yamlnow describes successful discovery plus a passing matrix or confirmed empty selection. Only the explanatory comment changes..shand.jsonchanges and exclude.md/ plugin manifests. The three old Python-only comments now defer to eligible files, linked sources, dependents, and shared-input selection.Test plan
Current local checks
Generated-by:attribution retainedThe PowerShell skips are not executed gate cases. Whole-dev Windows support is not claimed: the earlier full runs on
3987c7adreported 39 failed / 265 passed / 1 skipped / 30 errors under Git Bash, and 39 failed / 237 passed / 29 skipped / 30 errors under native PowerShell. An isolated native replay of2c0beae4had the same 69 FAILED/ERROR node IDs. These include missing Windows symlink privileges and existing environment/tree assumptions outside this module. The gate module was rerun on the final helper revision in both Windows shells; no new whole-dev Windows replay is implied.Reverse checks against the final 47-case module
deb35662workflowd0e14d18workflowThese are local fault-injection results, not hosted cancellation experiments.
Hosted verification on the final revision
a301d0df: discovery selects 51 members in eight batches. All eightRun pyteststeps succeed; logs record all 51 selected member invocations, including 335 passing dev tests.tests-oksucceeds withpytest gate satisfied (matrix result: success).982a3a1e: the base is fixeda301d0df, the comparison changes onlytools/dev/README.mdby +2/-0, discovery prints canonical[], pytest is skipped, andtests-oksucceeds withpytest gate satisfied (matrix result: skipped).prekworkflows pass, including the whole-tree lychee check: PR #19, temporary PR #20.The matrix contains optional skips in other workspace members; success is not a claim that every workspace test executed. The documentation-only comparison and actual job results establish the empty-selection case without relying on local shell simulation.
Temporary Draft PR #20 was closed without merging after verification. Its branch and run records are retained. It was aligned to the final base by an appended merge, preserving its earlier history and keeping the comparison documentation-only.
Fork workflow execution was restored with explicit authorization. Default token permissions remain read-only, review-approval permission stays disabled, and the manually disabled npm-pin workflow stays disabled. This PR remains Draft; upstream submission and merge are separate actions.
Historical evidence
The previous 50-case module and 338-pass Linux dev suite belong to
2c0beae4. The first hosted matrix/empty runs on3987c7ad/a76589f4passed, but that head's independent CodeQL aggregation check exposed the control-flow diagnostic above. Those records are retained as historical evidence and are not retagged as final-head results.RFC-AI-0004 compliance
Linked issues
Closes #18.
Notes for reviewers
The
BATCHEScontract matches the producer's existing literal[]; no JSON parser or package dependency is added. Public APIs and the required check name are unchanged. Raw logs, mutation copies and local harnesses stay outside the worktree; no public download location is implied.Generative AI disclosure
AI-generated with Codex (GPT-6). Human review is pending. This draft stays in the contributor's fork before any separately authorized upstream submission.