Repository navigation
test: prune redundant tests and make slow gates opt-in - #2
Merged
Merged
Conversation
…allel gates Drops shape/range/determinism duplicates in test_forest, test_predict_chf, test_nsplit, test_minimal_depth, test_oob_predict_score, test_smoke and test_forest_device_dispatch, and the whole of test_forest_flat_path. Each is subsumed by test_properties, test_parallel_equivalence, or a stronger test in the same area. test_public_api now pins the exact __all__ set.
…covers Removes duplicate shape, range and determinism checks in test_tree, test_hist_tree, test_hist_splits, test_splits, test_estimators, test_flat_tree_builder and test_time_grid_coarsening, plus the vacuous nsplit-positive tree test that never compared its two trees.
nsplit defaults to 0 and rng to None in build_tree, build_tree_hist, find_best_split and find_best_split_hist, so each of these made the same call twice. The logrank nsplit test keeps its determinism half.
Schema-only and duplicate checks in test_importance, test_importance_oob, test_shap, test_evaluation, test_gpu_detect and test_penalized_fine_gray, plus two vacuous ones: score_cr cause=2 passed even if cause was ignored, and the SHAP/VIMP top-3 overlap passes by chance 70% of the time. test_cause_specific_cox's competing-censoring check is implied by R parity.
fastcmprsk equals crrp to <=4e-14 on the LASSO fixtures, so its LASSO items duplicated the crrp gate; its SCAD/MCP items and crrp's were strict-xfails that only confirmed a known mismatch. test_objective_dominates_crrp already fails on a regression to the v=1 prox. Removes the six unused fastcmprsk fixtures and tests/cross_check_fastcmprsk.R.
…hecks toy_input changes already fail the rfSRC split fixture test, the split and synthetic generators are pinned by their parquet round-trip tests, and the _rpy2_available tests could not fail. test_equivalence_gate folds the two-row grid case and the n_seeds check into existing tests, and now asserts the within_rf and max-over-seeds keys by value.
Removes the calibrate/run/report subcommands with runner.py, report.py, calibrate.py and the baselines.py loader, plus their tests. The CLI keeps bench-vimp. config.py, datasets.py, splits.py and the baselines/ data stay; validation/README.md no longer lists the removed modules.
Both rebuilt the same tree with seed=42 and compared runs; the merged test checks features and leaf_table across five runs.
test_batched_matches_legacy_exhaustive 20 -> 5 seeds, observed_bins and the sparse-leaf round-trips 10 -> 3 seeds. The sparse pickle-size guard keeps only the rfsrc half; test_serialization's compact-vs-dense check is far tighter for default mode than the 20 MB bound was.
On CPU hosts both returned early through the RuntimeError branch, so they asserted nothing about n_jobs warnings. They now carry the gpu marker, and a CPU-host test pins that device='cuda' without CUDA raises.
No caller in src/, validation/, docs/ or __all__; only its own three tests used it.
The expected bounds were computed with _wilson_ci, the helper under test. They are now literals from R's prop.test(correct = FALSE), which a stdlib-only evaluation of the Wilson formula matches to 1e-16.
The whole-forest pickle SHA was re-pinned 12 times, 11 of them for schema or storage changes with tree building unchanged. The digest now covers walk_tree() output only, and was pinned from a fit that still matched the old pickle anchor. The separate byte-count assertion is gone.
test_equivalence_pbc_stochastic is the heaviest default test (80 forest fits, half in reference mode) and only catches catastrophic drift. It is now marked slow. The module docstring stated 0.04/0.12 bounds; the code enforces median < 0.20 and max < 0.30, and the docstring now says so.
test_default_mode_faster_than_reference_pbc was perf-marked but still ran on every push, including shared Windows runners. addopts now deselects perf too; run it with -m perf. The slow marker description no longer refers to the retired validation harness.
…n CI test_shap_alignment.py is the only external-reference check for SHAP (bit-exact against shap.TreeExplainer) and was skipped everywhere because shap was in no extra. The lock resolves shap 0.52.0 (0.49.1 on Python 3.10) without moving any existing pin.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On the Ubuntu 3.12 CI job,
test_equivalence_pbc_stochastictook 552 of 766 s of the pytest run (671 of 927 s on Windows). It is now opt-in with-m slow.test_shap_alignment.pywas skipped in every environment becauseshapwas not installed. This PR adds it to thedevextra, so the TreeExplainer comparison runs in CI.