This repository provides causal online action-detection evaluation code, frozen derived summaries, explicit protocols, and audit records for studying the reliability effects of temporal postprocessing. The primary evidence package is versioned separately from earlier releases and exploratory analyses.
- Causal data adapters and tested predictor implementations for linear and recurrent models.
- Accuracy, calibration, transition-delay, and missed-transition metrics.
- Frozen point estimates, clustered-bootstrap deltas, provenance records, and reproduction logs.
- Protocol tests for active-set top-1 scoring, clean-transition selection, finite transition windows, and missed-transition handling.
- A byte-preserved source table and audit record for the transition-recovery figure.
- Earlier published reproducibility packages, extended analyses, and an archived legacy baseline.
The implementation-scope audit records additional tested modules that exist in the codebase but are not dependencies of the primary evidence chain: audits/implementation_scope.json.
Python 3.9 or later is required.
git clone <repository-url>
cd oad-stress-test
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"PyTorch is optional for the core evaluation and synthetic check. It is required only for model training and tests that exercise torch-backed predictors.
This deterministic synthetic comparison completes in a few seconds, needs no research dataset, and writes no result files:
python reproducibility/paper_v4/quick_check.pyIt prints native and EMA values for accuracy, expected calibration error, mean transition delay, and missed-transition rate. The expected output is:
condition accuracy ece mean_transition_delay missed_transition_rate
native 0.5087281796 0.1554578017 0.0000000000 0.0000000000
ema_alpha_0.50 0.5137157107 0.0849600804 0.8000000000 0.0500000000
The sequence contains 20 ground-truth transitions. Its native-to-EMA changes are approximately +0.0050 accuracy, -0.0705 ECE, +0.8 timestep delay, and +5 percentage points missed-transition rate, so the toy example illustrates the scale of the frozen result without amplifying it through a two-transition denominator.
The single-version evidence package is in reproducibility/paper_v4/. Its checks require only the repository's development dependencies:
python -m pytest -q reproducibility/paper_v4/testsThe package includes derived summaries, reproduction logs, split and metric protocols, source data for the transition-recovery figure, discrepancy audits, tests, and a complete checksum manifest. Large checkpoints, frozen probability arrays, and bootstrap draw arrays are intentionally excluded; their source paths, sizes, and SHA-256 digests are recorded in provenance instead.
Earlier versioned packages remain unchanged in reproducibility/paper_v1/, reproducibility/paper_v2/, and reproducibility/paper_v3/. They are retained for historical traceability and are not rewritten by the current package.
Temporal flip counts, prediction-switch counts, coefficient 0.25, multi-bin calibration scans, alternate transition horizons, leave-one-video-out checks, and all-transition analyses are indexed in reproducibility/extended/. They remain available but are outside the primary evidence chain.
Known numerical discrepancies are preserved as audit assets rather than changed to match display text:
reproducibility/paper_v4/audits/cmert_boxcar_accuracy_mismatch.jsonrecords the frozen accuracy delta, its correct four-decimal rendering, the different displayed value, and an unresolved suspected origin that is explicitly not treated as an explanation.audits/manuscript_literal_trace.jsonrecords the frozen sources for three literals used by the manuscript-generation path.reproducibility/paper_v4/derived_summaries/ek100_causal_gru_ema/provenance.jsonlocalizes the recovered run-log checksum mismatch and preserves both digests.
No value is edited to remove a discrepancy. See reproducibility/paper_v4/audits/ for the five-instance confidence-interval audit.
audits/ repository-level scope and source-trace audits
docs/ repository design and implementation records
legacy/ preserved baseline from a separate research line
reproducibility/ versioned evidence packages and extended-analysis index
scripts/ active data preparation and replay entry points
src/oad_stress_test/ installable Python package
tests/ active unit and pipeline tests
tools/ active validation utilities
The archived budget, policy, and wait/abstain baseline is in legacy/budget_policy_wait_abstain/. Its files and extracted factory behavior are preserved for traceability, but it does not participate in primary conclusions. The active evaluator now depends on a structural streaming interface rather than importing the archived policy implementation.
Research datasets, third-party features, checkpoints, frozen prediction caches, and bootstrap draw arrays are not redistributed. Obtain datasets from their original providers and follow DATASET.md for local layout guidance. The repository publishes derived summaries and cryptographic provenance for excluded large artifacts.
The ignore rules exclude common data and model formats from accidental commits. The versioned CSV and JSON evidence records are intentionally trackable.
Run the active suite with:
python -m pytest -qThe continuous-integration workflow executes this pytest command on supported Python versions. Archived legacy tests are preserved under legacy/ and excluded from default discovery.
Use the unchanged author metadata in CITATION.cff. Original source code is released under the MIT License. Third-party datasets, annotations, features, and model artifacts are not covered by this license.