Repository navigation

Make our analyses more statistically rigorous - #345

Merged

fitzgen merged 10 commits into

bytecodealliance:mainfrom

fitzgen:more-rigorous-stats

Oct 14, 2026

Merged

Make our analyses more statistically rigorous#345
fitzgen merged 10 commits into
bytecodealliance:mainfrom
fitzgen:more-rigorous-stats

Commits on Oct 14, 2026
- Avoid geomean rescaling and injecting geomean rows
  fitzgen
  committed
  ba127b4
- Fix a few statistical inconsistencies
  fitzgen
  committed
  103266f
- Fix percent-delta formulas off by a factor of two
  fitzgen
  committed
  0912a67
- Add a Mann-Whitney U-test
  fitzgen
  committed
  15a821c
- Compare benchmarks with a Mann-Whitney U-test instead of Welch's t-test
  fitzgen
  committed
  f2db73d
- Use Mann-Whitney for the HTML report's significance tests
  fitzgen
  committed
  628a488
- Display the median in the HTML reports, which is what is being tested for significance
  fitzgen
  committed
  b5dc351
- Raise callgrind's default process count
  fitzgen
  committed
  1ca451e
- Document the U-test in effect-size calculation docs
  fitzgen
  committed
  4b996b5
- Report each benchmark's noise in the output tables
  fitzgen
  committed
  1b3594f