[YusefSyed]

Summary

• stabilize NormalizedMultiChoiceProbability by subtracting the largest
  finite log-probability before exponentiation
• preserve the existing behavior for non-finite and empty inputs
• add a regression for long-sequence log-probabilities that previously
  underflowed to zero

Why

The metric currently computes exp(logprob) before dividing by the sum. For
finite sequence log-likelihoods such as [-1000.0, -1001.0], both exponentials
underflow to zero. The metric then returns 0.0 and emits an invalid-divide
warning, even though the gold choice has normalized probability
1 / (1 + exp(-1)), approximately 0.7310586.

Subtracting one shared finite maximum is algebraically neutral for the
probability ratios and prevents both underflow and unnecessary overflow.

Verification

Red-on-main control at 932e1f2f:

result=0.0
warning_count=1
invalid value encountered in divide

Patched branch:

result=0.7310585786300049
warning_count=0

Checks:

• pytest tests/test_unit_base_metrics.py -q -k 'mc_probability_metric' — 2 passed
• pytest tests/test_unit_base_metrics.py -q -k 'not f1_dynamic_metric and not exact_match_dynamic_metric' — 8 passed, 3 skipped
• PYTHONPATH=. pytest tests/unit/metrics/test_normalizations.py -q — 4 passed
• ruff format --check on changed files — passed
• ruff check on changed files — passed
• git diff --check — passed

The two excluded tests require the optional multilingual extra. A separate
metrics request run also reaches the repository's existing xxhash 4 string
compatibility failure tracked in #1330; neither failure touches this metric or
diff.

Closes #1375.

[YusefSyed]

Following up on this normalized multiple-choice probability fix. It shifts finite log-probabilities by their shared maximum before exponentiation, while preserving the existing non-finite and empty-input behavior. Could someone confirm whether this is the preferred way to handle the underflow case in #1375? Happy to revise it.

[YusefSyed]

Rebased on current main. The new test_mc_probability_metric_avoids_logprob_underflow and the rest of tests/test_unit_base_metrics.py pass locally (the two *_dynamic_metric tests need the multilingual extra and fail the same way on main without it). Still happy to revise if there is a preferred way to handle #1375.