Skip to content

feat(backtest): add Value at Risk, CVaR, Omega Ratio, and Tail Risk analytics - #324

Merged
brokermr810 merged 3 commits into
OpenByteInc:mainfrom
huuan26:feat/tail-risk-var-cvar-omega
Oct 27, 2026
Merged

brokermr810 merged 3 commits into
OpenByteInc:mainfrom
huuan26:feat/tail-risk-var-cvar-omega

Conversation

@huuan26

@huuan26 huuan26 commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Expands Strategy API V2 backtesting analytics with tail risk and non-normal return distribution metrics in app/services/backtest/metrics.py and app/services/strategy_v2/runtime.py:

  1. Value at Risk (VaR) (calculate_tail_risk):
    • Computes historical non-parametric loss quantile at 95% and 99% confidence levels:
      $$\text{VaR}_\alpha = - \text{Quantile}(R, 1 - \alpha)$$
    • Exposed as valueAtRisk95 and valueAtRisk99.
    • Also provides parametric Gaussian estimates via parametricVaR95 and parametricVaR99 ($z_\alpha \cdot \sigma - \mu$, using exact normal inverse CDF without external dependencies).
  2. Conditional Value at Risk (CVaR / Expected Shortfall):
    • Evaluates conditional expected loss beyond the VaR threshold:
      $$\text{CVaR}\alpha = - \mathbb{E}[R \mid R \le - \text{VaR}\alpha]$$
    • Exposed as cvar95 and cvar99.
    • Coherent risk measure capturing extreme tail loss / catastrophe severity beyond VaR.
  3. Omega Ratio (calculate_omega_ratio):
    • Evaluates the probability-weighted ratio of gains relative to losses above a target return threshold $L$ (default 0.0):
      $$\Omega(L) = \frac{\sum_{t=1}^T \max(R_t - L, 0)}{\sum_{t=1}^T \max(L - R_t, 0)}$$
    • Unlike Sharpe Ratio, Omega Ratio accounts for all higher statistical moments without presuming a Gaussian return distribution.
    • Safely returns None when downside losses are zero with positive returns (infinite payoff) and 0.0 when returns are flat or zero.
  4. Tail Ratio:
    • Quantifies asymmetry between right-tail gains and left-tail losses:
      $$\text{Tail Ratio} = \frac{|\text{Quantile}(R, 0.95)|}{|\text{Quantile}(R, 0.05)|}$$
    • Values $> 1.0$ indicate favorable upside skewness over extreme downside tail risk. Safely handles zero denominator.
  5. Distribution Moments (Skewness & Kurtosis):
    • Computes sample Fisher-Pearson skewness (skewness, $g_1$) and sample Fisher excess kurtosis (kurtosis, $g_2$, zero for Gaussian distribution) to quantify non-normality and fat-tail thickness.
  6. Strategy API V2 Integration:
    • Integrated into runner._result() in app/services/strategy_v2/runtime.py, exposing valueAtRisk95, valueAtRisk99, cvar95, cvar99, parametricVaR95, parametricVaR99, omegaRatio, tailRatio, skewness, and kurtosis in the primary backtest results dictionary.
    • 100% backward compatible with all existing backtest result fields.
  7. Bilingual Documentation:
    • Updated docs/product/BACKTEST_CENTER.md and docs/product/BACKTEST_CENTER_CN.md with detailed LaTeX mathematical formulations, interpretation guidelines, and edge-case behaviors.

Changes

  • backend_api_python/app/services/backtest/metrics.py:
    • Implement calculate_omega_ratio with target threshold support and safe zero-downside handling.
    • Implement calculate_tail_risk supporting configurable confidence levels (default 95% and 99%), historical VaR, CVaR/Expected Shortfall, parametric Gaussian VaR, Tail Ratio, sample skewness, and sample excess kurtosis.
    • Export calculate_omega_ratio and calculate_tail_risk in __all__.
  • backend_api_python/app/services/strategy_v2/runtime.py:
    • Import calculate_omega_ratio and calculate_tail_risk.
    • Wire calculations into _result() and populate the backtest result dictionary.
  • backend_api_python/tests/test_backtest_metrics.py:
    • Add deterministic unit tests for calculate_omega_ratio (known values, custom threshold, all-gain/all-loss edge cases, empty series, invalid input validation).
    • Add deterministic unit tests for calculate_tail_risk (known quantile values, CVaR coherence check $\text{CVaR} \ge \text{VaR}$, parametric VaR verification, tail ratio, skewness, kurtosis, single observation, flat return series, custom confidence levels, Series vs List compatibility).
  • backend_api_python/tests/test_strategy_v2_runtime.py:
    • Add integration assertions verifying that _result() includes and correctly calculates all tail risk and Omega metrics.
  • docs/product/BACKTEST_CENTER.md & docs/product/BACKTEST_CENTER_CN.md:
    • Document the mathematical definitions, practical interpretation, and API fields in English and Chinese.

Test plan

  • All 96 tests in test_backtest_metrics.py and test_strategy_v2_runtime.py pass (pytest tests/test_backtest_metrics.py tests/test_strategy_v2_runtime.py)
  • ruff check app/ tests/ passes with zero errors/warnings
  • python scripts/backend_quality_check.py passes
  • git diff --check passes with zero formatting/whitespace errors
  • 100% backward compatibility preserved for existing backtest outputs

API documentation (if routes/schemas changed)

  • No routes or OpenAPI schemas changed; expands result dictionary payload in Strategy API V2 backtests

…nalytics

Introduce tail risk and non-normal distribution analytics into Strategy API V2 backtests:
- calculate_tail_risk: historical VaR (95%, 99%), historical CVaR (Expected Shortfall), parametric Gaussian VaR, Tail Ratio, sample skewness, and sample excess kurtosis.
- calculate_omega_ratio: probability-weighted ratio of gains to losses relative to a benchmark return threshold.
- Strategy API V2 integration: populate valueAtRisk95, valueAtRisk99, cvar95, cvar99, parametricVaR95, parametricVaR99, omegaRatio, tailRatio, skewness, and kurtosis in _result().
- Documentation: update English and Chinese backtest center guides with mathematical formulations and interpretation.
- Unit and integration tests: deterministic test coverage with edge-case and input validation.

Signed-off-by: huuan26 <phanbuihuuan261103@gmail.com>
@huuan26
huuan26 requested a review from brokermr810 as a code owner October 27, 2026 07:08
@sonarqubecloud

Copy link
Copy Markdown

@codecov

codecov Bot commented Oct 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.22222% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ackend_api_python/app/services/backtest/metrics.py 97.14% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@brokermr810 brokermr810 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed after synchronizing the Alpaca test fix. Tail-risk estimates now include sample sufficiency thresholds, explicit status/observation/unit fields, non-negative loss semantics, expanded tests, and updated documentation.

@brokermr810
brokermr810 merged commit bb8aa03 into OpenByteInc:main Oct 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants