The six metrics in the previous part described what happened. These answer a harder question: was it repeatable skill, or luck in a single sample? Each one corrects a flaw in the one before it — which is why they are read together, and never alone.
1. Sharpe ratio
Formula: (return − risk-free) ÷ annualized volatility. Measures: how many units of return you earn for every unit of volatility you carried to get them. Hides: it punishes upside volatility exactly as hard as downside — a violent rally counts against you the same as a crash — and it assumes a normal distribution of returns that does not exist in markets with fat tails, where extreme events are far more frequent than the model expects. The number: the Bitcoin trend strategy scores 1.47 against 1.05 for buy-and-hold.
2. Sortino ratio
Formula: (return − risk-free) ÷ downside volatility only. Measures: Sharpe after exempting upside volatility from the penalty — nobody ever lost sleep over a sudden rally. Hides: it always looks prettier than Sharpe because its denominator is smaller, so it should only ever be compared with another Sortino, never quietly presented in place of Sharpe when Sharpe turns out disappointing. The number: a QQQ/GLD/XLE portfolio scores Sortino 2.48 against Sharpe 1.72.
3. Calmar ratio
Formula: compound annual growth ÷ maximum drawdown. Measures: how much annual return you earn for every point of the deepest pain you endured along the way. Hides: its denominator is a single orphan event — the worst drawdown on record — so it is extremely sensitive to the length of the window and the moment it begins; changing one year can double it or halve it. The number: the same portfolio, 25.2% ÷ 13.5% = 1.86.
4. Profit factor
Formula: gross profit ÷ gross loss, in absolute terms. Measures: how many dollars you made for every dollar you lost, across all trades combined. Hides: it says nothing about the distribution of trades; one exceptional trade can lift it single-handedly, so a system looks robust while resting on a single passing incident. The test: drop the largest winner and recompute — if the number collapses, the edge was one lucky trade. The number: 1.95 — $1.95 earned for every $1 lost.
5. Expectancy
Formula: win rate × average win − loss rate × average loss. Measures: what you expect to make on an average trade — the number that actually converts into money. Hides: win rate alone is thoroughly misleading. 44% winners is more than enough as long as the average win is close to three times the average loss; what matters is the size of the win, not how often it arrives. The number: (0.44 × 18.33%) − (0.56 × 6.71%) = +4.3% per trade.
6. Statistical significance (t-stat)
Formula: Sharpe × √(number of years). Measures: the strength of the evidence that the result is not pure chance — not the size of the profit, but the confidence in it. Hides: the old threshold of 2.0 no longer holds now that thousands of strategies have been tested on the same data; the accepted bar today is t > 3.0, after correcting for the effect of multiple testing. The number: 1.47 × √10 = 4.65 — comfortably above the strict bar.
The six at a glance
| Metric | What it measures | What it hides | The number |
|---|---|---|---|
| Sharpe | Return per unit of volatility | Punishes upside like downside; assumes no fat tails | 1.47 vs 1.05 |
| Sortino | Sharpe, downside penalty only | Always prettier than Sharpe | 2.48 vs Sharpe 1.72 |
| Calmar | Return per point of worst pain | A single orphan event; window-sensitive | 25.2% ÷ 13.5% = 1.86 |
| Profit factor | Dollars made per dollar lost | Nothing about distribution; one trade can carry it | 1.95 |
| Expectancy | Expected profit per trade | Win rate alone lies | +4.3% per trade |
| t-stat | Confidence it isn't chance | The 2.0 bar is dead — it's 3.0 now | 4.65 (bar: t > 3.0) |
Why "significant" got harder
The last metric hides the sharpest twist. A t-stat of 2.0 used to be the line between signal and noise — but that line assumed you tested one strategy. Today thousands of variations are run against the same history, and with enough attempts some will clear 2.0 on luck alone, the same way that if enough people buy lottery tickets, someone wins. Correct for all those hidden attempts and the honest bar rises to t > 3.0. A result that would have been "significant" a decade ago is now just one of the many things that were quietly tried.
Never read alone
Every metric here hides what another reveals — Sharpe misses fat tails, Sortino flatters, Calmar swings on one event, profit factor can ride a single trade, expectancy needs the win and loss sizes, and significance depends on how many strategies you tried first. That is why none of them is ever read alone. Any strategy presented to you as a single shiny number — a Sharpe of 3, or "1,000% return" — is a strategy whose full record is being kept from you.
The next part turns from the metrics to the test itself: the specific, repeatable ways a backtest lies before any of these numbers are even computed.