The Cleanest Backtest Is Often the Most Lying

Smooth equity curves are emotionally satisfying, but markets are lumpy.
Backtests often ignore or understate spreads, slippage, partial fills, and regime shifts. That cleans the curve while poisoning reality.

Another common lie is selection: using instruments that survived, or using data that has been revised, or testing only the “good” sample.
When the curve is too clean, ask: what did I exclude that will be present when money is real?

Your job is not to make a pretty curve. Your job is to survive a messy one.

Model risk: small parameter changes that flip your results are a warning, not a feature. Robustness is a survival requirement.

• Look for wide plateaus, not sharp peaks.
• Measure drawdown shape, not just final equity.
• Assume the future will be different in the exact way that hurts.

Beautiful backtest → deploy live → friction + regime changes appear → expectancy flips negative → tweak parameters → overfit → blow up by attrition.

How it kills accounts:

1. Model works on clean history.
2. Regime changes and execution friction increases.
3. Performance decays slowly, so you rationalize.
4. You optimize parameters instead of reducing risk.
5. Drawdown becomes the teacher.

Force ugliness into the backtest: worse costs, worse fills, and random delays.
Stress-test across regimes and time windows.
If small assumption changes break it, it’s fragile, not robust.

Rule that survives:

• Stress test tails and execution, not averages.
• Prefer robust plateaus over optimized peaks.
• When performance decays, reduce risk before “fixing” the model.

You backtest a scalper with near-perfect results, then go live and discover the spread widens, fills slip, and the strategy dies. The backtest wasn’t wrong. It was incomplete.

Tell: if small parameter tweaks flip your results, your model is fragile.

What to do instead

Build pessimism into your model: add higher spreads, worse slippage, and assume missed fills. If it still works, it may be real.

Combine this with Parameter sensitivity reveals fragility and overfitting.

Related

Glossary: look-ahead bias, survivorship bias.

Field checklist

• Stress-test the tails. The worst days define survival.
• Use variable spreads and slippage in testing.
• Prefer robust plateaus over optimized peaks.
• Look at drawdown shape, not only profit.
• If a tiny parameter change breaks the model, the model is fragile.