Backtests Are Biographies, Not Prophecies

Backtests are vulnerable to hidden assumptions: stable spreads, stable liquidity, consistent regime behavior, and clean data.

Markets evolve. Participants adapt. Costs change. Correlations shift. A strategy can be “true” and still stop being profitable because the environment that rewarded it disappears.

The most dangerous mistake is emotional: treating backtest success as permission to size up.

Testing reality: backtests hide execution, spreads, and regime clustering. If your edge survives only in clean data, it’s a data story, not a trading edge.

• Test with variable spreads and slippage by session.
• Stress the tails: worst 1% days matter more than average days.
• Prefer rules you can execute consistently over rules that optimize the past.

Backtest looks great → confidence → size increases → regime shifts / friction rises → drawdown → panic tweaks → overfit → abandon.

How it kills accounts:

1. Model works on clean history.
2. Regime changes and execution friction increases.
3. Performance decays slowly, so you rationalize.
4. You optimize parameters instead of reducing risk.
5. Drawdown becomes the teacher.

Demand robustness: out-of-sample, walk-forward, and cost stress tests.
Make a ‘regime definition’ for the strategy.
Scale slowly and re-measure at each size step.

Rule that survives:

• Stress test tails and execution, not averages.
• Prefer robust plateaus over optimized peaks.
• When performance decays, reduce risk before “fixing” the model.

A trend system looks brilliant in one decade of data. Live, volatility and correlations change, and the system enters late and exits worse. The edge wasn’t eternal; it was historical.

Tell: if small parameter tweaks flip your results, your model is fragile.

What a backtest is actually good for

It’s a hypothesis generator and a sanity check. It is not a guarantee. Treat it like a lab result: useful, but conditional.

Related: Data is not neutral and Regimes kill models.

Glossary: overfitting, look-ahead bias, regime.

Field checklist

• Stress-test the tails. The worst days define survival.
• Use variable spreads and slippage in testing.
• Prefer robust plateaus over optimized peaks.
• Look at drawdown shape, not only profit.
• If a tiny parameter change breaks the model, the model is fragile.