Mechanism
Real edges are usually robust across reasonable parameter ranges. Noise-only edges require precise tuning to look good.
High sensitivity often means the model is exploiting quirks of a specific sample, feed, or regime. That’s why it collapses live.
People mistake a perfect curve for an edge. A perfect curve is often proof you overfit.
Testing reality: backtests hide execution, spreads, and regime clustering. If your edge survives only in clean data, it’s a data story, not a trading edge.
- Test with variable spreads and slippage by session.
- Stress the tails: worst 1% days matter more than average days.
- Prefer rules you can execute consistently over rules that optimize the past.
How it kills accounts
Tune parameters → backtest perfection → go live → small condition change → performance collapses → more tuning → endless chase → death by optimisation.
How it kills accounts:
- Model works on clean history.
- Regime changes and execution friction increases.
- Performance decays slowly, so you rationalize.
- You optimize parameters instead of reducing risk.
- Drawdown becomes the teacher.
Rule that survives
Prefer broad plateaus of performance over sharp peaks.
Stress-test across multiple instruments and regimes.
If it only works at one exact setting, it’s not an edge.
Rule that survives:
- Stress test tails and execution, not averages.
- Prefer robust plateaus over optimized peaks.
- When performance decays, reduce risk before “fixing” the model.
Example archetype
A moving-average strategy works at 21/55 but fails at 20/56. That’s not a discovery. That’s a coincidence you fell in love with.
Tell: if small parameter tweaks flip your results, your model is fragile.
Deep dive
Robustness is a moat
Finance rewards strategies that survive small changes: different feeds, different volatility, different execution. Fragile strategies die when reality deviates from the backtest.
Related: Complexity is usually an alibi and Backtests are biographies, not prophecies.
Glossary: overfitting, payoff distribution.
Field checklist
- Stress-test the tails. The worst days define survival.
- Use variable spreads and slippage in testing.
- Prefer robust plateaus over optimized peaks.
- Look at drawdown shape, not only profit.
- If a tiny parameter change breaks the model, the model is fragile.