Mechanism
Price data is not a single truth. Different brokers, venues, and aggregation rules produce different highs/lows and different bar shapes.
Strategies that depend on exact “touches” or precise bar patterns can be extremely sensitive to these differences. In a backtest, it looks clean. Live, it behaves differently.
When you move brokers or markets, the ‘edge’ vanishes because the artifact vanishes.
Model risk: small parameter changes that flip your results are a warning, not a feature. Robustness is a survival requirement.
- Look for wide plateaus, not sharp peaks.
- Measure drawdown shape, not just final equity.
- Assume the future will be different in the exact way that hurts.
How it kills accounts
Backtest on feed A → go live on feed B → signals differ → confusion → override rules → friction rises → performance collapses → abandon.
How it kills accounts:
- Model works on clean history.
- Regime changes and execution friction increases.
- Performance decays slowly, so you rationalize.
- You optimize parameters instead of reducing risk.
- Drawdown becomes the teacher.
Rule that survives
Test across multiple feeds and session settings.
Prefer signals based on structure and ranges, not single-tick perfection.
If a strategy needs one exact data construction, treat it as fragile.
Rule that survives:
- Stress test tails and execution, not averages.
- Prefer robust plateaus over optimized peaks.
- When performance decays, reduce risk before “fixing” the model.
Example archetype
Your breakout triggers on one broker’s candle but not another. You think the broker is ‘manipulating’. Reality: your strategy was tuned to a data artifact.
Tell: if small parameter tweaks flip your results, your model is fragile.
Deep dive
Build for reality
If your edge disappears when the feed changes, you didn’t find market structure. You found a measurement quirk. That’s why robustness testing matters.
Related: Data is not neutral and Charts lie about fills.
Glossary: overfitting, spread, slippage.
Field checklist
- Stress-test the tails. The worst days define survival.
- Use variable spreads and slippage in testing.
- Prefer robust plateaus over optimized peaks.
- Look at drawdown shape, not only profit.
- If a tiny parameter change breaks the model, the model is fragile.