Filters Shrink Sample Size and Inflate Confidence

Filters reduce the number of trades. Fewer trades means wider uncertainty around results.

In backtests, it’s easy to “discover” rules that happened to fit a small set of historical trades. This is overfitting disguised as sophistication.

The more conditions you stack, the easier it becomes to fit past noise and the harder it becomes to reproduce performance live.

Model risk: small parameter changes that flip your results are a warning, not a feature. Robustness is a survival requirement.

• Look for wide plateaus, not sharp peaks.
• Measure drawdown shape, not just final equity.
• Assume the future will be different in the exact way that hurts.

Add filters → backtest improves → sample collapses → confidence rises → go live → regime changes → performance fails → endless tuning → decay.

How it kills accounts:

1. Model works on clean history.
2. Regime changes and execution friction increases.
3. Performance decays slowly, so you rationalize.
4. You optimize parameters instead of reducing risk.
5. Drawdown becomes the teacher.

Track sample size and confidence intervals, not just profit.
Prefer fewer filters with clearer roles.
Require robustness: performance should survive parameter nudges and different feeds.

Rule that survives:

• Stress test tails and execution, not averages.
• Prefer robust plateaus over optimized peaks.
• When performance decays, reduce risk before “fixing” the model.

You add a time filter, volatility filter, trend filter, and indicator filter. The backtest becomes beautiful, but only 20 trades exist across years. Live, the strategy doesn’t behave the same because it was never proven.

Tell: if small parameter tweaks flip your results, your model is fragile.

Robustness beats elegance

If small changes break it, it’s not edge, it’s fit.

Related: Data is not neutral and Complexity is usually an alibi.

Glossary: overfitting, base rate, look-ahead bias.

Field checklist

• Stress-test the tails. The worst days define survival.
• Use variable spreads and slippage in testing.
• Prefer robust plateaus over optimized peaks.
• Look at drawdown shape, not only profit.
• If a tiny parameter change breaks the model, the model is fragile.