FXRISK Manual

Filter Creep → Tiny Sample → False Confidence

More filters can make a backtest prettier while starving the sample size until the result is meaningless. What to watch for: the chain starts before PnL screams. It starts when decision quality drops and you keep trading anyway.

Steps

Strategy has drawdowns; discomfort rises.
Add a filter to remove losing trades.
Performance improves in backtest; sample shrinks.
Add another filter; sample shrinks further.
The remaining trades look perfect; confidence inflates.
You size up because the curve looks smooth.
Live regime differs; the rare setup misbehaves.
A small set of bad outcomes erases months of gains.

Intervention points:

  • Cut size at the first sign the chain is forming.
  • Pause when you start “fixing” the last loss with a new trade.
  • Stop trading when execution quality degrades.
Antidote

Treat sample size as a risk constraint.
If a filter reduces trades drastically, demand stronger evidence and wider robustness checks.
Prefer simple rules that survive many regimes over perfect curves in one regime.

  • Stop the sequence: one loss is information, two losses is a warning, three losses is a system failure. Have a hard cut.
  • Reduce degrees of freedom: fewer pairs, fewer timeframes, fewer discretionary choices.
  • Re-enter only after reset: calm state, checklist passed, size reduced.
Notes

Filters are seductive because they feel like intelligence. Past a point, they are just overfitting with nicer charts.

Related: parameter sensitivity, overfitting.


Field checklist

  • Stress-test the tails. The worst days define survival.
  • Use variable spreads and slippage in testing.
  • Prefer robust plateaus over optimized peaks.
  • Look at drawdown shape, not only profit.
  • If a tiny parameter change breaks the model, the model is fragile.