One of the most common backtesting mistakes is drawing conclusions from far too few trades. A 70% win rate over 10 trades and a 70% win rate over 500 trades are not the same claim — the first could easily be noise, the second is meaningful evidence.
Why Small Samples Lie
Random variation dominates small samples. Flip a fair coin 10 times and getting 7 heads is unremarkable — that's within normal statistical noise. Flip it 500 times and get 350 heads (70%), and something is clearly off with the coin. Trading results behave the same way.
Standard Error of Win Rate
SE = √( p × (1 − p) / n )Where p is the observed win rate and n is the number of trades. This formula tells you how much your observed win rate could realistically swing due to chance alone.
| Trades (n) | Observed win rate | Approx. standard error | 95% range of true rate |
|---|---|---|---|
| 20 | 50% | ±11% | 28% – 72% |
| 100 | 50% | ±5% | 40% – 60% |
| 500 | 50% | ±2.2% | 45.6% – 54.4% |
With only 20 trades, a strategy's true win rate could realistically be anywhere from 28% to 72% even if the observed rate is 50%. Never size real capital based on a sample this small.
How Many Trades Do You Actually Need?
- ▸Minimum credible sample: 100 trades to see a rough shape of the edge.
- ▸Reasonably confident: 200-300 trades across varied market conditions.
- ▸High confidence for position sizing decisions: 500+ trades.
- ▸More trades matter more than more indicators — sample size beats sophistication.
Sample Size Also Applies to Time Periods
Even 500 trades taken entirely from a single strong uptrend in 2021 don't tell you how the strategy performs in a range-bound or high-volatility market. Diversify your sample across bull, bear, and sideways regimes, and across different years, not just trade count.
Practical Rule of Thumb
Treat any backtest under 100 trades as a pilot study, not a verdict. Treat 100-300 trades as a working hypothesis to forward test. Only treat 300+ trades across multiple regimes as strong enough evidence to size up meaningfully.