Backtesting a Strategy

What Is Backtesting?

A backtest simulates what would have happened if you had applied an investment strategy to historical data. It's the primary tool for evaluating systematic (rules-based) investment strategies before risking real money.

The core question a backtest answers: "Given the rules of this strategy, what would my return and risk have been over the historical period?" But there are countless ways to fool yourself with backtests, and professional quantitative researchers spend enormous effort avoiding these pitfalls.

A Simple Backtest Framework

Every backtest needs:

1. Universe: Which stocks/assets can the strategy trade?
2. Signal generation: When does the strategy buy or sell? (e.g., when 50-day MA crosses above 200-day MA)
3. Position sizing: How much capital to allocate to each position?
4. Execution assumptions: What price do you assume you get? (Next-day open is realistic; same-day close often cheats.)
5. Transaction costs: Commissions, bid-ask spread, market impact.
6. Performance metrics: Total return, Sharpe ratio, max drawdown, etc.

The Holy Trinity of Backtest Metrics

Total Return: How much did the strategy make? Important but meaningless without risk context.

Sharpe Ratio: Risk-adjusted return. Sharpe = (Strategy Return − Risk-Free Rate) / Strategy Volatility. A good strategy targets Sharpe > 1.0 before costs.

Maximum Drawdown: The largest peak-to-trough decline in portfolio value during the test period. A strategy that returns 20%/year but occasionally loses 60% is psychologically and practically very hard to implement — you'd likely panic and stop at the worst moment.

Max Drawdown = max(Peak − Trough) / Peak

In practice, professionals care deeply about the Sharpe-to-max-drawdown relationship (Calmar ratio = Annual Return / Max Drawdown). A Calmar ratio above 1 is respectable.

The Deadly Sins of Backtesting

1. Look-ahead bias: Using data in your signal that wasn't available at the time. Using today's closing price to generate today's trading signal means you'd need a time machine. Always ensure signals use data available before the trade executes.

2. Survivorship bias: Testing only on stocks that still exist today. Most databases exclude companies that went bankrupt or were delisted. This creates a massive upward bias — you're testing on the survivors, ignoring the losers.

3. Overfitting (data mining): Running thousands of parameter combinations and reporting the best-performing one. If you run enough tests, something will look amazing by pure chance. True edge should be robust across a range of parameters, not just a single "magic" setting.

4. Transaction cost underestimation: High-frequency strategies can appear profitable on paper but wipe out all gains from bid-ask spreads and market impact in practice.

5. Regime dependence: A strategy that worked perfectly in 2010–2020 may fail in 2022–2023 when interest rate conditions changed dramatically. Test across multiple market regimes.

Walk-Forward Testing

The gold standard for avoiding overfitting: divide your data into an in-sample period (for developing the strategy) and an out-of-sample period (for testing). Never touch the out-of-sample data until you're satisfied with the in-sample strategy — otherwise you're unconsciously fitting to it too.

Walk-forward optimization goes further: train on a rolling window of history, make one period of trades with those parameters, then roll forward and repeat. This mimics live trading and provides the most realistic performance estimate.

When a Backtest Is "Too Good"

Sharpe above 3 in a backtest almost always means overfitting, look-ahead bias, or transaction cost ignorance. Real live-trading strategies by elite hedge funds target Sharpe of 1–2 after costs. Be very suspicious of any backtest claiming 5+ Sharpe before real-world verification.