Backtesting and validation: verifying a strategy without telling yourself stories

You've written a strategy: good. Now comes the uncomfortable question: does it really work? The only honest way to know is to measure it over many cases, with fixed rules and costs included. Doing it before risking real money is what separates the trader who builds from the one who hopes. This lesson teaches you manual backtesting (the most useful to start), the biases that distort it and how to interpret the results.

What a backtest is

A backtest is the simulation of a strategy on past data: you apply the rules to every historical situation and record what would have happened. There are two ways:

• Manual: you scroll the chart candle by candle hiding the future (with TradingView's replay function or by covering the screen) and record every trade the rules would have generated. Slow, but it teaches you a great deal.
• Automatic: you write a program that applies the rules to the data (which you'll see in the advanced course). Fast and on lots of data, but it requires programming skills.

Procedure for a correct manual backtest

1. Write the rules in full (context, setup, trigger, stop, target, risk) before looking at the data.
2. Choose the sample: one pair, a period (for example 12 months), several market regimes (trend, range, high/low volatility).
3. Hide the future: use replay or scroll manually. Don't look at what happens afterwards.
4. Record every trade the rules generate: date, entry, stop, target, outcome in R, notes. Even the uncomfortable ones.
5. Don't change the rules while testing. If you want to modify them, start over on new data.
6. Include costs: subtract the spread and commission from every trade (for example 1 pip for EUR/USD, more for the others).
7. Reach a sufficient sample: at least 100 trades; with fewer, results are unstable.

The biases that distort a backtest

• Look-ahead (peeking at the future): "recognising" the setup only because you already know the outcome. Replay exists to avoid this.
• Selection bias: counting only the good-looking setups or skipping "doubtful" ones. If the rules say enter, you enter.
• Survivorship / cherry-picking: choosing the period or pair where the system does well.
• Overfitting: tweaking parameters until the past fits. The system learns noise, not signal.
• Ignored costs: spread, commissions and slippage reduce each trade; on low timeframes they can wipe out the edge.
• Small sample: 20 trades prove nothing.
• Optimism about execution: in reality you'll enter a bit later and with slippage.

How to split the data: in-sample and out-of-sample

A simple method not to fool yourself:

• In-sample (development): a part of the data (say the first 12 months) on which you develop and define the rules.
• Out-of-sample (verification): data never looked at (the following 6 months) to which you apply the rules without modifying them.

If the results on the second sample collapse, the system was fitted to noise. If they hold (even slightly worse, which is normal), it's a good sign.

Interpreting the results

With the recorded sample calculate (previous lessons):

• Number of trades and period covered.
• Win percentage and average R of wins and losses.
• Expectancy in R (costs included).
• Profit factor.
• Maximum drawdown and maximum losing streak.

Questions to ask yourself: is expectancy positive after costs? Is the drawdown bearable for you (emotionally and in money)? Are results similar across different periods? Is the number of trades sufficient?

The next step: demo

If the backtest is promising, move to demo for at least 50 trades, recording everything in the journal. Why? Because on demo you verify things a backtest doesn't show: real execution, slippage, your emotional behaviour, following the rules in real time. If demo results are consistent with the test, you can move to live with minimal risk.

Monte Carlo (the idea)

With your trades in R you can run an experiment: shuffle their order a thousand times and see how many different drawdowns can arise from the same results. It shows you the realistic range of what you might experience, not just the lucky or unlucky historical sequence. You don't have to do it by hand: free spreadsheets and tools automate it.

In short

• A backtest simulates the rules on the past: it tells you whether there's an edge before risking money.
• Manual method: rules written first, replay without looking at the future, all trades recorded, costs included, at least 100 cases.
• Biases (look-ahead, selection, overfitting, ignored costs, small sample) distort the result.
• Split the data into in-sample and out-of-sample to check the system isn't fitted to noise.
• Before live: demo for at least 50 trades.

Practical exercise

1. Take the strategy written in the previous lesson and test it for 12 months of EUR/USD with TradingView's replay, recording every trade in R.
2. Recalculate expectancy subtracting 1 pip of cost per trade.
3. Test the same rules on another 6 months never looked at: what changes?