Part 2: In-sample and out-of-sample · Chapter 6 of 14 · 9 min
Walk-forward
Walk-forward re-optimizes every month on the previous three months and trades the next month: every trade is out of sample by construction. On the 480 rules it gives 96 trades at +0.16 ± 0.33 R (+15.6 R) from March to September 2026; on random series the mean is +0.01 (95th percentile +0.27). With windows of 1, 2, 3 and 6 months the result goes from −0.13 to +0.45 R: the window length is one more parameter.
The idea
A split in two parts gives a single test, on a single period. Walk-forward gives many: you choose the rule on the data up to a certain date, trade the next period with that rule, then move everything forward and repeat. The trades of each test period are out of sample by construction, because the rule was chosen looking only at the past. At the end you chain the test periods into a single series of results, which imitates what you would have done if you had re-optimized at every deadline.
Walk-forward answers the question that matters: does the process of choice (trying 480 rules and taking the best) work, on data it has not seen? It does not evaluate a rule: it evaluates the method by which you choose it.
Our walk-forward
Every month, the rule is chosen like this: among the 480, the best by mean R in the previous three months (at least 15 trades). Then you trade the month. From March to October 2026:
| Month | Rule chosen (breakout, duration in candles) | Mean R over the 3 months before | Month: trades | Mean R | Sum |
|---|---|---|---|---|---|
| March | N = 80, 1.5 ATR stop, 3 R target, 48 | +1.25 (22 tr.) | 11 | +0.51 | +5.6 R |
| April | N = 80, 1.5 ATR, 3 R, 48 | +0.97 (28 tr.) | 8 | −0.51 | −4.1 R |
| May | N = 80, 1 ATR, 2 R, 12 | +0.44 (33 tr.) | 10 | −0.25 | −2.5 R |
| June | N = 80, 1.5 ATR, 3 R, 24 | +0.38 (25 tr.) | 10 | +1.05 | +10.5 R |
| July | N = 10, 1 ATR, 3 R, 12 | +0.46 (100 tr.) | 37 | −0.06 | −2.2 R |
| August | N = 80, 1.5 ATR, 3 R, 24 | +0.69 (23 tr.) | 10 | +1.04 | +10.4 R |
| September | N = 80, 1.5 ATR, 3 R, 48 | +0.94 (23 tr.) | 10 | −0.21 | −2.1 R |
| October (to the 6th) | N = 80, 1.5 ATR, 3 R, 48 | +0.48 (24 tr.) | 0 |
The overall result: 96 trades, mean R +0.16 ± 0.33, sum +15.6 R. Compared with the +58.1 R of the best chosen looking at everything, it is much less, because walk-forward does not know the future. The value is positive but the interval contains zero. In three months out of seven the chosen rule lost.
A useful fact also shows: the choice changes little. Almost always it is the same region (N = 80, 3 R target), with small variants; only once, in July, it chooses another family (N = 10, 100 trades), and makes −0.06. This is consistent with what we saw: the region N = 80 with a 3 R target had a good period.
Read the chart in words
Daily gold chart with, below, three cumulative R curves from March 2026: the walk-forward rising moderately, the fixed rule higher and the mean of the rules slightly falling.
What it is compared with
From March on, the mean R of all 480 rules is −0.02 (breakout +0.06, fade −0.11). The walk-forward (+0.16) sits above the grid average and below the fixed rule chosen with hindsight (+0.49 from March, +28.6 R). The distance between +0.49 and +0.16 is the price of not knowing the future: the first is not out of sample, because it was chosen on all the data; the second is.
The comparison with chance: we run the same walk-forward on 100 random series (with no trend). The mean R is +0.01 on average (5th percentile −0.18, 95th +0.27), with a median of 109 trades, and is positive in 48% of the series. The real +0.16 lies inside the interval of the random ones: above the mean, below the 95th percentile. It does not tell it apart from chance, and it should be said with the same weight with which one would say it if it had been negative.
The window is a parameter
In the walk-forward we chose a three-month window. It was the most natural thing, and we did not choose it for the results. But if we change it:
| Choice window | Trades | Mean R | Sum |
|---|---|---|---|
| 1 month | 232 | −0.13 | −30.3 R |
| 2 months | 76 | +0.06 | +4.7 R |
| 3 months | 96 | +0.16 | +15.6 R |
| 6 months | 35 | +0.45 | +15.7 R |
From −0.13 to +0.45 R depending on how much history you look at to choose. With a one-month window the chosen rule has very few trades to rely on (in the extreme case 15) and changes every month: the walk-forward loses. With six months it almost always chooses the same region and gives the highest value, on only 35 trades. None of the four is "the" result: the window length is a parameter, and trying four and telling the best one would be the same mistake as in chapters 2 and 3. If you choose it, choose beforehand, and report them all.
Merits and limits
- Merit: it uses all the data and provides many out-of-sample tests in a row; it evaluates the process, not a rule; it imitates real life (you re-optimize now and then).
- Limit 1: window and frequency are parameters, and must be declared beforehand.
- Limit 2: the test periods are few (here seven months with trades) and the trades few (96): the error stays large (±0.33).
- Limit 3: all the test periods come from the same ten months. If that stretch of market is special (chapter 12), the walk-forward inherits it.
- Limit 4: the tests are not truly independent, because the choice windows overlap.
What to do with it
- Use walk-forward to evaluate the process, not a rule chosen at the end.
- Declare in advance window, frequency and choice criterion, and do not change them after seeing the result.
- Compare it with the same walk-forward on random series and with the mean of the rules.
- Report all the windows tried, not just the best.
- Look at how much the chosen rule changes: if it always changes, the process is chasing noise.
Procedure in four steps
- Choose choice window, frequency and criterion, and write them down.
- At each deadline choose the rule on the previous data and trade the next period without modifying it.
- Chain the test periods and compute trades, mean R and error.
- Compare with the mean of the rules and with the same procedure on random series.
Exercises
Chapter exercises
1. Why is every trade of a walk-forward out of sample?
2. The walk-forward gives +0.16 ± 0.33 R over 96 trades; on 100 random series it gives +0.01 (5th-95th percentile −0.18 / +0.27). How is it read?
3. With windows of 1, 2, 3, 6 months the walk-forward gives −0.13, +0.06, +0.16, +0.45 R. What does it mean?
4. Why does the walk-forward (+0.16) sit below the fixed rule chosen looking at the whole period (+0.49)?
Have a question or want to share your exercise?
Post in the community, or join the free signals room on Telegram.
Educational content, not financial advice. Trading involves risk.