Part 2: In-sample and out-of-sample · Chapter 7 of 14 · 9 min
Looking into the future
A timing error turns a rule from −0.02 R into one of +0.83 R: the same 20-candle rule, entering at the open of the candle whose signal uses the close, wins 92% of the time and makes +342 R. A daily rule that uses the day's close to buy at the open gives +1.61 R per trade and 98% wins, against −0.15 R if you enter the next day. Numbers this good are the alarm signal.
What it is
A backtest looks into the future when, at a point in time, it uses information that did not yet exist at that moment. It is the most damaging mistake, because it leaves no trace: the code runs, the numbers come out, and they are beautiful. A one-candle offset is enough.
The rule to avoid it is a question to ask at every decision: at this hour, which data really exist? If a number you use to decide forms after the moment you decide, the test is wrong.
First example: the signal and the entry in the same candle
Our signal uses the close of a candle (beyond the high of the previous N). The close is known only at the end of the candle. The correct entry is at the close (as in the books) or at the open of the next candle. The mistake is entering at the open of the same candle: it seems harmless, but you buy at the open already knowing the candle will close higher.
Let us measure the effect on the simplest rule, that of chapter 1 (20-candle breakout, 1 ATR stop, 1 R target, duration 24):
| Entry | Trades | Mean R ± error | Winning | Sum |
|---|---|---|---|---|
| At the close (honest) | 307 | −0.02 ± 0.11 | 50% | −4.6 R |
| At the open of the same candle (mistake) | 413 | +0.83 ± 0.05 | 92% | +342.3 R |
Read the chart in words
Daily gold chart with, below, two cumulative R curves: the one with the mistake, very high, and the honest one, near zero.
A single wrong line takes the rule from zero to +342 R, and the winning share to 92%. On the whole grid the mistake gives a mean of +0.085 R against −0.017 honest, 240 positive rules out of 480 (against 189), breakouts at +1.005 R and fades at −0.835: the direction of the breakout is that of the candle, so knowing how the candle closes is enough to choose the direction. The best with the mistake (80-candle breakout, 0.5 ATR stop, 3 R target, duration 12) makes +2.02 ± 0.25 R over 181 trades, winning 76%.
Entry at the next candle (at the open after the signal, the most realistic choice) does not change the picture: grid mean −0.017, best +0.73 (against +0.74).
Second example: the whole day
A more subtle form is born when timeframes are mixed. Daily rule: "if today's close exceeds yesterday's high I buy, if it falls below yesterday's low I sell", with a stop equal to half the mean range of the previous 14 days and exit at the close. If you enter at today's open, you decide to buy by already using today's close, which at that hour does not exist. If you enter at tomorrow's open, it is correct.
| Entry | Days with a signal | Mean R ± error | Winning | Sum |
|---|---|---|---|---|
| At the open of the same day (mistake) | 112 | +1.61 ± 0.25 | 98% | +180.4 R |
| At the open of the next day (honest) | 112 | −0.15 ± 0.28 | 44% | −16.9 R |
Read the chart in words
Daily gold chart with, below, two cumulative R curves: the one with the mistake rising strongly and the honest one falling slightly.
98% of days in profit is not a system: it is a mistake. A professional trader who sees a backtest with 98% wins on the same rule does not think "I found the Grail", but "where am I looking into the future?".
The forms in which it appears
- Signal on the close, entry at the open of the same candle (first example).
- Mixed timeframes: using the day's (or week's) high, low or close during the day itself (second example).
- Indicators that use future candles: centred averages, peaks (swings) recognised with later candles and drawn at the point of the peak, zigzags that redraw (repainting). The peak that "can be seen" only five candles later was not visible at that point.
- Data cleaned or adjusted afterwards: corrections, contract substitutions, cuts of odd data decided after seeing what happened.
- Parameters and filters chosen on the whole period and then applied "as if" from the start: it is a form of time out of place (chapter 5).
- Selection of today's instruments (those that exist today and do well): who disappeared is not in the sample.
How to find it
- Numbers too good. Winning share above 70-80% with non-trivial targets, tiny errors (±0.05 over 413 trades), almost straight curves: they are the first bell. A real edge is small and noisy.
- The delay test. Delay every entry by one candle (or one day). If the result collapses (+0.83 → −0.02; +1.61 → −0.15), there was a timing error. A real edge loses little with a small delay.
- The list of timestamps. For every number used in a decision, write at what hour it forms. If one is after the decision, the test is wrong.
- The hand test. Take five trades, and redo the decision looking at the chart up to that instant, hiding the rest.
What to do with it
- Enter at the close of the signal or at the next open, never at the open of the same candle.
- Use data of the higher timeframe only when they are closed (yesterday's close from today on).
- Check indicators that redraw or use future candles.
- Always do the delay test and distrust numbers that are too good.
- Write the formation time of every data that enters a rule.
Procedure in four steps
- For every rule, write at what hour each data it uses forms.
- Enter at the close of the signal or at the next open.
- Do the delay test of one candle (or one day) and compare.
- If the numbers are too good, redo five trades by hand hiding the future.
Exercises
Chapter exercises
1. A backtest gives 92% of trades in profit, with +0.83 R and an error of ±0.05, for a simple breakout rule. What do you suspect?
2. You delay every entry by one candle and mean R goes from +0.83 to −0.02. What does it mean?
3. At 00:00 you buy "if today's close exceeds yesterday's high". Where is the mistake?
4. Which of these is a correct entry for a signal based on the close of the candle?
Have a question or want to share your exercise?
Post in the community, or join the free signals room on Telegram.
Educational content, not financial advice. Trading involves risk.