Part 2: In-sample and out-of-sample · Chapter 8 of 14 · 9 min

Costs and execution in the backtest

A backtest must decide what happens when a candle touches both stop and target, at what price you enter and how much it costs. The best rule of the grid does not change (+0.74 R in all cases), but a narrow-stop rule (10-candle breakout, 0.5 ATR stop, 1 R target) goes from −0.17 R with the prudent rule to +0.12 with the optimistic one: the sign depends on the assumption. With costs from 0 to 2 $ the grid mean goes from −0.001 to −0.110 R and the ranking of the rules stays almost the same (correlation 0.98).

What the backtest decides for you

A backtest works on candles, not on orders. Inside a candle you do not know what happened first. For every trade you therefore have to fix three assumptions:

  1. Ambiguous candles. If a candle touches both the stop and the target, which came first? With candles alone you cannot tell.
  2. Entry price. At the close of the signal candle, at the open of the next, at a worse price?
  3. Costs. Spread, commission, slippage, in dollars per ounce.

Each is a choice of the researcher, and each can move the result. Here we measure them on the whole grid and on two rules.

Ambiguous candles

In the books we used the prudent rule: if stop and target are touched in the same candle, the stop counts. The alternatives are the optimistic rule (the target counts) and a coin flip (50/50). Effect on the grid:

Ambiguous candleMean of mean RPositive rulesWith lower limit > 0Best
Stop first (prudent)−0.01718932+0.74
Coin flip+0.00923133+0.74
Target first (optimistic)+0.03627372+0.74

On the grid, with the optimistic one 3,103 trades out of 108,309 (2.9%) turn out different. That is enough to move the positive rules from 189 to 273 and those with a lower limit above zero from 32 to 72. The best rule does not change in any of the three: it has wide stops (median risk 28.4 $), and in none of its 79 trades does a candle touch both stop and target.

For a narrow-stop rule the picture is different. 10-candle breakout, 0.5 ATR stop (about 11 $), 1 R target, duration 12: 499 trades, 74 of which (15%) have an ambiguous candle.

Narrow-stop rule, ambiguous candleMean R
Stop first (prudent)−0.17
Coin flip−0.00
Target first (optimistic)+0.12

The sign of the result depends on the assumption. With a small stop candles often cover both levels, and a backtest on hourly candles cannot say which came first. A trader who tests it with the optimistic assumption "discovers" a rule that earns; with the prudent one the same rule loses. To know what really happened you would need finer data (one-minute candles or ticks). Until you have them, choose the prudent assumption and declare the choice: if a result lives only in the optimistic assumption, it does not exist.

Figure 1. Narrow-stop rule (10-candle breakout, 0.5 ATR stop, 1 R target, duration 12; 499 trades): cumulative R with cost 0 $ (bars, −70.0 R), 0.3 $ (pink, −86.5 R), 1 $ (blue, −125.0 R) and 2 $ (dashed, −180.0 R), with the prudent assumption for ambiguous candles.Data: gold, XAUUSDT perpetual future (real candles) · daily (D1) · from 15 Dec 2025 to 5 Oct 2026.
Read the chart in words

Daily gold chart with, below, four falling cumulative R curves, steeper the higher the cost.

The entry price

Entering at the close of the signal candle (as in the books) or at the open of the next (more realistic for whoever watches the candle close and then sends the order) changes little: grid mean −0.017 in both cases, rules with a lower limit above zero 32 against 36, best +0.74 against +0.73. It is an assumption that moves nothing here, and that it is right to always check (chapter 7 showed when it matters).

Costs

With costs of 0, 0.3, 1 and 2 $ per ounce:

CostMean of mean RPositive rulesWith lower limit > 0BestSum of the best rule
0 $−0.00120538+0.75+58.9 R
0.3 $ (base case)−0.01718932+0.74+58.1 R
1 $−0.05516726+0.71+56.2 R
2 $−0.11014114+0.68+53.5 R

The grid mean falls by 0.11 R going from 0 to 2 $ and the rules in profit from 205 to 141. But the best rule loses barely 0.07 R (from +0.75 to +0.68): its stops are wide and the cost weighs little (chapter 10 of book 11). The ranking of the rules hardly changes at all: the rank correlation between the results at cost 0 and at 1 $ is 0.976. The cost moves everything down, and more so for narrow-stop rules; the narrow-stop rule above goes from −0.14 R at zero cost to −0.36 R at 2 $.

Figure 2. The best rule (wide stop): cumulative R with cost 0 $ (bars, +58.9 R), 0.3 $ (pink, +58.1 R), 1 $ (blue, +56.2 R) and 2 $ (dashed, +53.5 R). The four curves hardly separate.Data: gold, XAUUSDT perpetual future (real candles) · daily (D1) · from 15 Dec 2025 to 5 Oct 2026.
Read the chart in words

Daily gold chart with, below, four cumulative R curves very close to one another, all rising.

What the backtest does not contain

Even with prudent assumptions many things stay out:

  • Slippage on stops. A candle that opens beyond the stop leads to a loss of −2.58 R instead of −1 (chapter 8 of book 11: 2 trades out of 276).
  • The spread that changes with the hour and with news, and the lack of liquidity.
  • Orders not executed, partial fills and latency. An entry "at the close" is not always obtainable at the closing price.
  • The cost of holding the position (swap, perpetual-contract funding), which on the short-duration rules of the book counts little but on longer ones does.
  • Differences between platforms and brokers, and the weekend jumps of CFDs (absent in our data).

The practical rule: everything the backtest does not contain goes in the worst direction. A result that holds with double costs and with the prudent assumption has a margin. One that lives only with ideal costs and optimistic candles has none.

What to do with it

  1. Write the execution assumptions (ambiguous candle, entry, cost) next to the result.
  2. Choose the prudent assumption: stop first, entry at the next candle if you are not sure, a cost that is not minimal.
  3. Redo the backtest with worse assumptions (double cost, ambiguous against you) and see whether the sign holds.
  4. For narrow-stop rules use finer data, or do not trust the result.
  5. Add to the result the costs the test does not see.

Procedure in four steps

  1. Write the three assumptions: ambiguous candle, entry price, cost.
  2. Measure how many trades have an ambiguous candle.
  3. Redo the test with the worst assumptions and with double cost.
  4. If the sign changes, there is no result: there is an assumption.

Exercises

Chapter exercises

1. A narrow-stop rule makes −0.17 R with the prudent rule for ambiguous candles and +0.12 R with the optimistic one. What do you do?

2. Why does the best rule of the grid not change with the assumptions on ambiguous candles?

3. The rank correlation between the results at cost 0 and at cost 1 $ is 0.976. What does it say?

4. Which set of assumptions is prudent?

Have a question or want to share your exercise?

Post in the community, or join the free signals room on Telegram.

Educational content, not financial advice. Trading involves risk.