Part 3: Chance and robustness · Chapter 11 of 14 · 10 min

Parameter robustness

A robust rule is worth almost the same if you change the parameters a little; a fragile rule is a spike. For the best (+0.74 R) the five neighbours (one parameter changed by one step) are worth from +0.28 to +0.66, on average +0.43: all positive, but at little more than half. The neighbours/best ratio is 0.58 against 0.52 for the best of the random series: it is not more robust than noise. Choosing by the mean of the neighbours did not help: +0.12 in the second part against +0.43.

Plateau and spike

If a rule really works, its result should change little when you change a parameter by a step (a length of 80 instead of 50, a stop of 1.5 ATR instead of 2). You have a plateau: around the rule, the values are similar. If the rule is a spike (high itself, low all around) it is more likely that its result depends on a coincidence of those data. The reason is simple: the future market will not have exactly the same characteristics, and a useful rule must tolerate small differences.

In the grid the neighbours of a rule are those that differ in a single parameter and by a single step (N at the previous or next value, stop, target, duration); the direction stays the same.

The neighbours of the best

The best is: 80-candle breakout, 1.5 ATR stop, 3 R target, duration 48 (+0.74 R). Its five neighbours:

NeighbourMean R
N = 50 (instead of 80)+0.28
Stop 1 ATR (instead of 1.5)+0.45
Stop 2 ATR+0.44
Target 2 R (instead of 3)+0.32
Duration 24 (instead of 48)+0.66
Mean of the neighbours+0.43

All five are positive, and this is a good sign: no neighbour reverses the result. But none reaches +0.74, and the mean stands at +0.43, 58% of the value. The rule is not an isolated spike, and it is not a flat plateau either: it has a marked maximum inside a positive region.

Figure 1. Cumulative R of the best (pink, +58.1 R) and of its five neighbours (from +31 to +57 R): N = 50 (+32.7), 1 ATR stop (+48.8), 2 ATR stop (+31.3), 2 R target (+31.9), duration 24 (+56.5). The six curves have the same shape and rise together.Data: gold, XAUUSDT perpetual future (real candles) · daily (D1) · from 15 Dec 2025 to 5 Oct 2026.
Read the chart in words

Daily gold chart with, below, six cumulative R curves rising together, the pink one highest.

The parameter map

To see it better you make a map. We fix 3 R target, duration 48 and breakout, and vary the length N (rows) and the stop in ATR (columns). Mean R (trades) over the whole period:

NStop 0.5Stop 1Stop 1.5Stop 2
10−0.00 (441)+0.19 (299)+0.22 (210)+0.23 (161)
20−0.10 (322)+0.16 (232)+0.15 (178)+0.24 (140)
30−0.10 (258)+0.17 (190)+0.26 (141)+0.16 (123)
50−0.07 (200)+0.19 (151)+0.28 (118)+0.20 (103)
80+0.03 (151)+0.45 (109)+0.74 (79)+0.44 (72)

Two things, at once. Along the stop (columns) there is a plateau: from 1 ATR up all the values are positive, from +0.15 to +0.74. Along N (rows) there is a jump: from lengths 10-50, which sit between +0.15 and +0.28, to N = 80, which reaches +0.74. The best rule is not at an isolated point of the map, but in a row that stands out from the others. That row might be a good region or the edge of the grid (80 is the highest value we tried: we do not know what 120 or 200 would do).

First and second part, same map:

NFirst part: stop 0.5 / 1 / 1.5 / 2Second part: stop 0.5 / 1 / 1.5 / 2
10−0.05 / +0.19 / +0.32 / +0.41+0.08 / +0.18 / +0.05 / −0.09
20−0.10 / +0.23 / +0.27 / +0.49−0.11 / +0.02 / −0.09 / −0.23
30−0.12 / +0.17 / +0.43 / +0.34−0.06 / +0.15 / −0.10 / −0.25
50−0.08 / +0.27 / +0.51 / +0.39−0.05 / +0.04 / −0.12 / −0.17
80−0.04 / +0.43 / +0.82 / +0.53+0.20 / +0.48 / +0.43 / +0.12

In the second part almost the whole map switches off (values around zero or negative): only the N = 80 row stays consistently positive (from +0.12 to +0.48), while the N = 10 row is barely positive (from −0.09 to +0.18). This is the information the map gives and the single number does not: the good structure, over the whole period, sits in the N = 80 row, and there the second part is positive too. The other lengths had positive results in the first part, and then went to zero.

The comparison with noise

Even the best of a random series has lower neighbours: it is the regression to the mean of chapter 5, applied to parameters. How much lower? Ratio between the mean of the neighbours and the value of the best:

BestMean of its neighboursRatio
Real data+0.74+0.430.58
Random series (mean)+0.28+0.150.52

The real best has a ratio of 0.58, almost equal to that of the random best (0.52). So robustness to neighbouring parameters does not tell the real rule from one picked from noise: even the best of a random series does not collapse when one parameter is changed. A good ratio is necessary, and not enough.

Choosing by the mean of the neighbours

A remedy often proposed: do not choose the rule with the highest mean R, but the one with the highest neighbour mean (more "plateau", less "spike"). Let us try, in the first part, and see how it does in the second:

Selection criterionRuleFirst part (neighbour mean)Second part
Highest mean R80-candle breakout, 1.5 ATR stop, 3 R target, duration 48+0.82 (+0.51)+0.43 ± 0.73 (25 tr.)
Highest neighbour mean80-candle breakout, 2 ATR stop, 3 R target, duration 48+0.53 (+0.55), 50 tr.+0.12 ± 0.73 (23 tr.)

The neighbours criterion chose a nearby rule (stop 2 instead of 1.5), and in the second part it gave less (+0.12 against +0.43). With 23-25 trades and an error of ±0.73, the difference is neither proof for nor against. The rank comparison over the 480 rules is also almost equal: the correlation with the second part is +0.43 using the rule's own mean R and +0.46 using the neighbour mean. Honest conclusion: looking at neighbours helps to distrust isolated spikes, but it is not a guarantee and, here, it does not improve the choice.

What to do with it

  1. Always look at the neighbours of the chosen rule: one parameter at a time, one step forward and one back.
  2. Draw the map of the two most important parameters: you look for a plateau, not a spike.
  3. Check the edges of the grid: if the best sits at the extreme of a parameter, you do not know whether the maximum is inside or outside.
  4. Compare the neighbours/best ratio with that of random series: you need a value better than noise.
  5. Do not use robustness as a substitute for the out-of-sample test.

Procedure in four steps

  1. List the neighbours of the chosen rule and their result.
  2. Draw the map of the two main parameters, over the whole period and over the two parts.
  3. Compute the neighbours/best ratio and compare it with that on random series.
  4. Check whether the best sits at the edges of the grid, and if so, extend it.

Exercises

Chapter exercises

1. The best is worth +0.74 R and the mean of its neighbours +0.43 R. What is the ratio, roughly?

2. What distinguishes a plateau from a spike?

3. The neighbours/best ratio is 0.58 on real data and 0.52 for the best of the random series. What does it mean?

4. Choosing by the neighbour mean gave +0.12 R in the second part against +0.43 R choosing by mean R, with 23-25 trades and an error of ±0.73. What do you conclude?

Have a question or want to share your exercise?

Post in the community, or join the free signals room on Telegram.

Educational content, not financial advice. Trading involves risk.