Part 3: Chance and robustness · Chapter 12 of 14 · 9 min

Periods and regimes

Let us split the ten months into five periods of about two months. The 240 breakout rules have a positive mean R in four periods out of five (from +0.02 to +0.22), the 240 fade rules are negative in all (from −0.19 to −0.02). The ranking of the rules is kept from one period to the next only at the start (rank correlation +0.72, +0.29, then −0.01 and +0.03). The best rule is positive in every period, but 48% of its gain comes from the first, when gold rises 18%.

The market changes

A ten-month backtest is a photograph of a stretch of market, not of "the market". In ten months gold had a strong rise (at the start), stretches of long decline and stretches almost flat. These stretches are called regimes: periods in which the market behaves in a certain way (trend, range, high or low volatility). A rule that follows the movements earns in trending regimes and loses in directionless ones, and the opposite holds for one that goes against them. A backtest that crosses only one type of regime measures the rule and the regime together, and they cannot be separated.

The simplest check: look at the result period by period.

Five periods

We split the 4,865 candles into five periods of equal length (about 973 candles, two months). For each: the movement of gold and the mean R of the 240 breakout rules and the 240 fade rules (mean over the rules), the number of positive rules and the best rule of chapter 2.

PeriodGold in the periodBreakout: mean RFade: mean RPositive rulesThe best of chapter 2: trades, mean R, sum
1 (15 Dec - 11 Feb)+17.8%+0.22−0.19194 out of 48016, +1.66, +26.5 R
2 (11 Feb - 10 Apr)−5.6%+0.06−0.1817917, +0.54, +9.2 R
3 (10 Apr - 9 Jun)−10.7%+0.02−0.0622617, +0.21, +3.6 R
4 (9 Jun - 7 Aug)+0.1%+0.17−0.1923814, +0.72, +10.1 R
5 (7 Aug - 6 Oct)−2.5%−0.02−0.0221518, +0.32, +5.8 R

What it reads.

  1. Fade is negative in all periods (from −0.19 to −0.02). In the ten months going against the movements always cost.
  2. Breakout is positive in four periods out of five, but the size changes a lot: +0.22 in the first (gold rises 18%) and +0.17 in the fourth, close to zero in the second and third, −0.02 in the fifth. The edge of breakouts does not coincide with the direction of gold: in the fourth period the price ends where it started and breakouts earn all the same. What explains it, with these data we cannot say.
  3. The best rule is positive in every period, but the sum is very different: +26.5 R in the first, +9.2, +3.6, +10.1 and +5.8 in the others. The first period makes 48% of the total; without it the rule is worth about +28.7 R over four periods.
Figure 1. Mean cumulative R per rule: the 240 breakout rules (pink, +12.5 R per rule) and the 240 fade rules (bars, −27.4 R per rule), December 2025 - October 2026. The two curves separate at once, and breakout rises above all in the first part.Data: gold, XAUUSDT perpetual future (real candles) · daily (D1) · from 15 Dec 2025 to 5 Oct 2026.
Read the chart in words

Daily gold chart with, below, two curves of mean cumulative R per rule: that of breakouts rising, that of fades falling steadily.

The ranking is not kept

A rule is worth something if, good in one period, it is also good in the next. Let us compute the rank correlation between the results of the 480 rules in consecutive periods:

Periods comparedRank correlation
1 and 2+0.72
2 and 3+0.29
3 and 4−0.01
4 and 5+0.03

At the start the ranking is kept (+0.72), then it is lost: from the third period on, which rules do well in one period says almost nothing about which do well in the next. Part of this is real structure that changes (the regime), and part is noise (with 14-18 trades per period, the mean R of a rule has an error of ±0.8 or more). In practice: a two-month result says little about the following two months.

Where the gain comes from, month by month

The best rule, month by month (R of the trades closed in that month):

Figure 2. R of each month of the best rule (pink bars, marked at the end of the month): +7.9 (December), +15.6 (January), +2.0, +8.3, −0.1, +4.9, +7.8, −0.4, +14.1 (August), −2.1 (September). The sum is +58.1 R; two months (January and August) are worth +29.7 R.Data: gold, XAUUSDT perpetual future (real candles) · daily (D1) · from 15 Dec 2025 to 5 Oct 2026.
Read the chart in words

Daily gold chart with, below, ten bars of the monthly R of the best rule, mostly positive.

Seven months out of ten are positive, three near zero or negative (April, July, September). Two months, January (+15.6) and August (+14.1), make +29.7 R by themselves, more than half the total. This is typical of rules that aim at a large target (3 R): they lose more than half the time (they win 46%, with an average loss of 0.99 R) and now and then earn a lot (average win 2.79 R). The gain depends on a few big moves, and if none occurs, the rule can spend months near zero.

What it means for the future

What can we say, with these numbers, and what not?

  • We can say that in the ten months following the movements paid and going against them did not, with some variation, and that the edge of breakouts was positive in most periods.
  • We cannot say that it will continue. We have seen a single stretch of market (a strong rise, a long decline, stretches of wide swings), and we do not know what the same rules would do in a year of narrow range and low volatility, or in a phase with a long opposite trend.
  • We cannot separate the rule from the regime. To do so you would need more years, more instruments, or different regimes.

This is the largest limit of the book, and it holds for every backtest: the sample is a stretch of history. The shorter the sample, the more it resembles a single regime.

What to do with it

  1. Always look at the result by period (two, three months) and by month: where does the gain come from?
  2. Remove the best period and see what remains.
  3. Ask in which regime you have been: trend, range, high or low volatility, and how many different regimes there are in the sample.
  4. Do not count nearby periods as independent trials: the correlation between periods says how much new information there is.
  5. Try the rule on other periods and other instruments, when you can, before believing it.

Procedure in four steps

  1. Split the sample into 4-6 equal periods and compute the result of each.
  2. Remove the best period and recompute.
  3. Look at the result month by month and how many months make half the total.
  4. Declare which regimes the sample contains and which it does not.

Exercises

Chapter exercises

1. The best rule has sums of +26.5, +9.2, +3.6, +10.1 and +5.8 R in the five periods (total +55.2 R over the periods). What share comes from the first period?

2. Fade is negative in all five periods and breakout is positive in four. How is it read?

3. The rank correlation of the ranking of rules between consecutive periods is +0.72, +0.29, −0.01, +0.03. What does it mean?

4. Why are five periods of two months a weak test of stability?

Have a question or want to share your exercise?

Post in the community, or join the free signals room on Telegram.

Educational content, not financial advice. Trading involves risk.