Part 1: What a backtest is · Chapter 4 of 14 · 8 min
Parameters and fitting
Every parameter left free is one more way to fit the rule to noise. On random series, choosing the best with 1 free parameter the mean R is +0.05 in the first part and +0.01 in the second; with 5 parameters it is +0.35 and +0.02. On the real data the rule chosen with 3-5 parameters goes from +0.75-0.82 R in the first part to +0.43-0.46 in the second: more than half of the gain, but with 25-27 trades the error is ±0.7.
One more parameter
Draw ten points scattered at random on a sheet. With a straight line you touch a few; with a nine-parameter curve you pass through all of them, and the result "describes the data perfectly". The curve has discovered nothing: it has learned the noise. Predicting a new point with that curve is worse than with the line. This is overfitting: a rule with too many free parameters fits the data you have, chance included, and loses strength on new data.
In a backtest the parameters are the breakout length, the stop, the target, the duration, the direction and any threshold or filter. Every parameter you let the data choose is a degree of freedom.
The same family, with more free parameters
Let us run an experiment. We split the data in two parts: the first (65%, until 24 June 2026, 3,162 candles) to choose, the second (35%, 1,703 candles) to test. Then we leave free, one at a time, more parameters: from N alone (the others fixed at 1 ATR stop, 1 R target, duration 24, breakout) to all five. In each case we choose the best of the first part (at least 20 trades) and look at how it does in the second. Mean R:
| Free parameters | Rules | Random series: first part → second | Real data: first part → second |
|---|---|---|---|
| 1 (N) | 5 | +0.05 → +0.01 | +0.22 (103 tr.) → +0.17 (42 tr.) |
| 2 (N, stop) | 20 | +0.10 → +0.01 | +0.22 → +0.17 (same rule) |
| 3 (N, stop, target) | 80 | +0.26 → +0.06 | +0.75 (59 tr.) → +0.46 (27 tr.) |
| 4 (+ duration) | 240 | +0.30 → +0.08 | +0.82 (55 tr.) → +0.43 (25 tr.) |
| 5 (+ direction) | 480 | +0.35 → +0.02 | +0.82 (55 tr.) → +0.43 (25 tr.) |
(The random series are 300; the second part is not exactly zero because inside each one-day block the structure remains.)
On the random series the picture is clear: the more free parameters, the better the result in the first part (from +0.05 to +0.35) and no improvement in the second (from +0.01 to +0.08 at most). The gap between the two parts goes from 0.03 to 0.33 R. It is the cost of fitting the noise: everything the first part "gains" more with each parameter is, on average, fictitious.
On the real data the first two rows give the same rule (80-candle breakout, 1 ATR stop, 1 R target, duration 24), with +0.22 R in the first part and +0.17 in the second: it holds. With three free parameters the first-part result rises to +0.75 (80-candle breakout, 1.5 ATR stop, 3 R target) and in the second falls to +0.46: it loses 39%. With four and five the rule is that of chapter 2 (duration 48), +0.82 → +0.43: it loses 48%. The value in the second part stays positive, but with 25 trades the error is ±0.7: it is neither a confirmation nor a refutation.
Read the chart in words
Daily gold chart with, below, three cumulative R curves and a grey band over the last third of the period, where the curves slow down.
| When | What | Price |
|---|---|---|
| 24 Jun 2026 → 5 Oct 2026 | second part, not used to choose |
Times in UTC.
Trades per parameter
A rule with many parameters and few trades is fragile. The rule with five free parameters and 55 trades in the first part has about 11 trades per parameter. There is no magic threshold, but the order of magnitude is this: below 20 trades per free parameter, almost everything you observe can be fitting. A rule with one parameter and 100 trades (the first row) is much harder to fool.
The parameters you do not see
You also count those that do not appear in the rule:
- The values you tried and discarded. If for N you looked at 10, 20, 30, 50, 80 and then others, each is a degree of freedom. A finer grid (twice the values for each of the five parameters) would give 32 times more rules.
- The choice of timeframe, instrument, period. Changing one of these after seeing a result is one more free parameter.
- The thresholds of the filters (hour of day, minimum volatility) that "look reasonable" after looking at the chart.
The way to reduce degrees of freedom is to choose beforehand, for reasons that do not come from the data: round values (20, 50), a length that makes sense, a stop in ATR (a unit that adapts to volatility), a single motivated filter. Fewer parameters, less fitting, and a result that is worth more.
What to do with it
- Count the free parameters, tried and discarded values included, and write them next to the result.
- Reduce the parameters by fixing some at round or motivated values before looking at the data.
- Check the trades per parameter: under 20, the rule is fragile.
- Always compare first and second part (chapter 5): the gap tells how much is fitting.
- Prefer the simplest rule that gives a similar result.
Procedure in four steps
- List the parameters of the rule and the values you tried for each.
- Fix, before looking at the results, those you can fix for a reason that does not come from the data.
- Compute the trades per free parameter.
- Compare the result of the two halves of the data and report the gap.
Exercises
Chapter exercises
1. On random series the best with 5 free parameters is worth +0.35 R in the first part. How much is it worth, roughly, in the second?
2. On the real data the rule with 3 parameters goes from +0.75 R in the first part to +0.46 R in the second. By how much did it fall?
3. What counts as a free parameter in a backtest?
4. A rule has 5 free parameters and 55 trades. What can be said?
Have a question or want to share your exercise?
Post in the community, or join the free signals room on Telegram.
Educational content, not financial advice. Trading involves risk.