Manual backtesting: how to test a setup without fooling yourself

On social media every method seems to work, because only the successful examples are shown. The only way to know whether a setup really has an edge is to measure it over many cases, with rules fixed in advance and costs included. Manual backtesting is the most accessible way: it needs no programming, lets you see the market "as it is" and teaches you a great deal. But it's also the easiest to distort without noticing. This lesson gives you a rigorous procedure.

Why manual backtesting (and its limits)

Advantages: you can test concepts that are hard to code (like an order block "by eye"), you learn to recognise real cases, no programming needed. Limits: it's slow (100 trades can take 6-10 hours), it's subject to the tester's biases, and it rarely goes beyond a few hundred cases. That's why we'll later pair it with an automated test.

The tools

• TradingView Bar Replay: lets you go back to a past date and advance candles one at a time. It's the correct way to "hide the future". (The free plan limits historical depth; for M15/M5 a few months are often enough.)
• A spreadsheet (Google Sheets/Excel) to record every trade.
• A fixed time zone (UTC) on the chart.

The procedure, step by step

1. Write all the rules BEFORE looking at the data

Context, setup, trigger, stop, target, management, risk. If it isn't written, it isn't testable. Better still: save the document with the date.

2. Define the sample

• Instrument and period: for example EUR/USD, last 12 months.
• Different regimes: make sure it includes trend, range, high and low volatility.
• Minimum number: at least 100 trades (the more the better). With fewer than 30, results are almost unusable.

3. Hide the future

Use replay and advance one candle at a time. Never look to the right. If your setup requires knowing whether "the MSS will have displacement", wait for the candle to close.

4. Record every trade the rules generate

Every one. Even the ugly ones, even those you "don't like", even those that lose immediately. If the rules say enter, you enter (on the sheet).

5. Don't change the rules while testing

If you notice a rule needs improving, note it and finish the sample with the original rules. Then create a version 2 and test it on new data.

6. Include costs

For each trade subtract spread and commissions (for example 1.0-1.5 pips on EUR/USD in the killzones, more elsewhere). Remember slippage: enter at the candle after confirmation, not at the "perfect" price.

7. Calculate the statistics (next lesson)

Win rate, average R, expectancy, profit factor, drawdown, maximum losing streak, with the margin of error.

The recording sheet

Add a column for every variable you'll want to analyse later (killzone, area type, entry depth). Record discarded cases with the reason ("no displacement", "insufficient R:R"): they'll show you whether the rules are too rigid.

The biases that distort a manual backtest

1. Look-ahead: "recognising" the setup because you already know what happens next. Replay reduces the problem but doesn't eliminate it: if you've already seen that chart (for example on Twitter), you're contaminated. Choose periods you've never analysed.
2. Selection (cherry-picking): skipping "doubtful" cases or marking only the good-looking setups in hindsight.
3. Rule drift: subtly modifying criteria as you go, because "you've got it now".
4. Overfitting: tweaking parameters until the sample "fits". The system learns noise.
5. Ignored or underestimated costs.
6. Small sample: 20 trades "say" something only to those who want to hear it.
7. Survivorship / pair choice: testing 10 pairs and showing the best.
8. Wrong hour: daylight saving handled badly (see the timing lesson).
9. Execution optimism: in reality you'll enter a bit later, with slippage and variable spread.

Splitting the data: development and verification

A simple method not to fool yourself:

• Development (in-sample): for example the first 6 months. Here you can refine the rules (in moderation).
• Verification (out-of-sample): the following 6 months, never looked at during development. The rules are frozen and applied without modification.

If results collapse in the second period, the system was fitted to noise. If they hold (even slightly worse, which is normal), it's a good sign.

Robustness: small changes, big differences?

A robust system doesn't depend on exact values. Check:

• Parameter sensitivity: change the displacement threshold from 1.5 to 1.3 and 1.7 × ATR, the FVG minimum size from 0.3 to 0.2 and 0.4. If the result changes radically, it was a fragile fit.
• Different pairs: apply the same rules to GBP/USD, USD/JPY. It doesn't have to work identically, but it mustn't collapse suspiciously.
• Different periods: strong trend vs. range.
• Execution delay: enter 1 candle later: what changes?

The random control: the question few ask

Your setup has an edge compared to what? Build a control case with the same management (same stops and targets in ATR, same R:R) but with "random" or simpler entries:

• Control 1: market entry at every killzone open, in the direction of the bias.
• Control 2: entry on the 50% retracement of every impulse, with no sweep or FVG.

If your model doesn't beat these controls (consistently higher expectancy), the additional components (sweep, FVG...) add no value. It's the same logic as a clinical trial with a control group.

When the sample is "enough"

There's no magic number. A practical rule: calculate the standard error of the expectancy (next lesson). If the 95% confidence interval includes zero, you can't say the system has an edge. In practice, to detect an expectancy of +0.2R with typical trade dispersion (standard deviation ~1.5R) you need about 200-300 trades. For +0.1R, about 900-1,000. That's why a statistic on 30 "nice" trades is worth nothing.

From backtest to demo

If the backtest is promising, the next phase is demo (50-100 trades) with the same rules and the same journal, to verify execution, slippage and discipline. Only after that, live at minimum risk.

Typical mistakes

• Running few tests and drawing conclusions.
• Testing until the result pleases you.
• Not recording broken rules.
• Forgetting costs and slippage.
• Having no control.
• Confusing "works in the test" with "works over time".

In short

• Manual backtesting with replay is accessible and instructive, but vulnerable to bias: rules written first, sample ≥ 100, all cases recorded, costs included.
• Split the data into development and verification and freeze the rules before verification.
• Check robustness (parameters, pairs, periods) and compare with a simple control.
• A small sample proves nothing: you need hundreds of trades for small expectancies.
• After the backtest, demo; only then live at minimum risk.

Practical exercise

1. Write the rules of the sweep-MSS-FVG model and save them with the date.
2. Test 100 trades on never-seen periods, recording everything with 1 pip of costs.
3. Apply two simple controls and compare expectancy.