What we really know about ICT and SMC: evidence, limits and how to evaluate claims

This is the most important lesson of the course, and the one you usually don't find in paid ICT/SMC courses. It has taught you to define the concepts precisely and to test them. Now we give you the honest picture: what is known, what isn't, why methods can seem effective even when they aren't, and how to evaluate any claim (including this course's) with a critical eye.

What research says about technical analysis

As far as we know at the time of writing:

• We know of no independent, peer-reviewed, high-quality studies demonstrating a statistical edge for the specific setups of the ICT/SMC approach (order blocks, FVGs, killzones...) after costs and on samples outside the development period. This doesn't mean they don't work: it means solid public evidence is missing.
• For technical analysis in general, academic research gives mixed results. A large literature review (Park and Irwin, 2007) found that many modern studies report some profitability, but that they often suffer from problems such as data snooping, unrealistic transaction costs and poor stability of results over time. In the foreign exchange market, various studies have documented profits from technical rules in past periods that then shrank or disappeared.
• On the behaviour of retail traders, regulatory data say that a large majority lose (European and UK brokers must disclose percentages typically between 70% and 85%).

This evidence isn't a condemnation: it's an invitation to caution and personal verification.

Why these methods can seem effective (even if they aren't)

The psychological and statistical mechanisms at play are well known:

1. Flexibility: the concepts have many definitions. In hindsight, almost every chart contains an order block or FVG that "worked", if you look for it. It's chart pareidolia.
2. Survivorship and selection bias: those who succeed show the results; those who lose stay quiet (or quit). Public examples are selected.
3. Confirmation bias: we remember the cases that confirm the idea.
4. Old ideas, new names: many concepts (support and resistance, supply and demand, stop hunting, Fibonacci retracements) are versions of classic ideas. They often work as a description of the past, much less as a forecast.
5. Partly self-fulfilling prophecies: if many traders watch the same levels, price can react to them because they watch them. This can give a temporary edge that disappears when levels become too popular.
6. Variance: on small samples, luck is indistinguishable from skill. A trader can have a streak of 20 winning trades with a system that has no edge.
7. Hindsight bias: after seeing the move, the explanation always seems obvious.

What remains useful

Even without direct proof of an "edge", these concepts offer concrete things:

• A precise vocabulary to describe structure, context and areas.
• Operational discipline: bias, hours, entry and stop rules defined beforehand.
• A way of thinking about risk: stops at points where the idea is invalidated, targets on defined areas, explicit R:R.
• Formalisable, testable ideas (which is what we're doing in this course).
• Structure for the journal and learning: if nothing else, it forces you to observe and record.

How to evaluate the claims of a course, a channel or a "mentor"

Use this checklist with any source:

The phrases that should raise an eyebrow

• "90% win rate."
• "The market is manipulated by the big players coming to take your stops" (without showing how it's verified).
• "I prove it with these 10 examples."
• "If it doesn't work it's because you don't apply it well."
• "Only for a few" / "limited spots" / "discount today".

A scientific attitude: the rules of the game

1. Formulate falsifiable hypotheses. Not "the order block works", but "OBs with displacement and FVG generate an expectancy > 0.1R after costs over 200 out-of-sample cases".
2. Declare beforehand what you'll test. It reduces data snooping.
3. Try to disprove yourself. Use controls and different periods. If you've found "an edge", ask which more boring explanation (chance, costs, bias) might have generated it.
4. Prefer simplicity. Fewer parameters, less risk of fitting noise.
5. Update beliefs when data change. A system that worked can stop.
6. Keep confidence proportional to the evidence. "I have a 300-trade sample with expectancy +0.15R and an interval of 0.03-0.27" is an honest sentence. "This method beats the market" is not.

And if my tests say it doesn't work?

It's a useful result, not a failure. You've saved money and time. You can:

• Simplify (sometimes structure and risk management alone have more value than "advanced" concepts).
• Change hypothesis and retry, with new data.
• Focus on what's under your control: costs, discipline, sizing.
• Decide that short-term discretionary trading isn't for you. It's a rational choice.

How to use all this, in practice

• Treat every concept as a hypothesis, not a truth.
• Test every rule on your data, with realistic costs and out-of-period samples.
• Be wary of anyone promising high win rates or guaranteed returns.
• Start with a demo or a tiny size: the cost of learning must stay sustainable.
• Always keep the journal: it's the only proof you'll have of your method.

In short

• No independent high-quality studies on ICT/SMC setups are known; on technical analysis in general results are mixed and often shrink with costs and over time.
• The methods seem effective through flexibility, selection, confirmation, variance and hindsight, not necessarily through a real edge.
• They remain useful as vocabulary, discipline and structure for testing ideas.
• Evaluate every source with the checklist: written rules, complete statistics, costs, losses, falsifiability, conflicts of interest.
• Be scientific: falsifiable hypotheses, out-of-sample tests, simplicity, confidence proportional to data.

Practical exercise

1. Take a "trading" video or post you follow and apply the checklist: how many questions does it pass?
2. Write one of your own hypotheses in falsifiable form and the data with which you'd refute it.
3. List three "boring" explanations (chance, costs, bias) that could produce a positive result in your test.