Something is wrong. Live is behind the backtest, or a preset that looked fine has spent three months going sideways, or a study that was beautiful on paper turned into a strategy that loses money.
There is one move that always feels like progress and almost never is: open the optimiser and find better parameters. It will find them. It finds them for random data too. Retuning is the one response guaranteed to produce a satisfying answer regardless of what is actually wrong, which is exactly why it is the wrong first move.
The disappointing result has one of four causes, and only the last is a tuning problem: you measured it wrong, you ran something other than what you think, the improvement you found is not an improvement, or the edge is gone. This module is how we tell them apart, using our own failures.
Did you measure it right?
Before blaming the market, suspect the instrument. Here is a study of ours that found a strong, consistent, statistically overwhelming effect that did not exist:
**The trap.** The first pass used server-day open→close on BID bars and found the Asian
session positive on every pair every weekday with t-stats up to 17, and the late session
negative everywhere. That is not a market effect: spreads balloon at the daily rollover
(00:00 / 23:55 server) and at the weekend, which pushes the bid down at the day open and the
Friday close, faking a positive Asia, a negative Friday and a positive Monday.A t-statistic of 17 is not a marginal result. It is the kind of number that makes you start building. And it was entirely an artifact of measuring bid prices across the moment of the day when the spread is widest.
The mechanism is worth understanding because it generalises to any instrument you quote one side of. Bid equals mid minus half the spread. At rollover the spread balloons, so the bid drops — with no trade, no flow, and no information. Measure a return that starts at that moment and it looks like a gain; measure one that ends there and it looks like a loss. Do that consistently every day and you manufacture a weekday pattern with a huge t-statistic and no cause.
The fix was a measurement rule, not a filter: anchor daily returns at 07:00 server, use mid prices, and keep session windows away from the rollover. Re-measured cleanly, the entire study collapsed to one surviving row — which was the spread.
If a result is much stronger than the effect could plausibly be, check the ruler before you celebrate.
Did you run what you think you ran?
The second question is whether the thing you tested is the thing you named.
Our session windows are configured as an hour range plus a fixed offset, and for months the presets were described as trading the London morning at 07:00–13:00 UTC. They were not. The broker's server sits at UTC+3 in summer, the tester's clock reports server time, and the hours were being applied in server terms — so the window that was labelled 07:00–13:00 UTC was actually running 04:00–10:00 UTC.
Every backtest was internally consistent, so nothing looked wrong. The strategy really had been tested; it had just been tested on a different three hours than the label claimed, and any attempt to reason about why it worked — "the European morning range" — was reasoning about the wrong window.
The fix was mechanical (gate on the same clock live and in the tester) but the durable part is the convention: every published session is now stated in server hours with the UTC translation spelled out, right down to the rejection notes.
New York session (server 15-24 = 12-21 UTC), London pv 0.6.0 values: 87 trades, PF 1.00,
-$0.08, 8 of 21 months positive. Breakeven; the session adds trades and no edge.Time zones, symbol suffixes, point-vs-pip and the modelling mode are all in this category. They do not produce errors; they produce results about a different question.
Build the test so it can tell you the entry is dead
Suppose the measurement is sound and you ran what you meant to. Now the question is whether the edge is weak or absent — and the way to find out is a test with a stated prediction that can come out the wrong way.
Our Trend entry was losing on M15 with profit factors of 0.85 to 0.93. The tempting story was that costs were eating it: the stop was tight, so spread was a large fraction of the risk. That story makes a prediction, and the prediction is testable on a higher timeframe where the same stop multiple is a much bigger distance.
EURUSD H1, 126 trades, real ticks, 2025-01-01 to 2026-09-05: PF 0.53, net -$80. SL took 70%
of exits; TP reached on 17% of SL-or-TP trades (M15 was 21-26%). Run to test whether the M15
result (ea-trend-multi-m15, PF 0.85-0.93 on 3 pairs) was spread-bound - on H1 spread is ~5% of
the stop instead of ~15%, and the result got worse, not better. The Trend entry (EMA 70/190 +
slope, RSI 48/52 reclaim, ADX 22-42) has no edge at either timeframe.That is the shape to copy. A hypothesis ("it is spread-bound"), a setting where the hypothesis implies a specific direction of change, and a result that went the other way. Cost was not the problem. The entry does not select well enough to hit its target often enough, and quieter costs just made that clearer.
The exit geometry got the same treatment — and this is where the optimiser earns its keep, used as a diagnostic instead of a fix:
Exit-geometry optimisation on EURUSD (TpR 0.8-2.4 x MaxHold 140-620, 25 passes, 2026-09-08)
found NO positive pass: best -$4.60 at TpR 1.6 / hold 380; tighter targets were worse (-$14 to
-$26). The entry has no edge net of spread at any exit geometry on M15. Not a tuning problem.Twenty-five passes over the exit surface, no positive cell anywhere. That is a far stronger statement than any single backtest: if no setting in a wide sweep is profitable, the problem is upstream of the settings. Run the optimiser to find out whether a winning configuration exists at all — then do not ship the winner.
Change one thing, and keep a control
When you do change something, the change has to be attributable.
We tested whether a daily-timeframe reading could pick which side to trade. The design detail that made the result trustworthy was leaving one arm of the system untouched:
2023-24 out of sample: 198 gated shorts, -$10.1, 53% win, PF 0.84 (EURJPY 0.48, EURUSD 0.94,
USDJPY 1.01, GBPUSD 1.15). Longs unchanged from the proven presets on every pair (the control
held).Longs were identical to the proven preset, so if the long results had also moved, something about the run was different and the short numbers would mean nothing. The control held, so the shorts' failure was the gate's.
The same logic isolates a variable across instruments. When a strategy was tried in a different session, the three pairs disagreed in a way that answered the question:
Not worth an out-of-sample run; the session, not the pair, is the problem (EURUSD 1.00,
EURJPY 0.40).One pair doing badly is a pair problem. Three pairs doing badly in the same new session, when all three worked in the old one, is a session problem. Vary one dimension at a time across several instruments and the failures tell you which dimension owns them.
Improvements that are only leverage
Here is the subtlest failure mode, and the one most likely to get shipped.
We tested sizing the fix trade by the morning's range — bigger on busy mornings, smaller on quiet ones. It worked: total pips went from +2,152 to +3,537 over the same 549 trades, an improvement of roughly 64%.
It was not an improvement. Average position size rose to 1.38×, maximum drawdown went from 75 to 141, and the ratio that actually matters fell:
| scheme | half | trades sized>0 | sum pips | mean per trade | mean/std | max DD (pips) | avg size |
|---|---|---|---|---|---|---|---|
| constant 1x | all | 549 | +2152 | +3.92 | +0.344 | 75 | 1.00 |
| step 0.5 / 1 / 2 (<15 / 15-30 / >=30) | all | 549 | +3537 | +6.44 | +0.307 | 141 | 1.38 |Mean over standard deviation went down, from 0.344 to 0.307. More pips, more risk, and slightly worse risk-adjusted return. The same test on the other pair moved 0.400 to 0.360 with drawdown going from 45 to 101.
That is leverage wearing the costume of an edge, and you can always manufacture it: any position-sizing rule that increases average size will increase total profit on a profitable system. The check is to compare per unit of risk, not per trade, and to notice whether average size changed at all. Constant size stayed.
When the losses are the mechanism
The last question is the hardest to accept: what if nothing is wrong?
Our fix strategy wins about 63–66% of the time. The losses are not small — the losers average about −6.8 pips against winners of about +9.8. The obvious instinct is to find what the losing days have in common and filter them out.
So we looked, across 549 trades per pair and every condition available at the moment of entry: the pre-move in the hour before, the overnight direction, the three-hour direction, the morning's range, the spread at the fix, the weekday, and which calendar day of the month it was. Split every one of those into halves and compare.
Almost every cell was positive in both halves. The exceptions had single-digit trade counts. No condition available at entry separates the winners from the losers — which means the −6.8 and the +9.8 and the 63% are not a defect in the system, they are the system. That distribution is what the mechanism looks like.
A filter fitted to those losers would have removed a random third of the sample, improved the backtest, and improved nothing else. The conclusion we wrote down was to stop: do not add entry filters, and do not expect indicators to add any either — a later test of RSI, MACD and EMA states at the fix minute found they carried nothing the calendar did not already have.
The counterpart is a filter that looks like it works and does not:
gated 71 trades +$35.5 vs ungated 188 trades +$94.0 - identical $0.50 per trade. The gate
removed 62% of the trades and 62% of the profit.A filter that removes trades and profit in the same proportion has selected nothing. It has only made the sample smaller — which makes every statistic noisier while looking, on a win-rate basis, like discipline.
What you should have now
An order of operations. Check the ruler, then check that you ran what you named, then build a test whose result can be "this is dead", keep a control arm when you change something, compare improvements per unit of risk rather than per trade, and be prepared for the answer that the losses are simply what your edge looks like.
Retune last, if at all. By the time you have done the other five honestly, you usually know whether there is anything left to tune.
Next module: knowing when a line of research is finished — how we closed the pair, session and timeframe axes for an entire strategy, and why writing down "do not test this again" is worth more than most positive results.