Every backtest you tune is tested on the data you tuned it on. That is why tuned backtests look good, and why they tell you almost nothing on their own. The fix is to keep some data back: tune on one stretch of history, then test on a stretch the tuning never saw. MT5 calls this forward testing.
This article explains MT5’s built-in forward setting, why we switch it off and do the same job by hand, and what out-of-sample testing has actually done to our own EAs. It is part of our guide to how to backtest an EA in MetaTrader 5, step by step.
What MT5’s Forward Setting Does
In the Strategy Tester, the Forward option splits your date range in two. The optimiser searches for the best inputs on the first part. The tester then runs those inputs on the held-back final part — a half, a third, a quarter, or from a date you pick — and shows the two results side by side.
It is a good feature, and if you optimise inside the tester you should use it. The question it answers is the right one: does what worked here still work over there?
Why We Switch It Off
We run our tests from a script, and the script turns the forward split off for every run. This is the line from the configuration it writes:
"ForwardMode=0",That is not a dismissal of out-of-sample testing. With the split off, each out-of-sample window is its own run and its own report: filed, dated, and published with its trade count and its tick source, so anyone can check it. A forward split lives inside one optimisation run. Separate runs can be read, compared and re-run years later.
So our protocol is a fixed set of windows:
- Tuning window: 1 January 2025 to 5 September 2026. Inputs are chosen here, and only here.
- Validation window: 1 January 2023 to 31 December 2024. No tune has ever seen it.
- Older windows (2019–2022, and back to 2010 for some EAs) as further out-of-sample tests.
Notice the direction. MT5’s split holds back the most recent data. Ours holds back older data. That has a cost: our out-of-sample test is not a test of the future, and an older market can differ from today’s in ways that flatter or punish a strategy unfairly. It also has an advantage: the tuning window is the one with real recorded ticks (our broker’s tick history starts in 2026) and the one most like the market the EA will trade. We accept the cost and say so. The real test of the future is the next step, live trading.
What Out-of-Sample Testing Did to Our EAs
These are our own results, all published with their reports on the presets page.
Two that looked good and were rejected. Our London range-reversion EA, tuned on AUDJPY, scored a profit factor of 1.49 on 49 trades in the tuning window. On the 2023–24 validation window it scored 0.75 on 75 trades. The NZDJPY version went from 1.27 on 45 trades to 0.73 on 60. Both are marked rejected, with their numbers still on the site. Without the validation window, both would have looked like working strategies.
One that passed, then didn’t. The EURUSD version (preset 0.6.0) scored 1.44 on 61 trades in the tuning window and 1.27 on 107 trades in 2023–24. A forward test of the usual kind would have stopped there and called it proven. The older 2019–2022 window came in at 0.94 on 209 trades. It now runs live at minimum size under a tier we call regime: it worked recently and lost earlier, so it is not proven, and backtests alone can no longer promote it.
One that went the other way. The GBPUSD version scored 0.80 in the tuning window and 1.39 on 132 trades in 2023–24. A good out-of-sample result, and it is still not promoted. Our promotion rule requires every filed window to have a profit factor of at least 1.0 and the most recent window to be positive. This one lost money in the most recent window, so it stays a candidate.
What We Learned From Doing It This Way
Out-of-sample results disagree with tuned results more often than they agree. Of the four EAs above, not one kept the same verdict across every window. In our research that has been the common case, not bad luck: of 28 studies written up since September, one mechanism held. It is why a single backtest, however good, is not evidence.
Small trade counts lie in both directions. 45 or 49 trades can produce a profit factor of 1.5 by chance. The validation windows above have 60 to 209 trades, and even those are modest. When a window has few trades, we treat its result as a hint, not a verdict.
More windows beat a cleverer split. The EURUSD case only showed up because we tested a third window. Two windows that agree can still both be lucky; a third that disagrees is the cheapest information you will ever get.
How to Do It Yourself
- Fix your windows before you tune. Write the dates down. Changing them after you see a result is tuning by another name.
- Tune only in the tuning window. If you use MT5’s optimiser, turn on the forward split, or keep the validation dates out of the optimisation range entirely.
- Run each held-back window as a separate test, with the tuned inputs unchanged, and keep every report.
- Decide the pass rule in advance. Ours is published in full in course module 6: A Backtest Is Not Evidence.
- Keep the failures. A rejected EA with its numbers is the most useful record you have. It tells you which ideas not to test again.
Forward testing in the tester is still a backtest. The forward test that counts is the one with real money, which is why every EA that passes ours goes on to the live track record and is judged there, one trade at a time.
Comments