EA Optimization Without Overfitting: How to Tune a Bot for Prop Firm Rules

EA optimization is a brute force search: the tester runs the same strategy over the same history many times, changing the input parameters each time, and sorts the runs by a score you choose. It ranks how well each combination fitted one stretch of past prices.

In MetaTrader 5 you mark which inputs to vary, give each a start, step and stop, and pick an algorithm. The slow complete algorithm tests every combination, while the fast genetic algorithm samples the space and breeds the better performers. You also pick the criterion: balance max, profit factor max, expected payoff max, drawdown min, recovery factor max, Sharpe ratio max, the complex criterion, or a custom score written in the EA's OnTester function.

The criterion is the definition of a good result, and the optimiser will find whatever you ask for. Ask for maximum balance and you get the combination that made the most money on that history, regardless of how close it came to blowing the account on its worst day.

Test setup (modelling, spread, commission, period) is covered in how to backtest an EA properly, and why live results trail the tester is in EA backtest vs live results. This post is narrower: what the optimisation step does to the numbers, and how to read its output without fooling yourself.

Overfitting is when a parameter set describes the random detail of one historical period instead of a behaviour that repeats. Every price series contains both, and the tester cannot separate them.

The problem grows with the number of combinations you test. If you test five settings, the best one is mostly the strategy. If you test fifty thousand, the best one is the strategy plus the luckiest draw out of fifty thousand. The top of the table is, by construction, the result with the most luck baked in, which is why it is the least likely to repeat.

Free parameters make this worse quickly. Each input you allow to vary multiplies the search space and gives the optimiser another dial to bend toward the past. Two inputs and a few hundred trades leave limited room to memorise history. Nine inputs, per session settings and a time filter can fit almost anything, including a series with no edge.

A useful rule of thumb: count the trades and count the free parameters. If the trade count is not many times larger than the number of things you tuned, the optimiser had enough freedom to describe the history rather than test a hypothesis about it.

An overfitting trading bot leaves fingerprints in the optimisation results, visible before a single live trade. Check each on any settings you plan to trust.

A robust parameter set is one that sits on a plateau: a broad region of the parameter space where nearby values produce similar, acceptable results. You are not looking for the highest point on the map. You are looking for high ground that stays high when you take a step in any direction, because live conditions never match the tuning period exactly.

In practice, optimise with coarse steps first, view the result as a heatmap, and pick a value near the centre of a stable region rather than the best cell. The centre usually earns less than the peak. The peak's extra profit was mostly the luck the optimiser selected for.

Walk-forward testing is the standard defence against out-of-sample collapse. You optimise on one window, validate the chosen settings on the next window, then slide both windows forward and repeat across the whole history. What you judge is the chain of validation windows stitched together, since none of those results were seen by the optimiser that chose the settings.

MetaTrader 5 has a built in version of the first step. The Forward setting splits the period, optimising on the first part and forward testing on the rest, with fixed fractions of a half, a third or a quarter, or a custom start date, and reports the forward results in their own tab next to the optimisation results. Rolling walk-forward has to be done by hand, but even one forward split exposes many settings that only fit the optimisation period.

Do not re-optimise until the forward column looks good too. That turns the forward period into training data and removes your only honest check.

For a prop firm account, the right objective is the best result among settings that never breach the firm's loss limits in testing, not the highest net profit. Touching the daily loss limit or the maximum drawdown commonly ends an evaluation on the spot.

Firms structure these limits differently. Daily loss may be measured from the start of day balance or equity, the maximum drawdown may be static or trailing, and both vary by plan and account type and change over time, so check the current rules on the firm's own site. The distinction between the two limits is in daily loss vs max drawdown, and the trailing mechanics are in trailing drawdown explained.

None of the default criteria in MT5 knows your firm's rules. Drawdown min uses the tester's own drawdown figure, measured from equity peaks over the whole test, which may not match how your firm calculates it, and no built in criterion tracks the worst single day. The fix is a custom criterion: in OnTester, compute the worst daily equity loss and the maximum drawdown as your firm measures them, return zero for any run that breaches either with a margin, and only then score the survivors.

Leave headroom. If the hypothetical limit is 5 percent per day, reject anything whose worst tested day exceeds about 3 percent, because the next path through the market will contain a day the tester never saw.

Every number here is invented to show the mechanism. Take a hypothetical $100,000 evaluation with a 5 percent daily loss limit ($5,000) and a 10 percent maximum drawdown ($10,000). The EA has four inputs: a fast average from 5 to 50 in steps of 5 (10 values), a slow average from 60 to 240 in steps of 20 (10 values), a stop from 20 to 100 points in steps of 10 (9 values) and a take profit multiple from 1.0 to 3.0 in steps of 0.5 (5 values). That is 10 x 10 x 9 x 5, or 4,500 combinations, optimised on three years of history, with a fourth, later year held out as a forward test.

Sorted by balance, the top run shows $41,300 net profit, a maximum drawdown of 8.9 percent and a worst single day of $6,200. That worst day already breaches the $5,000 daily limit, so on a real evaluation this setting would have failed in the backtest itself. On the heatmap it is a lone bright cell, and its neighbours earn between a loss of $2,000 and a profit of $9,000. In the forward year it loses $3,100. Raise the spread by 0.3 pips and its three year profit falls to $18,600.

Now apply the constraint: reject any run with a worst day above $3,000 or a maximum drawdown above $6,000. About 600 of the 4,500 runs survive. Among them, the setting at the centre of the widest stable region shows $14,800 over the three years, a worst day of $2,300 and a maximum drawdown of 4.1 percent, with neighbours between $10,000 and $16,000. In the forward year it makes $3,400, roughly two thirds of its in-sample yearly pace of about $4,900, and with 0.3 pips more spread the three year figure drops to $11,900.

The constrained setting looks far worse on the results table and is the only one worth demo time. It degraded instead of collapsing, survived higher costs, and never came near the limits that end the account. That is not a projection of any account's results, only evidence the setting was not disqualified.

Optimisation removes bad settings, it does not discover good ones. Run any candidate through these steps and stop at the first failure.

Most traders do not want to write OnTester code or run rolling walk-forward tests. The questions above still apply to any settings you receive: how many inputs were tuned, on which period, validated how, and against which rule set.

PraxAI, which publishes this blog, includes an Elite Optimizer that produces a custom configuration for a chosen firm, account size and risk level. Ask the same questions of it as of any other settings. The default configuration caps the grid at three layers, uses no martingale and places a stop on every order. How that compares with other options is covered in choosing an AI trading bot for prop firms. Whatever you use, verify the firm's current rules and its policy on automation yourself.

Trading carries risk. A well optimised EA can still lose an evaluation.