Backtest results incomprehension

[Yoloic]

Hello fellow traders,

I'm running optimizations on an EA I'm currently developing and when I backtest on a 4 years period from 2020.01.01 to 2024.01.01, this is the results I get:

And when I backtested it on a 3 years period from  2021.01.01 to 2024.01.01 here are the results I get:




I don't understand why the drawdown values are so different.

The only thing I've done is changing the starting date from 2020 to 2021.
The backtest is performed on a custom symbol using data from tickstory, 1 minutes timeframe.
I use relative lot size calculation. Initial deposite 10.000€.

Shouldn't the drawdown of the 4 years period test be bigger or at beast equal to the 3 years period test drawdown?

I don't know if I'm missing something obvious or if there is an issue somewhere.
Hopefully someone can help.

Thank you in advance for your time! 

[Alain Verleyen]

Because you are using compounding lots size (lot size increasing with balance) and you have a different starting point.

The check, test and compare a strategy, never use "relative lot size", use something fixed (for example using the initial starting balance).

[Yoloic]

Alain thank you very much for the feedback, that's very interesting!

I understand that the monetary drawdown are different due to the lot size being relative to the equity, but shouldn't the percentage drawdown be the same regardless of the monetary amount?
Also what do you mean by "for example using the initial starting balance"? Do you mean to use minimum lot size/fixed lot size instead?

I didn't use fixed lot size because on a series of videos from darwinx they compare the different optimization methods, and CAGR/MeanDD was ranked first to get the most robust parameters.(I can send the link of the video series if you want to have a look but I'm not sure if I'm allowed to share Youtube videos here)

That's why I was using relative lot size based on equity as CAGR/MeanDD custom metric doesn't work with fixed lot size.

I'm really curious to know what you think about that, and also what you recommand to run optimizations? (fixed lot size I suppose)

Always happy to learn :)

[Yoloic]

I think I got it about using the initial starting balance.

You meant to use a percentage of the initial starting balance instead so that the lot size remains the same throughout the optimization, correct? 

[Alain Verleyen]

Yoloic #:
I think I got it about using the initial starting balance.

You meant to use a percentage of the initial starting balance instead so that the lot size remains the same throughout the optimization, correct? 

Yes. Not necessarily the same but independent of the increasing balance.

Why ? You just experimented why...because it gives such results as you get (not necessarily so spectacular difference, but misleading results).

[Alain Verleyen]

Yoloic #:
Alain thank you very much for the feedback, that's very interesting!

I understand that the monetary drawdown are different due to the lot size being relative to the equity, but shouldn't the percentage drawdown be the same regardless of the monetary amount?
Also what do you mean by "for example using the initial starting balance"? Do you mean to use minimum lot size/fixed lot size instead?

I didn't use fixed lot size because on a series of videos from darwinx they compare the different optimization methods, and CAGR/MeanDD was ranked first to get the most robust parameters.(I can send the link of the video series if you want to have a look but I'm not sure if I'm allowed to share Youtube videos here)

That's why I was using relative lot size based on equity as CAGR/MeanDD custom metric doesn't work with fixed lot size.

I'm really curious to know what you think about that, and also what you recommand to run optimizations? (fixed lot size I suppose)

Always happy to learn :)

Yes I am curious about this CAGR/MeanDD ?

The %DD is different due to your starting point. If we start at a time there is a good series your account will increase quickly and so your lot size, but then with bigger lot size you got a DD time, it will be a lot bigger than if it was at the start (with smaller lot size).

Depending of your strategy and coding, a different starting point can also lead to completely different trades, and so DD%, but that point I can't judge in your case from what you posted.

[Yoloic]

Alain Verleyen #:

Yes I am curious about this CAGR/MeanDD ?

The %DD is different due to your starting point. If we start at a time there is a good series your account will increase quickly and so your lot size, but then with bigger lot size you got a DD time, it will be a lot bigger than if it was at the start (with smaller lot size).

Depending of your strategy and coding, a different starting point can also lead to completely different trades, and so DD%, but that point I can't judge in your case from what you posted.

I understand that a different starting time can modify the sequence of trades a little bit at the beginning and make the results vary a little bit, but I was using 1% of equity for the risk.
So even if the starting time, balance, and opened lot size are different, it should still be 1% of the equity. And it should logically have the same impact on the %DD as the distance from entry in drawdown is the same and based on 1% of equity. (assuming that we have the exact same positions, or no completely different sequence of bad positions, because one is less than 10% and the other one more than 66%...)

About the CAGR/MeanDD metric, here is the video series:
https://www.youtube.com/playlist?list=PLv-cA-4O3y95J6xmwSaCILL4FlGJZO0PJ


The first introduction to the CAGR/MeanDD metric is this video:

10.1) Using CAGR / Mean Drawdown as a Trading System Performance Metric in Backtests & Optimizations


The videos where he compares the different optimization methods are these videos:

15.1) Research Study | Which Optimization Performance Metric is best? (PART 1)

15.2) Profit Factor, Return/Max Drawdown, Sharpe Ratio, Expected Payoff.. which is best? (PART 2)


Other explainations and the code for this metric here, from another video series of them:

4.1) Why Return/MEAN-Drawdown is a better measure of trading performance than MAX Drawdown

4.2) Coding MT5 Custom Performance Criteria in MQL5 using Return / Avg Drawdown instead of Max DD


I will try with fixed lot size as you advised.
Although, I'm just not sure yet how to asses which parameters are the best using fixed lot size.
Do you run a second test using relative lot size once you have a good set of parameters from the first test made using fixed lot size? Or maybe a second test with custom metrics?

[Yoloic]

The final result of the experiment is in this video:
16.2) The answer to which optimization performance criteria works best | Research (PART 5)

[Yoloic]

After consideration, fixed lot size might provides consistency in trade sizes, making it easier to compare the performance of different parameter sets.

But it evaluates the performance of the strategy without considering the effect of changing account equity and does not reflect how the strategy would perform in real-world scenarios where the trade size changes with equity.

So is it really reliable...?  Wouldn't Normalized/Relative Profit Factor provides a more accurate and robust measure of trading strategy performance? 

Normalized/Relative Profit Factor criteria normalizes the profits and losses by trade size, ensuring that the performance metric is consistent regardless of the variability in trade sizes.

What do you think?

[Said M Attiya]

Do you know how to get reliable historical data? Thanks