What's your criteria for "good enough for live"?

New comment
 

What metrics do you use to objectively measure performance of your EAs? And what do you consider good enough to put into production?

At the moment, I'm focused on prop trading, and using that as a good reason to focus more on minimizing drawdown and maximizing consistency, more so than maximizing profits. At this point, I use two primary metrics to decide what EAs/markets/timeframes I'll roll out to live (with a 0.01 lot size to start):

  1. Calmar ratio - The annualized Recovery Factor from the last 3 years. Optimize for Recovery Factor and divide by 3. That means an RF of at least 9.0 in the 3-year backtest.
  2. LR Correlation - This actually does a great job of helping avoid overfitting and finding strategies and settings that perform well in out-of-sample data. The past 3 years have had just about every market regime imaginable, but just Recovery Factor alone isn't enough. The LR Correlation tells you how consistent it is. In my experience so far, strategies with very high R values generally perform well on out-of-sample data as well. I look for an LR Correlation of 0.95+. If the Recovery Factor and/or Profit Factor or Sharpe Ratio is really exceptional, I'll look at algos down to a 0.9, but not below that.
  3. # of trades - Have to have enough trades to trust the numbers above. I look for at least 1-2 trades per week from a strategy, but don't always get it.

It's hard enough to find algos and settings that meet those criteria, I don't have enough choices to be picky over other metrics.  I have 11 in production so far, though several don't have the trading frequency I'd like. If I find one new one per week, I'm happy.

What about you?  What's your criteria for moving an algo/market from backtesting/forwardtesting into a live account?

 
Scott Allen:
algos d

I don't think LR ratio can save you from over-fitting.

But consistent results over three years(on a monthly basis) is good enough to go for prop challenge.

Personally, If I want to pass a prop challenge will use all the knowledge acquired from all my past trading experience to achieve profit target.

Relying solely on expert advisors to pass challenges is pretty optimistic I think.

 
Scott Allen:

What metrics do you use to objectively measure performance of your EAs? And what do you consider good enough to put into production?

At the moment, I'm focused on prop trading, and using that as a good reason to focus more on minimizing drawdown and maximizing consistency, more so than maximizing profits. At this point, I use two primary metrics to decide what EAs/markets/timeframes I'll roll out to live (with a 0.01 lot size to start):

  1. Calmar ratio - The annualized Recovery Factor from the last 3 years. Optimize for Recovery Factor and divide by 3. That means an RF of at least 9.0 in the 3-year backtest.
  2. LR Correlation - This actually does a great job of helping avoid overfitting and finding strategies and settings that perform well in out-of-sample data. The past 3 years have had just about every market regime imaginable, but just Recovery Factor alone isn't enough. The LR Correlation tells you how consistent it is. In my experience so far, strategies with very high R values generally perform well on out-of-sample data as well. I look for an LR Correlation of 0.95+. If the Recovery Factor and/or Profit Factor or Sharpe Ratio is really exceptional, I'll look at algos down to a 0.9, but not below that.
  3. # of trades - Have to have enough trades to trust the numbers above. I look for at least 1-2 trades per week from a strategy, but don't always get it.

It's hard enough to find algos and settings that meet those criteria, I don't have enough choices to be picky over other metrics.  I have 11 in production so far, though several don't have the trading frequency I'd like. If I find one new one per week, I'm happy.

What about you?  What's your criteria for moving an algo/market from backtesting/forwardtesting into a live account?

I like one metric that is based on a metric of sunny harris, you calculate it with 3 variables: minimum of profit factor and recover factor; percentage of profit trades; average risk:return. You multiply everthing, and usually if a system is above 1.4 I sent it to the market.
 
Ricardo Rodrigues Lucca #:
I like one metric that is based on a metric of sunny harris, you calculate it with 3 variables: minimum of profit factor and recover factor; percentage of profit trades; average risk:return. You multiply everthing, and usually if a system is above 1.4 I sent it to the market.

Wow, that's very interesting. I will have to test that on mine. I'm assuming that's annualized profit or recovery factor?

EDIT:  OK I went and tried it...

It's not straightforward for me to calculate average risk:return, because all of my stop losses are algorithmic, sometimes like an ATR multiplier, sometimes something else, like the high or low of the last candle or the open of the current one. And my exits are pretty much all driven by indicators or price action, not a set risk:return. And since I do hard trailing stops, my trading log will show the final stop loss, not the initial one. So I don't know how I'd even get that data.

Also, being primarily focused on drawdown, in the context of prop trading, in a lot of my strategies, the annualized Recovery Factor is 2-3x the Profit Factor, which would prioritize it over RF.

Cool idea, but it doesn't fit my trading and risk management strategies. It does make me think that there may be a way for me to quantify a mix of my important metrics into a single metric that could be prioritized for in the strategy tester. For starters, the idea of a correlation-adjusted recovery factor (RF * LR) comes to mind.

 

After few years

1. Only Walk Forward Optimization is best approach for non-stationary timeseries models like "ours" EA's. On MQL5 site you can find 2 or 3 good articles on this.

2. Walk Forward is "hard" to implement and time consuming for optimization in MQL5. If I have no time, resources and etc Im using simple optimization with follow formula

double OnTester(){

   double Trades = TesterStatistics(STAT_TRADES);
   double EquityDD = TesterStatistics(STAT_EQUITYDD_PERCENT);
   double EquityDDRel = TesterStatistics(STAT_EQUITY_DDREL_PERCENT);
   double NetProfit = TesterStatistics(STAT_PROFIT);
   double ReturnOfInvest = NetProfit / 1000.0;

   if((Trades < 200) || (EquityDD >= 50.0) || (EquityDDRel >= 50.0) || (ReturnOfInvest < 2)) return (0);   

   double ProfitFactor = TesterStatistics(STAT_PROFIT_FACTOR);
   double RecoveryFactor = TesterStatistics(STAT_RECOVERY_FACTOR);

   double ProfitTrades = TesterStatistics(STAT_PROFIT_TRADES);
   double LossTrades = TesterStatistics(STAT_LOSS_TRADES);
   
   double GrossProfit = TesterStatistics(STAT_GROSS_PROFIT);
   double GrossLoss = TesterStatistics(STAT_GROSS_LOSS);
   
   double AvgGrossProfit = GrossProfit / ProfitTrades;
   double AvgGrossLoss = GrossLoss / LossTrades;
   double AvgProfitToLossRatio_Real = AvgGrossProfit / MathAbs(AvgGrossLoss);

   double WinPercent = 100.0 * ProfitTrades / Trades;
   double AvgProfitToLossRatio_Theory = (110.0 - WinPercent) / (WinPercent - 10.0) + 1.0;
   
   double CSTS = AvgProfitToLossRatio_Real / AvgProfitToLossRatio_Theory;

   //return Trades * ReturnOfInvest * AvgProfitToLossRatio * ProfitFactor * RecoveryFactor * (1/EquityDD) * (1/EquityDDRel);
   //return Trades * ReturnOfInvest * ProfitFactor * RecoveryFactor * (1/EquityDD) * (1/EquityDDRel);
   //return Trades * ReturnOfInvest * AvgProfitToLossRatio_Real * (1/EquityDD) * (1/EquityDDRel);
   //return  CSTS;

   return Trades * ReturnOfInvest * CSTS * (1/EquityDD) * (1/EquityDDRel);
}
New comment