Load once for math calcs in tester ? - page 2

New comment
 
Nice my pc bricked xD
 

It takes 1 + a half minutes to run two passes (2) with a data file that takes 12 secs to load normally.

The tester function also has nothing in it.Just the load.

So , there is no caching from what i can tell .

Since there's so much overhead in loading and dumping and loading and dumping the same file over and over 
its more efficient to construct your own genetic algorithm , load the data ONCE and have at it with one core.

dissapointing 

EDIT : 

Also , with property tester_file , you have to compile , WAIT depending on how big your file is , close the IDE , close MT5 , 

go to the tester folder . Delete all files , reopen MT5 and test.

EDIT 2 : 

if i manage to make this work i'll post the method here 

 

I think your tester has memory leak issues on math calcs mode.

I shrank the test file to 800MB , i've got 4 cores.

On a normal chart it loads fine and properly (the file with the same code)

On math calcs mode i run out of memory.

I assume it needs to load it 4 times , right ? one per core so 3.2 GB on a 16GB machine.

Memory bloats over 5GB at some point.

I have a strict ZeroMemory instruction after the load too , no change.

The load function unloads all arrays before it loads the file 

I checked my save/load functions for consistency.

I assume there is an issue with loading doubles on the tester from a file.

Will try to replicate with sharable code.

Cheers 

 

test scripts .

  • Run _Build first , the current settings will create a 900Megabyte file
  • Run _Test on a normal chart , verify the "collections are identical" (making sure the file loads properly normally)
  • Open _Test , compile it.
  • Wait 1 minute.
  • Close the editor
  • Close MT5
  • Go to the tester folder 
  • Nuke it
  • Open MT5 
  • Run a math calcs optimization from 1 to 10 (on _Test)

Let me know thanks .

[_]p beer

PS : dont try it on your main pc
 
Lorentzos Roussos #:

test scripts .

  • Run _Build first , the current settings will create a 900Megabyte file
  • Run _Test on a normal chart , verify the "collections are identical" (making sure the file loads properly normally)
  • Open _Test , compile it.
  • Wait 1 minute.
  • Close the editor
  • Close MT5
  • Go to the tester folder 
  • Nuke it
  • Open MT5 
  • Run a math calcs optimization from 1 to 10 (on _Test)

Let me know thanks .

[_]p beer

PS : dont try it on your main pc

It seems you don't understand that a data file of 900 MB will create memory objects (you have 3 imbricated arrays of classes) using around 10 times more memory ?

Do you think that using structs, classes and arrays is not using any memory ?

I run it with 1000 shelves instead of 10,000 : data file is 94.8 MB while the Terminal is using 1165 MB to keep them in memory.

 
Alain Verleyen #:

It seems you don't understand that a data file of 900 MB will create memory objects (you have 3 imbricated arrays of classes) using around 10 times more memory ?

Do you think that using structs, classes and arrays is not using any memory ?

I run it with 1000 shelves instead of 10,000 : data file is 94.8 MB while the Terminal is using 1165 MB to keep them in memory.

No i dont understand that thats true .

So the "tree" is the issue ? 

I have to reduce dimensionality?

And i'll also have to take into account user's mem size in exports too if that's the case.
12
New comment