MT5 RAM memory voraciousness, problems with reading/writing large files - page 3
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
oop
I can see that oop... why write arrays in strings when you can write them in binary at once
I can see that oop... why write arrays in strings when you can write them in binary at once?
It's written, the man processes them in excel.
It says the man processes them in Excel.
it doesn't say that
it doesn't say that.
The very first post. If you read carefully, this file may not have anything to do with MT5. Or handled in a specific way.
The very first post. If you read carefully, this file may not be related to MT5. Or handled in a specific way.
So let him determine where he's getting it from
Can be read as an array too, I guess.Thanks for the analysis of the code.
I'm having a hard time understanding OOP, so making edits to it is torture for me. As I understood it's about the class, which is responsible for reading, that there the whole file is split into string arrays, and only then, if I want, I convert these string values to another type and put them into arrays of int type in my script. If I understand correctly, then to speed up the process, I should read the entire file and understand how many rows and columns are there and how many symbols are in each cell, and then immediately select the array size? Then how to do it?
I would like to understand where 400 megabytes are taken from at once? Did I get it right that memory is allocated, but it's not cleared/given away that was allocated earlier?
400 Mb is 36 million cells with empty rows, 12 bytes each, 12*36=432.
To be told how to do it better, tell us about the problem you are solving. How often and how much the file changes, whether all have to read from it every time, whether the number of non-empty cells is constant from line to line, whether all cells contain only integers. Maybe 9 minutes should be spent once a week and it is not much at all - then there is no need to change anything.
In general, in terms of "where to go to be faster", the main direction is to declare a structure with 57 fields corresponding to integer values, and use it for both writing and reading (the file is binary). Then 78 Mb will be read faster than a second from disk directly into memory, without delimiters or string representations. For debugging purposes, save the output to .csv too. The file size will immediately determine the number of elements in the array of these structures.
400 MB is 36 million cells with empty rows, each 12 bytes, 12*36=432.
The maths is clear.
To be told how to do it better, tell us about the problem you are solving. How often and how much the file changes, whether everything needs to be read from it every time, whether the number of non-empty cells is constant from line to line, whether all cells contain only integers. Maybe 9 minutes should be spent once a week and it is not much at all - then there is no need to change anything.
The task is universal, it's reading files in CSV format, because this format is universal for different programs, which currently provide me with data. If you're referring specifically to this script, then yes, you just need to read and calculate everything, because Excel crashes because of formulas. The cells may contain non whole numbers (not specifically in this example). 9 minutes is not much if it's once a week... However, among other things, I need to work with such amount of data through optimizer, passing the file to the side and there conducting its processing with the help of agents, then get the result in csv format (here I can, of course, use another format) and conduct the processing again. That is why I decided to discuss this problem publicly, because I want to process large data arrays using MT5.
In general, from "where to go to be faster" point of view - mainstream: declare structure, 57 fields of which correspond to integer values, and use it for both writing and reading (file is binary). Then 78 Mb will be read faster than a second from disk directly into memory, without delimiters or string representations. For debugging purposes, save the output to .csv too. The file size will immediately determine the number of elements in the array of these structures.
I can't even imagine, how it can be done? Or is it that the initial file must be generated under special conditions? Then maybe you can make converter from CSV to binary format?
Can you look at the second class too, how to teach it to write an unlimited number of columns?
WriteArray / Read are fast, max size up to 300 mb, everything is very fast, does not consume RAM
Why is there so much code for reading/writing, it's all done in 4 lines.
Can you demonstrate how to do this?
so let him determine where he gets it from
can also be read as an array, I guessCSV is a universal file format, which is supported in various software, including that associated with the MO.
In addition, it is a convenient file to transfer data between different programs (scripts/advisors) MT5, and it does not take much space.
Can you demonstrate how to do this?
try reading your file like this
https://www.mql5.com/ru/docs/files/filereadarray
I don't have time to open the terminal now, maybe tomorrow)