Reading CSV files - some questions

New comment
 

In order to read out data from a CSV file, I have some questions.

The data is in following format

123.234 , 65.987 , 34.985 , 34535.909 ,

345.345, 5450.45 , 494.93 , 948.34 ,

Every line has the same number of entries and those are all doubles.

Now I seem to manage to read the values one by one. But how could I jump to a next line? 

The lines are a lot longer than the above given example and sometimes it seems to mess up the readings. I need the values of one line at a time and if I read them one by one and it does mess up then it can happen that I use for example all but one values from one line and one value from another line.

I've attached a sample of the code that I'm using for this.

void ReadNewLine(string FileName) {

   int handle;
   handle=FileOpen(FileName, FILE_CSV|FILE_READ,',');
   if(handle>0) { 
      FileSeek(handle,0,SEEK_SET);
      while(!FileIsEnding(handle)){     
         
         for(C1 = 0; C1 < EntriesCount; C1++){
            Entry[C1] =StrToDouble(FileReadString(handle,DOUBLE_VALUE));
            }
     //ProcessEntries();
             

         }

   FileClose(handle);
   }
  return(0);
  }
 
HammerJack:

In order to read out data from a CSV file, I have some questions.

The data is in following format

123.234 , 65.987 , 34.985 , 34535.909 ,

345.345, 5450.45 , 494.93 , 948.34 ,

Every line has the same number of entries and those are all doubles.

Now I seem to manage to read the values one by one. But how could I jump to a next line? 

The simplest way is to read the values from a line and "throw them away" . . . then read the next line and keep those values. 
 
RaptorUK:
The simplest way is to read the values from a line and "throw them away" . . . then read the next line and keep those values. 


Thank for your reply.


Do you mean to throw them away from the sourcefile?


Also, how can the program now that the line is ending?  I've seen the FileIsLineEnding() (https://docs.mql4.com/files/FileIsLineEnding)  function but that didn't work for me. 


PS the files are all around 0.5 GB in size, not sure if this info is relevant.

 
HammerJack:

Thank for your reply.

1.  Do you mean to throw them away from the sourcefile? 

2. Also, how can the program now that the line is ending?  I've seen the FileIsLineEnding() (https://docs.mql4.com/files/FileIsLineEnding)  function but that didn't work for me. 

1.  No,  I mean read the values from the line you want to skip but do nothing with the values you have read,  then the next line is the line that you want,  read it and do what you need to do with those values.

2.  When you read the first value from row/line also test for GetLastError() returning error 4099  (FileIsEnding() didn't work for me either,  I think you have to try and read past the end of the file for it to work)

 

            GetLastError();   // clear GLE internal value
            TicketNumber = FileReadNumber(CSVFileHandle);
            if(D1) Print("Number read from CSV file, Ticket: ", TicketNumber);
            if(GetLastError() == 4099)       // we have reached the end of the file
               {
               Print("End of file reached unexpectedly, terminating the for loop ! !");
               EOF = true;
               break;
               }
 

Thanks RaptorUK.

In the end in turned out that my calculations were off, I was looking for one value too much in every row..



Now another question:


How can the speed of an EA be increased? In order to find good settings I'm having an EA looping over data to find patterns/settings. But this looping takes really long. How can it be done faster? The processor isn't used too the max  so I guess there is another bottleneck. When I switch off the calculations but leave on the fileread part it goes much faster, that points me in direction of the calculations. How could those be improved as in terms of speed?

New comment