Working with files. - page 5

[TheXpert]

Yedelkin:

Question 1: Do you think there's a typo in the documentation, and that instead of "Last Read Date" it should say something like "Last Open Date"?

Yes, here's the verbatim:

lpLastAccessTime [out, optional]

A pointer to aFILETIME structure to receive the date and time the file or directory was last accessed. The last access time includes the last time the file or directory was written to, read from, or, in the case of executable files, run.

Loose translation -- last access includes the last time the file was read or written or executed (if executable).

I'm probably wrong about the handle override, but there's also this interesting text:

Not all file systems can record creation and last access times and not all file systems record them in the same manner. For example, on FAT, create time has a resolution of 10 milliseconds, write time has a resolution of 2 seconds, and access time has a resolution of 1 day (really, the access date). Therefore, theGetFileTime function may not return the same file time information set using the SetFileTime function.

NTFS delays updates to the last access time for a file by up to one hour after the last access. NTFS also permits last access time updates to be disabled. Last access time is not updated on NTFS volumes by default.

In the current situation, happiness is expected when file properties are retrieved without "reopening" the handle.

Apparently, no luck, at least not when it comes to seconds.

[Yedelkin]

TheXpert:

Yes, here's the verbatim:

Loose translation -- last access includes the last time the file was read or written or executed (if executable).

I must be wrong about the handle override, but there's also this interesting text:

Apparently there's no such luck, at least not when it comes to seconds.

Thanks for the broadening of the mind! Yeah... Bummer.

This script says that instead of "Last read date" identifier FILE_ACCESS_DATE returns the time of last file closing:

int handle_file;
void OnStart()
  {
   Print("===============================================");
   handle_file=FileOpen("Ye_file2.bin",FILE_READ|FILE_WRITE|FILE_BIN);
   switch(handle_file)
     {
      case  INVALID_HANDLE: break;
      default:
         Print("Дата создания файла Ye_file2.bin: ",(datetime)FileGetInteger(handle_file,FILE_CREATE_DATE));
         for(int i=0;i<3;i++)
           {
            Sleep(2000);
            FileReadInteger(handle_file,CHAR_VALUE);
            Print("Дата последнего чтения Ye_file2.bin: ",(datetime)FileGetInteger(handle_file,FILE_ACCESS_DATE));
           }
         Sleep(3000);
         Print("Время обращения к FileClose(handle_file): ",TimeTradeServer());
         FileClose(handle_file);
     }
  }

[Yedelkin]

TheXpert:

But despite the wild conclusions, the tests are revealing, so we await the developers' comments on how the function works when there is no change.

By the way, in the past three months I haven't received any comments, so I avoid usingFileFlush() for the time being.

[Михаил Янович]

It seems to me that FileFlush should not slow down/accelerate the program if it is used instead of FileClose.

It just does not make sense to use it in loop for each record. Imagine how slow Word would be if it re-saved every time a document was modified (especially if that document has lots of text and pictures). FileFLush only facilitates saving without closing files.

The developers must have meant that (For example):

OnInit - FileOpen

On Tick - FileWrite FileFlush

(and here data is saved -FileFlash - in the loop, not every pass of the loop, but after the loop is finished)

OnDeinit FileClose.

The point of FileFlash is to write into it all the time the Expert Advisor is running, so you don't constantly reinitialize the file handle and the mystical file buffer.

......niverse)

[TheXpert]

Yedelkin:
By the way, I haven't received any comments in the past three months, so I've avoided using FileFlush() for the time being.

It does not guarantee saving immediately anyway. So it only makes sense to use it to save the current state without closing the handle.

[is_vale]

Yedelkin:

By the way, I haven't received any comments in the past three months, so I avoid using FileFlush() for the time being.

To better understand the meaning of using FileFlush() function, we need to think about the concept of file input/output buffer... As you know disk information is stored in bytes and writing each byte individually (as they arrive) is the height of waste! If you look at this process from the "hardware" side, then at each request for a byte-write operation, the operating system would have to "wiggle the writer's head" of the hard disk drive! And a hard drive is kinematic! In other words, it is the slowest of all computer devices. So what's the solution then? A very simple solution! In computer memory is created a buffer of data (usually it is some tens kilobytes) in which data from function FileWrite and similar to it is sent. As soon as this buffer will be completely filled up, system writes it completely on a hard disk as the continuous block of data, thus unnecessary drive head manipulation is not performed, and data are simply written in ONE MACHINE! Now calculate how many times the speed of writing 32 kilobytes of information at a time increases compared to writing each byte separately of the same size ;) And the calculation is quite simple... For every write operation the drive head is positioned first, then the write operation is performed. And that's without getting into all the hard work of a read/write operation :) In case of a buffer, we position the drive head once and write the whole block in one stream!

But all that said, until your buffer is full, your data won't physically appear in the file (!!!), or until you close the file itself, in which case the buffer (or rather what's left in it) is written to the disk immediately. This is where the FileFlush function comes in. When you "wrote" some information to the file and you need it to appear in the file (for example, this information can be already used by another program, indicator, Expert Advisor...), then you can call the FileFlush function, which will physically dump the buffer's content to the disk (to a file)!

Conclusion: Frequent use of FileFlush function or its use in loops, to speed up work with the file, will give just the opposite result, because at each call, the system needs to calculate the amount of information really contained in the I/O buffer, call the operating system and give the command to write this memory area to the file! For example, the loop writes one byte to the file and immediately calls the FileFlush function, we get the slowest way to write to disk BACKGROUND!!! ;)

So how is the fastest way to write to a file? Very simple ;) Don't mess around and torture FileFlush :) In this case you get fast data exchange between FileWrite and clipboard (memory manipulation is the fastest). The system will watch for overflow in this buffer and, if necessary, stream data (the fastest operation with such a clumsy device like a hard drive!) and clear the buffer to receive new data!!!

Then the question arises - "Why do you need the FileFlush function???". And the answer is simple - it is needed when you need the data to be physically written to the file, regardless of buffer filling. I gave an example of such a necessity above.

[is_vale]

I had to rewrite my indicator for MQL5 and encountered some serious confusion :(

Here is the code:

  first_bar=ChartGetInteger(0,CHART_FIRST_VISIBLE_BAR,0);
  
  //--- установим доступ к массиву как к таймсерии
  ResetLastError();
  int copied=CopyTime(NULL,0,0,first_bar,TimeAsSeries);
  del();
  int handle=FileOpen("Price Label\\"+_Symbol+tpl_ext,FILE_READ|FILE_CSV,';',CP_ACP);
  int er=GetLastError();
  ResetLastError();
  
  string sTF="";
  int TF;
  string period_name;
  string price_label;
  string price1;
  string price2;
  
  if (handle>=1){
    while(FileIsEnding(handle)==false){
      sTF = FileReadString(handle);
      TF = ResolveTF(sTF);
      period_name=FileReadString(handle);
      price_label=FileReadString(handle);
      price1=FileReadString(handle);
      price2=FileReadString(handle);
      drawe_price(TF,handle,period_name,price_label,price1,price2);
    }
    FileClose(handle);
  }
  return(0);

I want to explain right away that I entered such a large number of variables to see what's going on in the debugger. And I saw...

The file contains a bunch of strings, each string has 5 sections divided by ";". The first call

sTF = FileReadString(handle);

puts the entire file into the sTF variable, in an incomprehensible encoding. Judging by what I see in debugger (in sTF variable it reads file contents as unicode! When opening the file, I tried all the available code pages, but the result is the same :( The file itself is written in Windows encoding.

Does anyone have any idea where the dog is buried?

[Yedelkin]

is_vale

Thank you all for sharing the information! I will dive into the topic anew.

[TheXpert]

FILE_ANSI

[Yedelkin]

is_vale:

Does anyone have any idea where the problem lies?

I haven't worked with file operations for a long time... Look, when using FileOpen() you have a CSV file declared. It used to specify that all written items are converted to unicode or ansi strings. Maybe this is where the dog is?