Libro Reti NeuraliSezioni
Neural Networks for Algorithmic Trading with MQL5Architectural solutions for improving model convergenceDropoutBuilding Dropout in MQL56.2.1.3 File operations 
Scarica in un unico file:
Libro Reti Neurali in PDF
Libro Reti Neurali in CHM

6.File operations

In the previous sections, we explored the operating algorithm of the Dropout approach and even managed to create the CNeuronDropout class to implement it within our library. Within the framework of this class, we have implemented the Dropout feed-forward and backpropagation algorithms. Now, for the full implementation of this class, we need to add file methods that will allow us to save and restore the operation of a previously trained model at any required time. This provides the opportunity to quickly restore the functionality of the model when needed.

Again, when starting this work, we critically evaluate the variables and objects of our class to decide whether to save them to a file in whole or in part or to restore them according to some parameters.

class CNeuronDropout    :  public CNeuronBase
  {
protected:
   TYPE              m_dOutProbability;
   int               m_iOutNumber;
   TYPE              m_dInitValue;
   CBufferType       m_cDropOutMultiplier;
 
public:
                     CNeuronDropout(void);
                    ~CNeuronDropout(void);
   //---
   virtual bool      Init(const CLayerDescription *desc) override;
   virtual bool      FeedForward(CNeuronBase *prevLayer) override;
   virtual bool      CalcHiddenGradient(CNeuronBase *prevLayer) override;
   virtual bool      CalcDeltaWeights(CNeuronBase *prevLayer, bool read)
                                                       override { return true; }
   virtual bool      UpdateWeights(int batch_size, TYPE learningRate,
                         VECTOR &Beta, VECTOR &Lambda) override { return true; }
   //--- methods for working with files
   virtual bool      Save(const int file_handle) override;
   virtual bool      Load(const int file_handle) override;
   //--- object identification method
   virtual int       Type(void) override     const { return(defNeuronDropout); }
  };

In addition to the objects inherited from the parent class, we create only one data buffer and three variables. These three variables have mathematical relationships between them. The masking vector buffer is redefined on each feed-forward pass. Thus, to restore the functionality of the Dropout layer, it is sufficient to save the objects of the parent class and one variable.

Therefore, the data-saving method will be quite simple and short. In parameters, the method receives a pointer to a file handle for saving. In the method body, we call a similar method from the parent class, in which all the controls and the saving of parent class objects are already implemented. After the successful execution of the parent class method, we will only write the dropout probability to the file, which represents the probability of dropping out neurons from processing. This particular variable was chosen because it is the parameter specified by the user, while the others are secondary and are calculated during class initialization.

bool CNeuronDropout::Save(const int file_handle)
  {
//--- call the method of the parent class
   if(!CNeuronBase::Save(file_handle))
      return false;
//--- save the probability constant of dropping out elements
   if(FileWriteDouble(file_handle, m_dOutProbability) <= 0)
      return false;
//---
   return true;
  }

The method for restoring the functionality of the CNeuronDropout::Load layer looks a little more complicated than the saving method. Just like the data-saving method, the data-loading method receives a file handle with data to load in its parameters. We remember the fundamental rule of data loading: data is loaded from the file in strict accordance with the sequence in which it was written. Therefore, in the method body, we first call a similar method from the parent class, where all the controls and loading of data inherited from the parent class objects are already implemented.

bool CNeuronDropout::Load(const int file_handle)
  {
//--- call the method of the parent class
   if(!CNeuronBase::Load(file_handle))
      return false;

We must always check the result of the parent class method execution because it confirms not only the data loading but also the passing of all implemented controls.

After the successful execution of the parent class method, we read the probability of dropping out neurons from the file. Based on the obtained value, we calculate the number of neurons to be dropped out on each feed-forward iteration and initialize the values of the masking buffer elements.

//--- read and restore constants
   m_dOutProbability = (TYPE)FileReadDouble(file_handle);
   m_iOutNumber = (int)(m_cOutputs.Total() * m_dOutProbability);
   m_dInitValue = (TYPE)(1.0 / (1.0 - m_dOutProbability));

Finally, at the end of the method for restoring the functionality of our layer, we initialize the buffer for recording the masking vector.

//--- initializing the data masking buffer
   if(!m_cDropOutMultiplier.BufferInit(m_cOutputs.Rows(), m_cOutputs.Cols(),
                                                              m_dInitValue))
      return false;
//---
   return true;
  }

After successfully loading data and initializing objects in our layer, we exit the method with a positive result.

At this stage, we are completing work on the Dropout layer class using standard MQL5 tools. In the next section, we will look at implementing a multi-threaded algorithm using OpenCL.