Metatrader 5 with Python and Tensorflow - new TF version needs to run on WSL2 subsystem

 

Hi,

Tensorflow recently announced that going forward with TF 2.11+ they won't include native GPU support in their windows versions anymore. TF 2.10 was the last version that allowed running TF with GPU on Windows. 

From TF 2.11 on, Windows users need to use WSL2, for example in Visual Studio Code or PyCharm.

Is there a manual about how we can use Metatrader5 Python package via WSL2 so that everything still works on the same Windows machine between Metatrader 5 and Tensorflow 2.11+? What I want is to use GPU support via WSL2 but at the same time open and read out data from Metatrader5 in Windows to train the models so that it still works with newer Tensorflow versions.

Documentation on MQL5: Python Integration / login
Documentation on MQL5: Python Integration / login
  • www.mql5.com
login - Python Integration - MQL5 Reference - Reference on algorithmic/automated trading language for MetaTrader 5
 
brunowillisTensorflow recently announced that going forward with TF 2.11+ they won't include native GPU support in their windows versions anymore. TF 2.10 was the last version that allowed running TF with GPU on Windows. From TF 2.11 on, Windows users need to use WSL2, for example in Visual Studio Code or PyCharm. Is there a manual about how we can use Metatrader5 Python package via WSL2 so that everything still works on the same Windows machine between Metatrader 5 and Tensorflow 2.11+? What I want is to use GPU support via WSL2 but at the same time open and read out data from Metatrader5 in Windows to train the models so that it still works with newer Tensorflow versions.

I don't know the answer and probably many users here will not know either, just like you.

So, why not just test and experiment with it and see what happens?

 

That takes time, and that's why I preferred to ask before.


Fernando Carreiro #:

I don't know the answer and probably many users here will not know either, just like you.

So, why not just test and experiment with it and see what happens?

 

Did anyone manage to successfully use the MT5 Python Package from wsl2 ?

 
brunowillis:

Hi,

Tensorflow recently announced that going forward with TF 2.11+ they won't include native GPU support in their windows versions anymore. TF 2.10 was the last version that allowed running TF with GPU on Windows. 

From TF 2.11 on, Windows users need to use WSL2, for example in Visual Studio Code or PyCharm.

Is there a manual about how we can use Metatrader5 Python package via WSL2 so that everything still works on the same Windows machine between Metatrader 5 and Tensorflow 2.11+? What I want is to use GPU support via WSL2 but at the same time open and read out data from Metatrader5 in Windows to train the models so that it still works with newer Tensorflow versions.

Follow these steps


Set up WSL2 with GPU support: Enable WSL2 on your Windows machine and configure it to utilize GPU support. This involves installing NVIDIA GPU drivers within WSL2 and setting up CUDA and cuDNN libraries. NVIDIA provides documentation on how to do this specifically for WSL2.


Install Metatrader5 on Windows: Install and set up Metatrader5 on your Windows machine as usual. Ensure that it's running and accessible.


Install Metatrader5 Python package on WSL2: Within your WSL2 environment, install the Metatrader5 Python package using pip .


Establish communication between Windows and WSL2: You need a way for your WSL2 environment to communicate with Metatrader5 running on Windows. One approach is to use TCP/IP sockets for communication. You can have Metatrader5 act as a server and your Python script in WSL2 act as a client to exchange data.


Develop TensorFlow models on WSL2: Write your TensorFlow code within your WSL2 environment, utilizing GPU support for training your models.


Data transfer between Windows and WSL2: To transfer data between Windows and WSL2, you can set up shared folders or use other methods supported by WSL2. For instance, you could save data generated by Metatrader5 on a shared folder accessible by both Windows and WSL2, allowing your TensorFlow code to access it for training.


Integration and Testing: Integrate the data obtained from Metatrader5 into your TensorFlow workflow. Test thoroughly to ensure that data exchange between Windows and WSL2 works seamlessly, and that your TensorFlow models train effectively using the data.


Automation and Monitoring: Once everything is set up and working as expected, consider automating the process of data retrieval from Metatrader5 and model training. You may also want to implement monitoring mechanisms to track the performance of your models over time.

You can utilize GPU support via WSL2 for TensorFlow model training while still accessing and utilizing data from Metatrader5 running on Windows.

 

Thank you for your answer, Nardus.

The third step has some issues, because installing the MetaTrader5 package on WSL2 requires installing Wine first to emulate Windows, and Wine emulates Windows 7, which is not supported by the most recent Python versions, required by the most recent TensorFlow versions.

However, it seems this step is not really necessary because, since there will be two separate instances communicating through the network, the Windows instance could easily route all necessary MetaTrader5 information to/from the WSL2 instance.

The solution of two instances exchanging data through shared folders (for batch processing tasks like training models) and through sockets (for real-time data streaming to/from the trained model during trading) seems to be a suitable solution.

Does anyone have a better (or simpler) suggestion?