Better NN EA development - page 85

New comment
 

Neural networks made easy (Part 37): Sparse Attention 

Neural networks made easy (Part 37): Sparse Attention

In the previous article, we discussed relational models which use attention mechanisms in their architecture. We used this model to create an Expert Advisor, and the resulting EA showed good results. However, we noticed that the model's learning rate was lower compared to our earlier experiments. This is due to the fact that the transformer block used in the model is a rather complex architectural solution performing a large number of operations. The number of these operations grows in a quadratic progression as the size of the analyzed sequence increases, leading to an increase in memory consumption and model training time.

However, we recognize the limited resources available to improve the model. Therefore, there is a need to optimize the model with minimal loss of quality.
Neural networks made easy (Part 37): Sparse Attention
Neural networks made easy (Part 37): Sparse Attention
  • www.mql5.com
In the previous article, we discussed relational models which use attention mechanisms in their architecture. One of the specific features of these models is the intensive utilization of computing resources. In this article, we will consider one of the mechanisms for reducing the number of computational operations inside the Self-Attention block. This will increase the general performance of the model.
 

Neural networks made easy (Part 38): Self-Supervised Exploration via Disagreement

Neural networks made easy (Part 38): Self-Supervised Exploration via Disagreement

This algorithm is based on a self-learning method, where the agent uses information obtained during interaction with the environment to generate "intrinsic" rewards and update its strategy. The algorithm is based on the use of several agent models that interact with the environment and generate various predictions. If the models disagree, it is considered an "interesting" event and the agent is incentivized to explore that space of the environment. In this way, the algorithm incentivizes the agent to explore new areas of the environment and allows it to make more accurate predictions about future rewards.
Neural networks made easy (Part 38): Self-Supervised Exploration via Disagreement
Neural networks made easy (Part 38): Self-Supervised Exploration via Disagreement
  • www.mql5.com
One of the key problems within reinforcement learning is environmental exploration. Previously, we have already seen the research method based on Intrinsic Curiosity. Today I propose to look at another algorithm: Exploration via Disagreement.
 
Sergey Golubev #:

Neural networks made easy (Part 38): Self-Supervised Exploration via Disagreement

This algorithm is based on a self-learning method, where the agent uses information obtained during interaction with the environment to generate "intrinsic" rewards and update its strategy. The algorithm is based on the use of several agent models that interact with the environment and generate various predictions. If the models disagree, it is considered an "interesting" event and the agent is incentivized to explore that space of the environment. In this way, the algorithm incentivizes the agent to explore new areas of the environment and allows it to make more accurate predictions about future rewards.
Impressive
 

Neural networks made easy (Part 39): Go-Explore, a different approach to exploration

Neural networks made easy (Part 39): Go-Explore, a different approach to exploration

We continue the theme of environmental exploration in reinforcement learning. In previous articles within this series, we have already looked at algorithms for exploring the environment through curiosity and disagreement in an ensemble of models. Both approaches exploited intrinsic rewards to motivate the agent to perform different actions in similar situations while exploring new areas. But the problem is that the intrinsic reward decreases as the environment gets better explored. In complex cases of rare rewards, or when the agent may receive penalties on the way to the reward, this approach may not be very effective. In this article, I propose to get acquainted with a slightly different approach to studying the environment – the Go-Explore algorithm.
Neural networks made easy (Part 39): Go-Explore, a different approach to exploration
Neural networks made easy (Part 39): Go-Explore, a different approach to exploration
  • www.mql5.com
We continue studying the environment in reinforcement learning models. And in this article we will look at another algorithm – Go-Explore, which allows you to effectively explore the environment at the model training stage.
 

Neural networks made easy (Part 40): Using Go-Explore on large amounts of data

Neural networks made easy (Part 40): Using Go-Explore on large amounts of data

In the previous article "Neural networks made easy (Part 39): Go-Explore, a different approach to exploration", we familiarized ourselves with the Go-Explore algorithm and its ability to explore the environment.

In this article, we will take a closer look at possible optimization methods for the Go-Explore algorithm to improve its efficiency over longer training periods.

Neural networks made easy (Part 40): Using Go-Explore on large amounts of data
Neural networks made easy (Part 40): Using Go-Explore on large amounts of data
  • www.mql5.com
This article discusses the use of the Go-Explore algorithm over a long training period, since the random action selection strategy may not lead to a profitable pass as training time increases.
 

Neural networks made easy (Part 41): Hierarchical models 

In this article, we will explore the application of hierarchical reinforcement learning in trading. We propose using this approach to create a hierarchical trading model that will be able to make optimal decisions at different levels and adapt to different market conditions.

In this article, we will consider the architecture of the hierarchical model, including various levels of decision making, such as determining entry and exit points for trades. We also present hierarchical model learning methods that combine global-level reinforcement learning and local-level reinforcement learning.

The use of hierarchical learning makes it possible to model complex decision-making structures, as well as effectively use knowledge at different levels. This helps to increase the generalizing ability of the model and its adaptability to changing market conditions.

Neural networks made easy (Part 41): Hierarchical models
Neural networks made easy (Part 41): Hierarchical models
  • www.mql5.com
The article describes hierarchical training models that offer an effective approach to solving complex machine learning problems. Hierarchical models consist of several levels, each of which is responsible for different aspects of the task.
 

Neural networks made easy (Part 42): Model procrastination, reasons and solutions 

Neural networks made easy (Part 42): Model procrastination, reasons and solutions

In the field of reinforcement learning, neural network models often face the problem of procrastination when the learning process slows down or gets stuck. Model procrastination can have serious consequences for achieving goals and requires taking appropriate measures. In this article, we will look at the main reasons for model procrastination and propose methods for solving them.
Neural networks made easy (Part 42): Model procrastination, reasons and solutions
Neural networks made easy (Part 42): Model procrastination, reasons and solutions
  • www.mql5.com
In the context of reinforcement learning, model procrastination can be caused by several reasons. The article considers some of the possible causes of model procrastination and methods for overcoming them.
 

Neural networks made easy (Part 43): Mastering skills without the reward function 

Neural networks made easy (Part 43): Mastering skills without the reward function

In this article, we introduce the concept of "Diversity is All You Need", which allows you to teach a model a skill without an explicit reward function. Variety of actions, exploration of the environment, and maximizing the variability of interactions with the environment are key factors for training an agent to behave effectively.

Neural networks made easy (Part 43): Mastering skills without the reward function
Neural networks made easy (Part 43): Mastering skills without the reward function
  • www.mql5.com
The problem of reinforcement learning lies in the need to define a reward function. It can be complex or difficult to formalize. To address this problem, activity-based and environment-based approaches are being explored to learn skills without an explicit reward function.
 

Neural networks made easy (Part 44): Learning skills with dynamics in mind

Neural networks made easy (Part 44): Learning skills with dynamics in mind

In the previous article, we got acquainted with the DIAYN method, which allows you to train separable skills. This makes it possible to build a model that can change the agent behavior depending on the current state.

In this paradigm, a question arises of learning skills whose behavior would be easily predictable. At the same time, we are not ready to sacrifice the diversity of their behavior. A similar problem is solved by the authors of the Dynamics-Aware Discovery of Skills (DADS) method presented in 2020. Unlike DIAYN, the DADS method seeks to teach skills that not only have variety in behavior, but are also predictable. 
Neural networks made easy (Part 44): Learning skills with dynamics in mind
Neural networks made easy (Part 44): Learning skills with dynamics in mind
  • www.mql5.com
In the previous article, we introduced the DIAYN method, which offers the algorithm for learning a variety of skills. The acquired skills can be used for various tasks. But such skills can be quite unpredictable, which can make them difficult to use. In this article, we will look at an algorithm for learning predictable skills.
 

Neural networks made easy (Part 45): Training state exploration skills

Neural networks made easy (Part 45): Training state exploration skills

In this article, I propose to get acquainted with the alternative method of teaching skills Explore, Discover and Learn (EDL). EDL approaches the problem from a different angle, which allows it to overcome the problem of limited state coverage and offer more flexible and adaptive agent behavior.

Neural networks made easy (Part 45): Training state exploration skills
Neural networks made easy (Part 45): Training state exploration skills
  • www.mql5.com
Training useful skills without an explicit reward function is one of the main challenges in hierarchical reinforcement learning. Previously, we already got acquainted with two algorithms for solving this problem. But the question of the completeness of environmental research remains open. This article demonstrates a different approach to skill training, the use of which directly depends on the current state of the system.
1...78798081828384858687
New comment