Discussing the article: "Neural networks made easy (Part 71): Goal-Conditioned Predictive Coding GCPC)"
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Check out the new article: Neural networks made easy (Part 71): Goal-Conditioned Predictive Coding GCPC).
In previous articles, we discussed the Decision Transformer method and several algorithms derived from it. We experimented with different goal setting methods. During the experiments, we worked with various ways of setting goals. However, the model's study of the earlier passed trajectory always remained outside our attention. In this article. I want to introduce you to a method that fills this gap.
Goal-Conditioned Behavior Cloning (BC) is a promising approach for solving various offline reinforcement learning problems. Instead of assessing the value of states and actions, BC directly trains the Agent behavior policy, building dependencies between the set goal, the analyzed environment state and the Agent's action. This is achieved using supervised learning methods on pre-collected offline trajectories. The familiar Decision Transformer method and its derivative algorithms have demonstrated the effectiveness of sequence modeling for offline reinforcement learning.
Previously, when using the above algorithms, we experimented with various options for setting goals to stimulate the Agent actions we needed. However, how the model learns the previously passed trajectory remained outside our attention. Now, the question arises about the applicability of studying the trajectory as a whole. This question was addressed by the authors of the paper "Goal-Conditioned Predictive Coding for Offline Reinforcement Learning". In their paper, they explore several key questions:
Are offline trajectories useful for sequence modeling or do they simply provide more data for supervised policy learning?
What would be the most effective learning goal for trajectory representation to support policy learning? Should sequence models be trained to encode historical experience, future dynamics, or both?
Since the same sequence model can be used for both trajectory representation learning and policy learning, should we have the same learning goals or not?
Author: Dmitriy Gizlyk