Discussing the article: "Neural Networks Made Easy (Part 86): U-Shaped Transformer"
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
Check out the new article: Neural Networks Made Easy (Part 86): U-Shaped Transformer.
We continue to study timeseries forecasting algorithms. In this article, we will discuss another method: the U-shaped Transformer.
Forecasting long-term timeseries is of specifically great importance for trading. The Transformer architecture, which was introduced in 2017, has demonstrated impressive performance in the areas of Natural Language Processing (NLP) and Computer Vision (CV). The use of Self-Attention mechanisms allows the effective capturing of dependencies over long time intervals, extracting key information from the context. Naturally, quite quickly a large number of different algorithms based on this mechanism were proposed for solving problems related to timeseries.
However, recent studies have shown that simple Multilayer Perceptron Networks (MLP) can surpass the accuracy of Transformer-based models on different timeseries datasets. Nevertheless, the Transformer architecture has proven its effectiveness in several areas and even found practical application. Therefore, its representative ability should be relatively strong. There must be mechanisms for its use. One of the options for improving the vanilla Transformer algorithm is the paper "U-shaped Transformer: Retain High Frequency Context in Time Series Analysis", which presents the U-shaped Transformer algorithm.
Through iterative learning, I managed to obtain a model capable of generating profit on both the training and testing datasets.
During the testing period, the model performed 26 traders, 20 of which, i.e. 76.92%, were closed with a profit. The profit factor was 2.87.
The results obtained are promising, but the testing period of 1 month is too short to reliably assess the stability of the model.
Author: Dmitriy Gizlyk