Discussion of article "Neural networks made easy (Part 11): A take on GPT"
New article Neural networks made easy (Part 11): A take on GPT has been published:
Author: Dmitriy Gizlyk
Interesting material but the code is all in one huge spaghetti bowl pretty hard to get to grips with it. One class per file would have been easier.
Would be good to have a diagram showing how all the classes are held together.
Right now I'm at NeuronBase which is derived from Object but uses NeuronProof which is derived from NeuronBase and also Layer which uses practically every type of Neuron class defined.
Plenty of forward declarations required in order to put everything in separate files and get to grips with it.
Probably easier to start clean and use the concepts you have explained in the various chapters.
I tried running the spaghetti bowl it made my GPU smoke, producing nothing very useful unfortunately.
Anyhow thanks for the great materials.
:-)
Interesting material but the code is all in one huge spaghetti bowl pretty hard to get to grips with it. One class per file would have been easier.
Would be good to have a diagram showing how all the classes are held together.
Right now I'm at NeuronBase which is derived from Object but uses NeuronProof which is derived from NeuronBase and also Layer which uses practically every type of Neuron class defined.
Plenty of forward declarations required in order to put everything in separate files and get to grips with it.
Probably easier to start clean and use the concepts you have explained in the various chapters.
I tried running the spaghetti bowl it made my GPU smoke, producing nothing very useful unfortunately.
Anyhow thanks for the great materials.
:-)
Hi,
Check NN.chm. Maybe it will help you.
Hi,
Yes it did thank you.
The code isn't as bad now and I can see clearly how to change the net architecture if necessary.
I have everything in separate class files, plus a much smaller spaghetti bowl which is still necessary because mt5 has both definition and implementations in the same files.
I tried doing some training on GBPUSD using the code as is but results don't appear to be good the error climbs from the natural 50% to 70% and stays there with no improvement over subsequent epochs.
Any suggestions where to start tweaking?
BTW this line:
#define FileName Symb.Name()+"_"+EnumToString((ENUM_TIMEFRAMES)Period())+"_"+IntegerToString(HistoryBars,3)+StringSubstr(__FILE__,0,StringFind(__FILE__,".",0))
Need to be corrected to:
#define FileName Symb.Name()+"_"+EnumToString((ENUM_TIMEFRAMES)Period())+"_"+IntegerToString(HistoryBars)+StringSubstr(__FILE__,0,StringFind(__FILE__,".",0))
Hi again :) I found the problem. On in include folder the Opencl config was fail, because i tested my system, and GPU dont dupport the code only the cpu, a little change and working perfectly, but slowwly :S
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
You agree to website policy and terms of use
New article Neural networks made easy (Part 11): A take on GPT has been published:
Perhaps one of the most advanced models among currently existing language neural networks is GPT-3, the maximal variant of which contains 175 billion parameters. Of course, we are not going to create such a monster on our home PCs. However, we can view which architectural solutions can be used in our work and how we can benefit from them.
The new class of the neural network was tested on the same data set, which was used in previous tests: EURUSD with the H1 timeframe, historical data of the last 20 candlesticks is fed into the neural network.
The test results have confirmed the assumption that more parameters require a longer training period. In the first training epochs, an Expert Advisor with fewer parameters shows more stable results. However, as the training period is extended, an Expert Advisor with a large number of parameters shows better values. In general, after 33 epochs the error of Fractal_OCL_AttentionMLMH_v2 decreased below the error level of the Fractal_OCL_AttentionMLMH EA, and it further remained low.
Author: Dmitriy Gizlyk