Finding a set of indicators to feed into the neural network inputs. Discussion. A tool for evaluating the results. - page 6

 

joo писал(а) >>

rip and IlyaA don't seem to understand that iliarr is using a teaching method without a teacher. What kind of learning error can we talk about, if the target function is profit? Or do you both think that having trained the network on history, you will run it on a test history and compare the obtained profit? The profit will be different, less or more, but different. The test history is different. Do not confuse with approximation please, where the criterion for the quality of approximation is the standard deviation of the original function and the obtained one.

What does MSE have to do with it?! The estimate of correctness of the network according to the author's idea is the profit function. I.e. longshort positions are opened and the trade.Profit is calculated relative to them or using them - this can be seen in the code. Ok, then the question is simply to see how an individual behaves in a test sample, which is considered the best in a given generation. Why do I think this is important, profit is counted relative to the training sample. And who's to say it will behave the same on samples that were not presented to the network.


No matter by what method the network is trained, with or without a teacher, a test sample with a known result allows us to estimate the degree of overtraining.

Or otherwise we go back to 'shamanism' - something was given to the network, something was received, and now let's interpret the result.

 
What does MSE have to do with it?! The estimation of the correctness of the network according to the author's idea is the profit function. I.e. long/short positions are opened and trade.Profit is calculated with respect to them - this can be seen from the code. Ok, then the question is simply to see how an individual behaves in a test sample, which is considered the best in a given generation. Why do I think this is important, profit is counted relative to the training sample. And who says it will behave the same on samples that were not presented to the network.

The f-from profit cannot be used as an estimation of correctness of work of a network. The Profit-Factor only describes a network from the viewpoint of an Expert Advisor's ability to earn the highest possible profit. We cannot say any more about the correctness of a network. Just think, for a certain period of time it is possible to gain the same amount of profit in many different ways. Every trader is mostly concerned with which way. For example, the maximum relative drawdown or other evaluation criteria of the TS can be used to evaluate the network performance. I.e. the author must maximize the f-from profit on the training sample, test it on a test sample and estimate it using the parameters described above. Or you optimizing a standard Makdi Expert Advisor, for example, will you also select the variant with the maximum profit?

Regardless of the method by which the network is trained, with or without a teacher, a test sample with a known result allows us to assess the degree of overtraining.

Or otherwise we come back to "shamanism" - we gave something to the network, we got something, and now let's interpret the result.

How will you get a known result on a test sample if the profit function is used?

 
Dear joo, I ask you not to discuss other people's behaviour or thoughts, especially if the person is unfamiliar to you. I also ask you to answer the following question. Do you think that a network will adapt to data if it is trained by genetics. Adapt and not generalise?
 
IlyaA >> :
Dear joo, I ask you not to discuss a behaviour or thoughts of other people, especially if the person is not familiar to you....

I apologise if it seemed like I was discussing someone else's behaviour or other people's thoughts, I didn't think it was personal. Again, I'm sorry if I did.

I also ask you to answer this question. Do you think that a network will adapt to data if it is trained with genetics? Adapt and not generalize?

The ability of a neural network to adapt to anything is not dependent on genetics. Genetics is just one of the methods of optimization. The key is in the data presented, in the topology of the network, in the methods of estimating the result.

 
joo >> :

The ability of a neural network to adapt to anything is not dependent on genetics. Genetics is just one of the methods of optimization. The key point is in the data presented, in topology of a network, in methods of estimation of result.


Well, I'm glad it's so friendly. Accordingly, the logic is that if there is no learning error curve then there is a learning efficiency curve. In that case, the public needs to look at it. Agreed.
 
IlyaA >> :
Accordingly, the logic is that if there is no learning error curve then there is a learning efficiency curve. In that case, the public needs to look at it. >> Agreed.

My logic goes like this:

By analogy with the animal world. There is a deer and a wolf living in a forest. Both weigh 80kg. A deer nibbles on grass 24 hours a day, while a wolf eats half of an elk and does not eat for a fortnight. Their average caloric intake is the same. But the way they get their food is different.

So does TC. We must choose how the profit will be obtained, and accordingly choose the criteria for evaluating TC, and in the context of this branch - NN.

 
joo >> :

My logic goes like this:

By analogy with the animal world. There is a deer and a wolf living in the forest. Both of them weigh 80 kilograms. A deer nibbles on grass around the clock, while a wolf eats half of an elk and does not eat for a fortnight. Their average caloric intake is the same. But the way they get their food is different.

So does TC. We have to choose how to get the profit, and accordingly choose the criteria for evaluation of TC, and in the context of this branch - NN.


You are assessing the performance of the network from the point of view of the TC. I am looking at the network itself, without reference to the TC. The network is only a mechanism for analysing data and nothing more.


But using your allegory, I want to see a graph as a deer and wolf gain weight separately (this training) in a zoo, and also a graph as a deer and wolf will gain weight, if we say they were released into nature (testing), well, in a national park, where they will be watched by rangers. On this basis it would be possible to build a hypothesis how they will behave in nature %)

 
rip >> :

You are assessing the performance of the network from the point of view of the TC. I am looking at the network itself, without reference to the TC. The network is only a mechanism of data analysis and no more.


But applying your allegory, I would like to see a graph of how the deer and wolf gain weight separately (this is a training) in a zoo, and also a graph of how the same deer and wolf will gain weight, if we say they are released into nature (testing), well, in a national park, where they will be watched by gamekeepers. On this basis we can make a hypothesis how they will behave in the nature %)


not much is wrong. We need to estimate somehow that if we raised a fawn, we ended up with a handsome stag and not a wolf. That is, to assess not the rate and volume of food intake, but the belonging to the species we want. Perhaps to carry out a classification, in essence, to determine the "similarity" of the consumption curve to the reference one. And this is a separate, difficult task.

 

rip 10.11.2009 23:18



Regardless of the method used to train the network, with or without a teacher, a test sample with a known result allows you to assess the degree of overtraining.

Otherwise we go back to "shamanism": we gave something to the network, we received something, and now let's interpret the result.

A perfectly legitimate observation. For this, I unload a trained neuronet into an MT4 trading Expert Advisor to check in the MT4 strategy tester what I got.

The quote is from the very first post of the topic:

iliarr 09.11.2009 13:13


I export the trained network to MQL4 Expert Advisor and check its functionality in the Strategy Tester of MT4. I will form inputs for neural network in MT4 indicator and export them to a file.

And ready to post these uploads here, in order to evaluate the quality of the data fed into the network. I propose to choose the M5 range from 1-08-2009 to 1-10-2009 and teach similar neural networks on it in the same way. This method does not pretend to be absolutely accurate, but I think it will do for comparison.

 
rip >> :

You are assessing the performance of the network from the point of view of the TC. I am looking at the network itself, without reference to the TC. The network is only a mechanism of data analysis and no more.


But applying your allegory, I would like to see a graph of how the deer and wolf gain weight separately (this is a training) in a zoo, and also a graph of how the same deer and wolf will gain weight, if we say they are released into nature (testing), well, in a national park, where they will be watched by gamekeepers. On this basis it will be possible to make a hypothesis how they will behave in the nature %)


During training a log is kept like this:

Traning log.
Thread-6: Generation 10 chromosome whith best fitness: 1007621 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-5: Generation 10 chromosome whith best fitness: 1008875 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-3: Generation 10 chromosome whith best fitness: 1009096 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-6: Generation 20 chromosome whith best fitness: 1009461 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-5: Generation 30 chromosome whith best fitness: 1009501 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-3: Generation 40 chromosome whith best fitness: 1010195 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-3: Generation 100 chromosome whith best fitness: 1010361 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-2: Generation 110 chromosome whith best fitness: 1010481 saved to /home/iliarr/Data/Result/9-10-1.nn
Thread-2: Generation 200 chromosome whith best fitness: 1010521 saved to /home/iliarr/Data/Result/9-10-1.nn

If the target function is larger than the current one, the neural network is unloaded into a file and the next line of the log is written. I don't think this is the answer to your question, but I don't see the point in keeping other statistics.