Using artificial intelligence at MTS - page 18

 
rip:
njel:
Ladies and gentlemen, what to give as an input? A candle?
Respect to the author ...
Please formulate a network training rule on the forum.

--
rip
Just the network topology is more or less clear from the code. At least the author has arranged the functions and their comments in such a way, that one can understand in general.

I searched further in the code and found out that my charts would not contain enough bars. I increased them up to 16000. I ran it and obtained 6 deals of the whole period with not very large, but a 9% loss on the end result. Few trades - a small drain.

In short, it is clear that nothing is clear at all, but how to use this Expert Advisor after all? There should be some recommendations, right? Maybe I'm doing something wrong?

There is no sense in continuing to dig through the code and try to analyze it by myself. Making claims that the Expert Advisor is either loss-making or profitable by 6 trades is useless. It is just a waste of time.
 

Apparently a little clarification is needed

The Expert Advisor is a drain, or at least it's not profitable. It is posted for those people who want to play with neural networks, but don't want to write everything from scratch.
The network configuration is rather universally written. Actually there are only two limitations - minimum two layers and one output neuron.

Now, as to why the Expert Advisor does not work as it is.
It cannot do so because the task of predicting n ticks by previous m ticks is obviously unsolvable. The grid cannot find dependencies for good prediction. If we try to classify indicators it will fail too, but the grid cannot divide the input data into classes.
This means that it is necessary to process the data for the grid in a more creative way. That is what I am going to do.

These are my conclusions. Upon understanding and analysis of my Expert Advisor, those who wish to do so will either come to the same or, perhaps, to some fresh ideas.

In fact, the Expert Advisor can be considered as a library for working with neural networks, give yourself the labor to understand them and use them.

If someone has got the idea after my previous message that this EA can be played, either you haven't read carefully, or I haven't made myself clear enough.
I paraphrased Figar0: "Not profitable at all, but informative.

Now for some personal answers.

Reshetov And in general, how it should be used correctly and whether it is worth using at all? Because understanding someone else's code is a tedious and thankless task. And if you know the algorithm, it's much easier to write your own program from scratch than to understand someone else's
.

If you mean trade, you probably shouldn't use it. At least I wouldn't. It is only valuable as code.
Actually, I partly agree that you can write your own program from scratch knowing the algorithm. I'm not so sure about simple.
But you are welcome to write algorithms. You can easily find books on neural networks, training principles and other things on the Internet. While I was writing this Expert Advisor, my collection easily reached two dozen books and articles.


Mathemat
I don't think this code has anything in common with Reshetov's Expert Advisor ...

What they both have in common is that they both have neural networks with indicators as inputs :) The difference is that I have not one but many neurons, respectively, I wrote my own algorithm for training.


Please formulate a learning rule for the network in the forum.

If it's for me, please elaborate. I don't really understand what you are interested in.





 
maveric писал (а):


Please formulate a network training rule in the forum.

If it's for me, please elaborate. I don't really understand what you are interested in.


You are well versed in neural networks, why don't you write an article on the subject, especially focusing on learning algorithms for multilayer networks. That would be of interest to a lot of people. Articles here are paid, your work will not go to waste.
 
Integer писал (а):
maveric wrote (a):


Please formulate a network training rule in the forum.

If it's for me, please elaborate. I don't really understand what you are interested in.


You have a good understanding of neural networks, why don't you write an article on the subject, especially focusing on learning algorithms for multilayer networks. That would be of interest to a lot of people. Articles here are paid, your work will not go to waste.
That does not sound like a good idea to me :)
I have learned about neural networks from books and articles written by serious people, not secret ones, but easy to find on the Internet.
I have a fear that it will turn out like in the anecdote when Moisha sang the Beatles to Abram :)
 

Please formulate a network training rule in the forum.

If it's for me, please elaborate. I don't really understand what you are interested in.

You answered my question ;) In this case, you have developed a network that approximates the time series n-steps ahead. You take Close values and/or indicator values as inputs. Ok, then another question. Regarding classification - you said we should try to classify the indicators, may I elaborate on that idea? Yes, I have another remark. Practice has shown that the best results are obtained when training networks (training with a teacher), the training sample of which has a normal distribution. I've done research of time series formed by quotes several times, long time ago ... Its distribution has nothing to do with the normal distribution. Maybe we should analyze the construction of a training sample and reduce it to a sample with a normal distribution. In this case the MSE will decrease much faster and you'll see better results in the test sample.
 
maveric:


Now, as for why the Expert Advisor does not work as it is.
It does not work because the task of predicting n ticks based on previous m ticks is obviously unsolvable. The grid cannot find dependencies for good prediction. If we try to classify indicators it will fail too, but the grid cannot divide the input data into classes.
This means that it is necessary to process the data for the grid in a more creative way. Which is what I am going to do next.


Maybe we could open a new thread, where we could discuss formation and pre/post processing of samples? I think it would be of interest to a number of visitors.
 
rip:

Please formulate a network training rule in the forum.

If it's for me, please elaborate. I don't really understand what you are interested in.

You answered my question ;) In this case, you have developed a network that approximates the time series n-steps ahead. You take Close values and/or indicator values as inputs. Ok, then another question. Regarding classification - you said we should try to classify the indicators, may I elaborate on that idea? Yes, I have another remark. Practice has shown that the best results are obtained when training networks (training with a teacher), the training sample of which has a normal distribution. I've done research of time series formed by quotes several times, long time ago ... Its distribution has nothing to do with the normal distribution. Maybe we should analyze the construction of a training sample and reduce it to a sample with a normal distribution. In this case the MSE will decrease much faster and you'll see better results in the test sample.

Not only a normal distribution. At least the sample sets should be consistent as well.
Generally speaking, it is the following way.
We find the beginning of the first trend ("up down flat") Some number of bars before this beginning will be a grid entry, the whole trend following it will be rolled in the form of 1 0 0 200 6 first three numbers are trend identifiers the last two are general price change and length (in bars) of the trend. The next example is formed not earlier than the end of this trend.
It is possible not to predict the length of the trend, but only the absolute price change over it.
This will make the sets of examples (more) consistent.
We will get N1 N2 N3 examples for three types of trends.
for example
UP ( 1 0 0 ) - 100
flat( 0 1 0) - 200
down( 0 0 1) - 250

How to make a normal distribution.
1. if the minimum number of examples (100) suits us, we can stupidly cut the rest
2. Those examples which are less numerous should be offered more often, to balance them out.
3. Add to the examples those which are few in number, giving them a little noise.
4. more examples for down than for up, the latter can be added to the latter with examples of the reverse quotes of USDEUR instead of EURUSD

Now I will analyze these examples in the descending order of coolness :)
4. The ideal There are no left figures, everything is flesh and blood of the market. But it's tedious to write, I will have to recalculate a lot of indices by myself + to calculate them myself.
3. It is a bit worse, as we give at least some levity to the grid in the form of that noise we add.
2. If we show the grid many times the same thing, it will simply remember it and will not generalize.
1. Although for the training to take enormously long story (to ensure that after the discarding of remaining enough examples) - then this method takes all three Powerfully beats all the other three For all parameters :)


So it goes something like this. By the way, my Expert Advisor is a mix of items 1 and 2.
 
maveric:

Not only normal. At the very least, the example sets should also be consistent.
In general it looks like this.
We find the beginning of the first trend ("up down flat") Some number of bars before this beginning will be a grid entry, the whole trend following it will be rolled in the form 1 0 0 200 6 first three numbers are trend identifiers the last two are general price change and the length (in bars) of the trend. The next example is formed no sooner than the end of this trend.
It is possible not to predict the length of the trend, but only the absolute price change over it.
This will make the sets of examples (more) consistent.
We will get N1 N2 N3 examples for three types of trends.
for example
UP ( 1 0 0 0 ) - 100
flat( 0 1 0 ) - 200
down( 0 0 1 ) - 250
The normal distribution is the probability distribution of a random variable. A data set cannot be contradictory or non-contradictory - it is the value of a f-function describing a physical process and it simply is. Only our interpretation of its values can be contradictory.

You know, I cannot understand one thing, what do you want a neural network to do? Approximate a series or get a classification of the market situation? As I see it, they are fundamentally different tasks, so the input and output data should be completely differently prepared.

Let's define the terms a bit:
1. Approximation of a series is the construction of a multidimensional mapping F: x->y which generalizes a given set of examples {xt, yt}. On the basis of this mapping, it is realizing a H-step view into the future based on a certain latent dependence in the terms of the series. I.e. it may be generalized - we are trying to operate with the value of the row members themselves as well as with some set of values describing the row - they are local extrema etc. (we define a price corridor for some interval, say)
2. Classification - i.e. a market image is formed - say, an ascending trend. It is necessary to recognize this image.

What do you make of this? Judging from your code - you are trying to predict entry and exit points of the market. Frankly speaking, I deeply doubt the correctness of
for such a task.

I'd be grateful - if you can prove me wrong.

--
rip
 
rip:
Maveric:

Not only normal. At least the example sets should be consistent as well.
In general it goes like this.
We find the beginning of the first trend ("up down flat") Some number of bars before this beginning will be a grid entry, the whole trend following it will be rolled in the form 1 0 0 200 6 first three numbers are trend identifiers the last two are general price change and length (in bars) of the trend. The next example is formed not earlier than the end of this trend.
It is possible not to predict the length of the trend, but only the absolute price change over it.
This will make the sets of examples (more) consistent.
We will get N1 N2 N3 examples for three types of trends.
for example
UP ( 1 0 0 ) - 100
flat( 0 1 0) - 200
down( 0 0 1) - 250
The normal distribution is the probability distribution of a random variable. A data set cannot be contradictory or non-contradictory - it is the value of a f-function describing a physical process and it simply is. Only our interpretation of its values can be contradictory.

You know, I cannot understand one thing, what do you want a neural network to do? Approximate a series or get a classification of the market situation? As I see it, they are fundamentally different tasks, so the input and output data should be completely differently prepared.

Let's define the terms:
1. Approximation of series is a construction of multidimensional mapping F: x->y, which generalizes a given set of examples {xt, yt}. On the basis of this mapping, we can imagine the future in H steps on the basis of certain hidden dependence of series terms. I.e. it may be generalized - we are trying to operate with the value of the row members themselves as well as with some set of values describing the row - they are local extrema etc. (we define a price corridor for a certain interval, say).
2. Classification - i.e. a market image is formed - say, an ascending trend. This image needs to be recognized.

What do you make of this? From what I understood in your code - you're trying to predict the entry and exit points of the market. Frankly, I have my doubts about the validity
of the task.

I would be grateful - if you can prove the contrary.

--
rip


For classification problems, it is our interpretation of the data that counts. Roughly speaking, in a letter recognition task, in a set of examples of the letter A you should not find the letters X and U, etc. :)

I want to do both. Maybe two grids, maybe one as it turns out. The first step of the grid is to classify the current situation. If it gives a sufficiently clear signal of the beginning of a trend, then the second step is an attempt to look into the future in order to estimate how much money can be made on this trend.

I assume that approximation of financial series is worse than classification.

 
maveric писал (а):

My assumption is that financial series lend themselves worse to approximation than to classification.

Price series, like any other continuous series, can be approximated without any problem. Just do not confuse interpolation with extrapolation. Interpolation by a multi-layer neuron can be done like two fingers on the asphalt. Extrapolation of non-periodic series by neural networks is a waste of time.