Dependency statistics in quotes (information theory, correlation and other feature selection methods) - page 9
You are missing trading opportunities:
- Free trading apps
- Over 8,000 signals for copying
- Economic news for exploring financial markets
Registration
Log in
You agree to website policy and terms of use
If you do not have an account, please register
not to me)))) if independence is required, why such a thing as conditional entropy, for example?
If the sequence of symbols of the alphabet is not independent (for example, in French the letter "q" is almost always followed by "u", and the word "vanguard" in Soviet newspapers was usually followed by "production" or "labor"), the amount of information that the sequence of such symbols carries (and, hence, entropy) is obviously less. Conditional entropy is used to account for such facts. https://ru.wikipedia.org/wiki/Информационная_энтропия
Oh yes, conditional entropy. Is it clear that there's a closed space of outcomes there anyway? And all the letters are quite clearly counted, etc. In fact we are talking about a simple extension of the original 26-character alphabet to an alphabet of how many syllables. That's a rough approximation.
so that's roughly the point of entropy))) Or the degree of compression by a perfect archiver.
Sorry, HideYourRichess, but you seem to have gone off the deep end. I don't know what to discuss with you now that you are so insistent on outright nonsense. Your logic of reasoning
is completely incomprehensible to me.
I will not believe it. Show me the source that states that
so that's roughly the point of entropy))) Or in the degree of compression by a perfect archiver
HideYourRichess, if you think that all tervers are reduced to Bernoulli's series or the law of large numbers, you are greatly mistaken.
Stop fooling around and familiarise yourself with the concept of conditional probability, which is directly used in determining conditional entropy and mutual information.
HideYourRichess, if you think the whole terver is reduced to Bernoulli's series or the law of large numbers, you are very wrong.
I don't think it, I know it for a fact.
The notion of independent tests, derived from Bernoulli, will that suffice? Or here, a formulation of the law of large numbers: Let there be an infinite sequence of equally distributed and uncorrelated random variables... Let there be an infinite sequence of independent equally distributed random variables...
I suggest picking up Shannon's own publications and reading. I think the opponents of this thread are just picking a "fight" for no particular reason. I studied probability theory at university, although my education is not mathematical. As far as I remember, the stationarity of the random variables being studied is an important characteristic.
And I'll say more from my own, non-mathematical, education. Let's take communication theory as applied to which TI was developed. There is a wire carrying signals, their meaning is not important for us. We want to calculate information losses in this wire and we consider the source and transmitter (RIGHT: transmitter and receiver) as two random variables. Aren't they connected a priori? I note - the assumption is that they are connected by the proverbial wire. What do you say to that?
HideYourRichess,
Stop fooling around and familiarise yourself with the concept of conditional probability, which is directly used in determining conditional entropy and mutual information.
One more thing to add for HideYourRichess
Mutual information counts for very strongly related, correlated variables and determines the amount of both the information itself and its loss. So the coupling of events at the physical level is an element of the whole theory. Or was Schennom wrong...
HideYourRichess, если Вы думаете, что весь тервер сводится к сериям Бернулли или закону больших чисел, то Вы сильно ошибаетесь.
I don't think so, I know that for a fact.
What independent events are you talking about? About a sequence of alphabetic characters from the source? No, they are not necessarily independent, it has already been explained to you. An ordinary Russian literary text is a sequence of dependent letters. If they were independent, literary texts would be much worse compressed by the archiver than they really are. Take and shuffle some literary text and compare the archiving results of the original and shuffled text.
Or do you think the source and receiver ensembles are independent variables?