Here is my problem. I have to compute the amount of information that is possible to encode in a string of bits. This string of bits represent a stream.Let us call such stream as $X_1,X_2,X_3,...,X_n$. Important: this succession of Random variable is not a Markov Process, but a process with memory!!I suppose that the right measure would be based on the information entropy of the collection of random variables:$H(X_1,X_2,X_3,...,X_n)=\sum_i H(X_i|X_{i-1},...,X_1)$Am I right?However I have more doubts:If $n \to \infty$ then a better measure would be the entropy rate, isn't it?.What is meaning of $H(X_n)$, i.e. the entropy of only the last random variable?In term of information theory, what is the meaning of $H(X_1, X_n)$ or $H(X_1| X_n)$ ?A point is that I can compute both $H(X_1, X_n)$ and $H(X_1| X_n)$ for my process, but the computation of $H(X_1,X_2,X_3,...,X_n)$ remains very hard, as the process is non markovian.Thanks!
Information of a stream of bits
information theory;coding theory