<commit_before># Markov Chains

Prediction of an event N based on previous (N-1) events using Markov Chains

Yes. That is right. I can see the future and I can read your mind. 

Actually, I can't... but Markov can.

To try it, just type a sentence in English on the first form above. We will provide a list of words that you might be typing next. Make a selection of the word above and keep typing. 

On the right you will see a distribution of the alternatives we found so far. A visual description of how certain we are about each of your possible future choices. 

Have fun!<commit_msg>changes in documentation<commit_after># markovchains

_Prediction of an event N based on previous (N-1) events_

Yes. That is right. I can see the future and I can read your mind. 

Actually, I can't... but Markov can.

To [try it](https://jfaleiro.shinyapps.io/markovchains/), just type a sentence in English on the first form on the top. We will provide a list of words that you might be typing next (we can indeed read your mind). You can also make a selection of the word on the form and keep typing. 

On the right you will see a distribution of the alternatives we found so far. The (_kinda_ exponential) distribution is a visual description of how certain we are about each of your possible future choices.

This is a demonstration of the use of Markov Chains for the prediction of sequenced events. This type of prediction assumes that a probability of an event ocurring at time $T = t$ is affected by previous events  occurring from $T = 0$ to $T = t-1$. 

This techinique is used on several domains like genetic sequencing, speech recognition (this notable example here) and my favorite: prediction of the next price tick in financial time series and random walks. 

If you are looking for details, the data analysis leading to this prototype is described in a [separate report](http://rpubs.com/jfaleiro/text-prediction-preliminary-analysis). You can also refer to a [set of slides](http://rpubs.com/jfaleiro/markovchains) describing the background and the overal structure of this exercise.

Have fun!
{'fraction_non_alphanumeric': 0.049476688867745006, 'fraction_numerical': 0.0019029495718363464, 'mean_word_length': 4.0311004784688995, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7403269', 'n_tokens_mistral': 552, 'n_tokens_neox': 528, 'n_words': 341}