<commit_before>
Experiments here are going to review those basic/popular frequent mining algorithms and also to explore new and better algorithms

*************************************************************

CONCEPTS & TERMINOLOGY

* Frequent Pattern Mining is an unsupervised learning
* <b>Support</b> - probability of an event to occur. For example, `Support(ice-cream) = number of transactions that contain ice-crea/totoal number of transactions`
* <b>Confidence</b> - conditional probability. For example, the probability that you will buy ice-cream after you have bought chocolate
* <b>Lift</b> - The ratio of confidence to expected confidence. <b>The probability of all of the items in a rule occurring together (otherwise known as the support) divided by the product of the probabilities of the items on the left and right side occurring as if there was no association between them.</b><commit_msg>update apriori<commit_after>
Experiments here are going to review those basic/popular frequent mining algorithms and also to explore new and better algorithms

*************************************************************

CONCEPTS & TERMINOLOGY

* Frequent Pattern Mining is an unsupervised learning
* <b>Support</b> - probability of an event to occur. For example, `Support(ice-cream) = number of transactions that contain ice-crea/totoal number of transactions`
* <b>Confidence</b> - conditional probability. For example, the probability that you will buy ice-cream after you have bought chocolate
* <b>Lift</b> - The ratio of confidence to expected confidence. <b>The probability of all of the items in a rule occurring together (otherwise known as the support) divided by the product of the probabilities of the items on the left and right side occurring as if there was no association between them.</b>


*************************************************************

ALGORITHMS REVIEW

* Apriori
  * It is almost the most basic alg in frequent mining, but it simply works. If, performance is important, Apriori may be slower than some other alg
  * [R] code: https://github.com/hanhanwu/Hanhan_Data_Science_Practice/blob/master/Frequent_Pattern_Mining/basic_alg_apriori.R
    * This R package could recommend you what items to put together, and show you support, confidence and lift at the same time
  * reference: https://www.analyticsvidhya.com/blog/2017/08/mining-frequent-items-using-apriori-algorithm/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29
{'fraction_non_alphanumeric': 0.13259229853116317, 'fraction_numerical': 0.004366812227074236, 'mean_word_length': 5.4781491002570695, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5504253', 'n_tokens_mistral': 667, 'n_tokens_neox': 625, 'n_words': 286}