<jupyter_start><jupyter_text>ASSOCIATION ANALYSIS WITH APRIORI**File:** Apriori.ipynb**Course:** Data Science Foundations: Data Mining in Python  INSTALL AND IMPORT LIBRARIESThe Python library `apyori` contains the implementation of the Apriori algorithm, which can be installed with Python's `pip` command. This command only needs to be done once per machine.The standard, shorter approach may work:<jupyter_code># pip install apyori<jupyter_output><empty_output><jupyter_text>If the above command didn't work, it may be necessary to be more explicit, in which case you could run the code below.<jupyter_code># import sys
# !{sys.executable} -m pip install apyori<jupyter_output><empty_output><jupyter_text>Once `apyori` is installed, then load the libraries below.<jupyter_code>import pandas as pd              # For dataframes
import matplotlib.pyplot as plt  # For plotting data
from apyori import apriori       # For Apriori algorithm<jupyter_output><empty_output><jupyter_text>LOAD AND PREPARE DATAFor this demonstration, we'll use the dataset `Groceries.csv`, which comes from the R package `arules` and is saved as a CSV file. The data is in transactional format (as opposed to tabular format), which means that each row is a list of items purchased together and that the items may be in different order. There are 32 columns in each row, each column either contains a purchased items or `NaN`.The code below opens the dataset and converts to to list format, which is necessary for the `apriori()` function.<jupyter_code>transactions = []

with open('data/Groceries.csv') as f:
    for line in f:
        transaction = [item for item in line.strip().split(',') if item != 'NaN']
        transactions.append(transaction)
    
transactions[:3]<jupyter_output><empty_output><jupyter_text>APPLY APRIORICall `apriori()` on `transactions`. As parameters `apriori()` can take the minimum support, minimum confidence, minimum lift and minimum items in a transaction. Only the pairs of items that satisfy these criteria would be returned.<jupyter_code>rules = list(apriori(
    transactions, 
    min_support=0.001, 
    min_confidence=0.10,
    min_length=2,
    max_length=2))

# Prints one rule
print(rules[0])<jupyter_output><empty_output><jupyter_text>Convert Rules to Readable FormatThe printed rule above is not very clear. Let's convert it to a more readable format. We'll add a `From` and `To` field to the DataFrame, to indicate a rule's antecedent and consequent respectively. Hence for a rule of the form `A->B`. The `From` will contain `A` and `To` will contain `B`. We'll also add the `Support`, `Confidence` and `Lift` corresponding to each rule in the DataFrame.<jupyter_code>rules_df = pd.DataFrame(
    [{'From': list(rule[0])[0],
    'To': list(rule[0])[1],
    'Support': rule[1],
    'Confidence': rule[2][0][2],
    'Lift': rule[2][0][3]} for rule in rules if len(rule[0]) == 2])
rules_df = rules_df.dropna()

rules_df.head()<jupyter_output><empty_output><jupyter_text>List Rules with N'sThe code below calls `plot()` on each row of the rules DataFrame to create a list of all the mined rules. First, we have to add two numeric columns corresponding to each item to `rules_df`.<jupyter_code># Pick top rules
rules_df = rules_df.sort_values('Support', ascending=False).head(50)

# List of all items
items = set(rules_df['From']) | set(rules_df['To'])

# Creates a mapping of items to numbers
imap = {item : i for i, item in enumerate(items)}

# Maps the items to numbers and adds the numeric 'FromN' and 'ToN' columns
rules_df['FromN'] = rules_df['From'].map(imap)
rules_df['ToN'] = rules_df['To'].map(imap)

# Displays the top 20 association rules, sorted by Support
rules_df.head(20)<jupyter_output><empty_output><jupyter_text>Plot RulesPlot each pair of items in the rule. If a rule is A->B, then the item A is in the bottom row of the plot (y=0) and B is in the top row (y=1). The color of each line indicates the support of the rule multiplied by 100 (support*100). The width of each line is controlled by the confidence of each rule.<jupyter_code># Adds ticks to the top of the graph also
plt.rcParams['xtick.top'] = plt.rcParams['xtick.labeltop'] = True

# Sets the size of the plot
fig = plt.figure(figsize=(15, 7))

# Draws a line between items for each rule
# Colors each line according to the support of the rule
for index, row in rules_df.head(20).iterrows():
    plt.plot([row['FromN'], row['ToN']], [0, 1], 'o-',
             c=plt.cm.viridis(row['Support'] * 10),
             markersize=20,
             lw=row['Confidence'] * 10)

# Adds a colorbar and its title  
cb = plt.colorbar(plt.cm.ScalarMappable(cmap='viridis'))
cb.set_label('Support*10')

# Adds labels to xticks and removes yticks
plt.xticks(range(len(items)), items, rotation='vertical')
plt.yticks([])
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10510014453850919, 'fraction_numerical': 0.011356597150526533, 'mean_word_length': 4.885783718104496, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '392653', 'license': 'Apache-2.0', 'n_tokens_mistral': 1570, 'n_tokens_neox': 1497, 'n_words': 581, 'path': 'Apriori.ipynb', 'repo_name': 'VladimirsHisamutdinovs/data-mining', 'size': '4843'}