<jupyter_start><jupyter_text>Ruleminer tutorial Welcome to the tutorial on ruleminer! This tutorial explains the most important features of the ruleminer package.The ruleminer package works with Pandas DataFrames.<jupyter_code>import pandas as pd
import numpy as np
import ruleminer<jupyter_output><empty_output><jupyter_text>Let's construct a simple dataframe to do some rule mining.<jupyter_code>col = ['Name', 'Type', 'Assets', 'TP-life', 'TP-nonlife', 'Own funds', 'Diversification','Excess']
insurers = [['Insurer  1', 'life insurer',     1000,  800,    0,  200,   12,  200], 
            ['Insurer  2', 'non-life insurer',   40,    0,   32,    8,    9,    8], 
            ['Insurer  3', 'non-life insurer',  800,    0,  700,  100,   -1,  100],
            ['Insurer  4', 'life insurer',       25,   18,    0,    7,    8,    7], 
            ['Insurer  5', 'non-life insurer', 2100,    0, 2200,  200,   12,  200], 
            ['Insurer  6', 'life insurer',      907,  887,    0,   20,    7,   20],
            ['Insurer  7', 'life insurer',     7123,    0, 6800,  323,    5,  323],
            ['Insurer  8', 'life insurer',     6100, 5920,    0,  180,   14,  180],
            ['Insurer  9', 'non-life insurer', 9011,    0, 8800,  211,   19,  211],
            ['Insurer 10', 'non-life insurer', 1034,    0,  901,  133,    1,  134]]
df = pd.DataFrame(columns = col, data = insurers)
df.set_index('Name', inplace = True)
df<jupyter_output><empty_output><jupyter_text>Can we find the errors in this report?  Rule with equal values<jupyter_code>templates = [{'expression': '({".*"}=={".*"})'}]
r = ruleminer.RuleMiner(templates=templates, data=df)
r.rules<jupyter_output><empty_output><jupyter_text>When using the equal-pattern you can define the accuracy of the equal pattern. For this you can use the decimal-parameter.<jupyter_code>params = {'decimal': 0}
templates = [{'expression': '({".*"}=={".*"})'}]
r = ruleminer.RuleMiner(templates=templates, data=df, params=params)
r.rules<jupyter_output><empty_output><jupyter_text>Patterns with value constant value<jupyter_code>params = {'filter': {'confidence': 0.5, 'abs support': 2}, 
          'decimal': 0}
templates = [{'expression': '({".*"}>=0)'}]
r = ruleminer.RuleMiner(templates=templates, data=df, params=params)
r.rules<jupyter_output><empty_output><jupyter_text>So we have six patterns (for each column), with one exception, namely that the column 'diversification' contains one negative value.  Sum-patterns With an expression this would look like this:<jupyter_code>params = {'filter': {'confidence': 0.5, 'abs support': 2}, 
          'decimal': 0}
templates = [{'expression': 'if ({"TP."}>0) then (({"TP."}+{".*"})=={".*"})'}]
r = ruleminer.RuleMiner(templates=templates, data=df, params=params)
r.rules
r.rules.values<jupyter_output><empty_output><jupyter_text>Conditional patterns With the conditional pattern you can find conditional statements between columns, such as IF TV-life = 0 THEN TV-nonlife > 0:<jupyter_code>params = {'filter': {'confidence': 0.5, 'abs support': 2}, 
          'decimal': 0}
templates = [{'expression': 'if ({"TP-life"}==0) then ({"TP-nonlife"} > 0)'}]
r = ruleminer.RuleMiner(templates=templates, data=df, params=params)
r.rules

# p2 = {'name'      : 'equal values',
#                           'expression'   : 'IF {"TV-life"} = 0 THEN {"TV-nonlife"} > 0',
#                           'parameters': {"min_confidence": 0.5,
#                                          "min_support"   : 2}}
# miner.find(p2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16332297259112744, 'fraction_numerical': 0.05001412828482622, 'mean_word_length': 3.452830188679245, 'pattern_counts': {'":': 1, '<': 29, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '367699', 'license': 'MIT', 'n_tokens_mistral': 1316, 'n_tokens_neox': 1179, 'n_words': 330, 'path': 'notebooks/Tutorial ruleminer package.ipynb', 'repo_name': 'zena3/dnb-tutorials', 'size': '3539'}