<jupyter_start><jupyter_text>Demo of the combination and simplification algorithm<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>For using the library, just import the _RuleCOSIClassifier_ class from **rulecosi** package<jupyter_code>from rulecosi import RuleCOSIClassifier<jupyter_output><empty_output><jupyter_text>The algorithm works with several type of tree ensembles and it uses the **sklearn** implementations.- Bagging Trees- RandomForests- Gradient Boosting Trees (original implementation)- XGBoost- Light GBM- CatBoost<jupyter_code>#from catboost import CatBoostClassifier
#from lightgbm import LGBMClassifier
from xgboost import XGBClassifier

#from sklearn.ensemble import BaggingClassifier, RandomForestClassifier, GradientBoostingClassifier

from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from imblearn.metrics import geometric_mean_score
from sklearn.metrics import make_scorer<jupyter_output><empty_output><jupyter_text>Load a sample dataset and split the dataWe use the Wisconsin diagnostic breast cancer dataset. There are two classes, malignant (0) and benign (1).<jupyter_code>data = pd.read_csv('data/wisconsin.csv')
data.head()
X = data.drop(['Class'], axis=1)
y = data['Class']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.10, random_state=1212)<jupyter_output><empty_output><jupyter_text>Simplifying an XGBoost classifierWe create a XGBClassifier instance. The ensemble can be fitted, or it can be just instantiated and RuleCOSI will fit the ensemble first and then simplify it.<jupyter_code>ens = XGBClassifier(random_state=1212)<jupyter_output><empty_output><jupyter_text>This is done by instanciating a **RuleCOSIClassifier** class with the desired parameters, _n\_estimator_, _tree\_max\_depth_, _conf\_threshold_ and _min\_samples_.<jupyter_code>rc = RuleCOSIClassifier(base_ensemble=ens, 
                        metric='gmean',n_estimators=100, tree_max_depth=3, 
                        conf_threshold=0.9, min_samples=0,
                        random_state=1212, column_names=X_train.columns)
%%time
rc.fit(X_train, y_train)<jupyter_output>Wall time: 414 ms<jupyter_text>Examining the simplified rulesThe rules will be stored in the _simplified\_ruleset_ \_ attribute of the RuleCOSI object. The function _print\_rules_ print the rules and its heuristics on the console. It can also return a string object or a pandas DataFrame object to be used for further analysis. Additionally, the decimal digits displayed on the heuristics values and the condition thresholds can be modified with the _heuristics\_digits_ and the _condition\_digits_ parameters.<jupyter_code>rc.simplified_ruleset_.print_rules(heuristics_digits=4, condition_digits=1)
rc.simplified_ruleset_.print_rules(return_object='dataframe',heuristics_digits=4, condition_digits=1)<jupyter_output><empty_output><jupyter_text>Checking the classification performance of the simplified rule-based classifier<jupyter_code>def get_n_rules(rulesets):
    n_rules = 0
    for ruleset in rulesets:
        for rule in ruleset:
            n_rules += 1
    return n_rules
print(f'Original XGBoost ensemble')
print(f'Number of rules: {get_n_rules(rc.original_rulesets_)} rules\n')
rc.simplified_ruleset_.print_rules()
y_pred = rc.predict(X_test)
if isinstance(rc.base_ensemble, XGBClassifier):
    y_pred_ens = rc.base_ensemble.predict(X_test, validate_features=False)
else:
    y_pred_ens = rc.base_ensemble.predict(X_test)
print("Combinations: {}".format(rc.n_combinations_))
print("Time: {}\n".format(rc.combination_time_))
print(f'============ XGBoost ====================')
print(classification_report(y_test, y_pred_ens,digits=4))
print('G-mean: {}'.format(geometric_mean_score(y_test, y_pred_ens)))
print(f'\n============ Simplified rules =======================')
print(classification_report(y_test, y_pred,digits=4))
print('G-mean: {}'.format(geometric_mean_score(y_test, y_pred)))    
print('\n')<jupyter_output>Original XGBoost ensemble
Number of rules: 603 rules

cov 	conf 	supp 	samples 		rule
0.3192	0.9490	0.3029	[186,10]	r_1: (BlandChromatin ≥ 2.500) ˄ (BareNuclei ≥ 1.500) ˄ (CellShape ≥ 2.500) → [0]
0.0163	1.0000	0.0163	[10,0]		r_2: (BlandChromatin ≥ 4.500) ˄ (NormalNucleoli ≥ 3.500) → [0]
0.0114	1.0000	0.0114	[7,0]		r_3: (EpithelialSize ≥ 3.500) ˄ (ClumpThickness ≥ 6.500) ˄ (CellShape ≥ 2.500) → [0]
0.0065	1.0000	0.0065	[4,0]		r_4: (BareNuclei ≥ 1.500) ˄ (ClumpThickness ≥ 6.500) → [0]
0.0016	1.0000	0.0016	[1,0]		r_5: (BareNuclei ≥ 1.500) ˄ (NormalNucleoli ≥ 3.500) ˄ (CellShape ≥ 2.500) → [0]
0.0016	1.0000	0.0016	[1,0]		r_6: (BareNuclei ≥ 5.500) ˄ (BlandChromatin ≥ 3.500) → [0]
0.6433	0.9873	0.6352	[0,0]		r_7: ( ) → [1]

Combinations: 1680
Time: 0.36617445945739746

============ XGBoost ====================
              precision    recall  f1-score   support

           0     0.9615    1.0000    0.9804        25
           1     1.0000    0.9773    0.9885        44

    accuracy       [...]
{'fraction_non_alphanumeric': 0.11890061740689106, 'fraction_numerical': 0.05875323640709022, 'mean_word_length': 5.254047322540473, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '651516', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1978, 'n_tokens_neox': 1767, 'n_words': 449, 'path': 'examples/RuleCOSI example.ipynb', 'repo_name': 'jobregon1212/rulecosi', 'size': '5021'}