<jupyter_start><jupyter_text>Metrics specific to imbalanced learningSpecific metrics have been developed to evaluate classifier whichhas been trained using imbalanced data. :mod:`imbalanced_ensemble` provides mainlytwo additional metrics which are not implemented in :mod:`sklearn`: (i)geometric mean (:func:`imbalanced_ensemble.metrics.geometric_mean_score`) and (ii) index balanced accuracy (:func:`imbalanced_ensemble.metrics.make_index_balanced_accuracy`).<jupyter_code># Adapted from imbalanced-learn 
# Authors: <NAME> <<EMAIL>>
# License: MIT
print(__doc__)

RANDOM_STATE = 42

# sphinx_gallery_thumbnail_path = '../../docs/source/_static/thumbnail.png'<jupyter_output><empty_output><jupyter_text>First, we will generate some imbalanced dataset.<jupyter_code>from sklearn.datasets import make_classification

X, y = make_classification(
    n_classes=3,
    class_sep=2,
    weights=[0.1, 0.9],
    n_informative=10,
    n_redundant=1,
    flip_y=0,
    n_features=20,
    n_clusters_per_class=4,
    n_samples=5000,
    random_state=RANDOM_STATE,
)<jupyter_output><empty_output><jupyter_text>We will split the data into a training and testing set.<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=RANDOM_STATE
)<jupyter_output><empty_output><jupyter_text>We will create a pipeline made of a :class:`~imbalanced_ensemble.sampler.over_sampling.SMOTE`over-sampler followed by a :class:`~sklearn.svm.LinearSVC` classifier.<jupyter_code>from imbalanced_ensemble.pipeline import make_pipeline
from imbalanced_ensemble.sampler.over_sampling import SMOTE
from sklearn.svm import LinearSVC

model = make_pipeline(
    SMOTE(random_state=RANDOM_STATE), LinearSVC(random_state=RANDOM_STATE)
)<jupyter_output><empty_output><jupyter_text>Now, we will train the model on the training set and get the predictionassociated with the testing set. Be aware that the resampling will happenonly when calling `fit`: the number of samples in `y_pred` is the same thanin `y_test`.<jupyter_code>model.fit(X_train, y_train)
y_pred = model.predict(X_test)<jupyter_output><empty_output><jupyter_text>The geometric mean corresponds to the square root of the product of thesensitivity and specificity. Combining the two metrics should account forthe balancing of the dataset.<jupyter_code>from imbalanced_ensemble.metrics import geometric_mean_score

print(f"The geometric mean is {geometric_mean_score(y_test, y_pred):.3f}")<jupyter_output><empty_output><jupyter_text>The index balanced accuracy can transform any metric to be used inimbalanced learning problems.<jupyter_code>from imbalanced_ensemble.metrics import make_index_balanced_accuracy

alpha = 0.1
geo_mean = make_index_balanced_accuracy(alpha=alpha, squared=True)(geometric_mean_score)

print(
    f"The IBA using alpha={alpha} and the geometric mean: "
    f"{geo_mean(y_test, y_pred):.3f}"
)
alpha = 0.5
geo_mean = make_index_balanced_accuracy(alpha=alpha, squared=True)(geometric_mean_score)

print(
    f"The IBA using alpha={alpha} and the geometric mean: "
    f"{geo_mean(y_test, y_pred):.3f}"
)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0887071406102548, 'fraction_numerical': 0.00817867253853413, 'mean_word_length': 7.174807197943445, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '306112', 'license': 'MIT', 'n_tokens_mistral': 1094, 'n_tokens_neox': 1044, 'n_words': 251, 'path': 'docs/source/auto_examples/evaluation/plot_metrics.ipynb', 'repo_name': 'ZhiningLiu1998/imbalanced-ensemble', 'size': '3206'}