<jupyter_start><jupyter_text>Classification using the embeddingsIn the classification task we predict one of the predefined categories given an input. We will predict the score based on the embedding of the review's text, where the algorithm is correct only if it guesses the exact number of stars. We split the dataset into a training and a testing set for all the following tasks, so we can realistically evaluate performance on unseen data. The dataset is created in the [Obtain_dataset Notebook](Obtain_dataset.ipynb).In the following example we're predicting the number of stars in a review, from 1 to 5.<jupyter_code>import pandas as pd
import numpy as np

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, accuracy_score

df = pd.read_csv('output/embedded_1k_reviews.csv')
df['babbage_similarity'] = df.babbage_similarity.apply(eval).apply(np.array)

X_train, X_test, y_train, y_test = train_test_split(list(df.babbage_similarity.values), df.Score, test_size = 0.2, random_state=42)

clf = RandomForestClassifier(n_estimators=100)
clf.fit(X_train, y_train)
preds = clf.predict(X_test)
probas = clf.predict_proba(X_test)

report = classification_report(y_test, preds)
print(report)<jupyter_output>precision    recall  f1-score   support

           1       0.82      0.67      0.74        21
           2       0.50      0.50      0.50         6
           3       1.00      0.46      0.63        13
           4       0.75      0.35      0.48        17
           5       0.88      1.00      0.93       143

    accuracy                           0.86       200
   macro avg       0.79      0.60      0.66       200
weighted avg       0.86      0.86      0.84       200<jupyter_text>We can see that the model has learnt to distinguish between the categories decently. 5-star reviews show the best performance overall, and this is not too surprising, since they are the most common in the dataset.<jupyter_code>from utils import plot_multiclass_precision_recall

plot_multiclass_precision_recall(probas, y_test, [1,2,3,4,5], clf)<jupyter_output>RandomForestClassifier() - Average precision score over all classes: 0.93
{'fraction_non_alphanumeric': 0.06143497757847534, 'fraction_numerical': 0.04932735426008968, 'mean_word_length': 3.005385996409336, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '576793', 'license': 'MIT', 'n_tokens_mistral': 726, 'n_tokens_neox': 668, 'n_words': 230, 'path': 'examples/embeddings/Classification.ipynb', 'repo_name': 'BastianZim/openai-python', 'size': '2230'}