<jupyter_start><jupyter_text>Multinomial Bayes Classifiers for Text ClassificationBayes Klassifikatoren werden auch Naive Bayes oder Indepedence Bayes genannt.<jupyter_code>import utils_bayes
from sklearn.naive_bayes import MultinomialNB
import numpy as np
available_themes = list(utils_bayes.newsgroups_categories.keys())
available_themes
themen = ["Space", "Hockey", "Baseball", "Computer4", "Politik"]

model, predict_category = utils_bayes.get_multinomial_nb_model(
    themen=available_themes
)<jupyter_output><empty_output><jupyter_text>`model` ist eine Pipeline aus einem sogenannten `TfidfVectorizer` und einem Bayes-Klassifikator `MultinomialNB`<jupyter_code>model
predict_category("Launching a rocket to Mars in 2035.")
predict_category("Team A won the championship")
predict_category("I bought a new MacBook Pro but it's actually slower than my Lenovo laptop!")
predict_category("I broke my keyboard. Accidentally spilled coffee all over it")
predict_category("I bought a new computer but it's actually slower than my Lenovo laptop!")
predict_category("new computer is broken")
predict_category("The team rocket won the MacBook-Price, that is like a MarsBall")
predict_category("hungry")
predict_category("I bought a new mouse and keyboard and it's broken")
predict_category("The mouse got the MacBook-Price, that is like a MarsBall")<jupyter_output><empty_output><jupyter_text>2. Scikit-Learn's Newsgroups Dataset<jupyter_code>from sklearn.datasets import fetch_20newsgroups

newsgroups = fetch_20newsgroups(subset="train")
print(newsgroups["DESCR"])
data = newsgroups["data"]
len(data)
print(data[2])
newsgroups["target_names"]
labels = newsgroups["target"]<jupyter_output><empty_output><jupyter_text>3. Text vektorisieren<jupyter_code>from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
labels
CountVectorizer?
count_vectorizer = CountVectorizer(stop_words="english")
count_vectorizer.fit(data)
# die Länge des Vokabulars entspricht der Menge unterschiedlicher Wörter im gesamten Trainingsdatensatz
len(count_vectorizer.vocabulary_)
# der CountVectorizer ist ein Transformer
vectorized_data = count_vectorizer.transform(data).todense()
vectorized_data.shape
vectorized_data
len(np.nonzero(vectorized_data[0, :])[1])
# der CountVectorizer ist ein Transformer
vectorized_data = count_vectorizer.transform(data)

bayes_classifier = MultinomialNB()

bayes_classifier.fit(vectorized_data, labels)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08536585365853659, 'fraction_numerical': 0.005691056910569106, 'mean_word_length': 9.21161825726141, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '596922', 'license': 'MIT', 'n_tokens_mistral': 809, 'n_tokens_neox': 771, 'n_words': 196, 'path': '2_6_bayes_klassifikatoren/Tutorial - Bayes Klassifkator - completed.ipynb', 'repo_name': 'layerwise/training', 'size': '2460'}