<jupyter_start><jupyter_text>Two-Step Techniques (Spy & NB)This method learns $P(y|x)$ by a two-step technique (step1: Spy, step2: Gaussian NB).<jupyter_code>import numpy as np
import pandas as pd<jupyter_output><empty_output><jupyter_text>Load the dataset<jupyter_code>from data import load_scar, load_sar, load_pg

train, valid, test, c = load_scar()<jupyter_output><empty_output><jupyter_text>Step1: Spy  Choose "spy" data from labeled data<jupyter_code>np.random.seed(0)

train_xs, train_ys, train_ss, train_es = train

labeled_data_indexes = np.where(train_ss == 1)[0]

n_spy = round(len(labeled_data_indexes) * 0.1)  # 0.1 is the ratio of spy

spy_data_indexes = np.random.choice(labeled_data_indexes, n_spy, replace=False)
spy = np.zeros_like(train_ss)
spy[spy_data_indexes] = 1<jupyter_output><empty_output><jupyter_text>Learn a non-traditional classifier $P(s'|x)$<jupyter_code># s_ (s'): a new label indicator after regarding spy data as unlabeled
new_train_ss = train_ss & (1 - spy)
from sklearn.linear_model import LogisticRegression

_clf = LogisticRegression(random_state=0).fit(train_xs, new_train_ss)<jupyter_output><empty_output><jupyter_text>Find $P_{spymin} = \min P(s_|x, spy=1)$<jupyter_code>train_ss_prob = _clf.predict_proba(train_xs)[:, 1]
threshold = train_ss_prob[spy.astype(bool)].min()<jupyter_output><empty_output><jupyter_text>Create a new dataset, regarding data points satisfying $P(s_|x) < P_{spymin}$ as negative<jupyter_code># x_: the dataset consisting of positive-labeled data and negative-labeled data from the dataset x
# y_: the label
data_indexes_pos = labeled_data_indexes
xs_pos = train_xs[data_indexes_pos]
ys_pos = np.ones(len(xs_pos), dtype=np.int32)

data_indexes_neg = np.where(train_ss_prob < threshold)[0]
xs_neg = train_xs[data_indexes_neg]
ys_neg = np.zeros(len(xs_neg), dtype=np.int32)

new_train_xs = np.concatenate([xs_neg, xs_pos])
new_train_ys = np.concatenate([ys_neg, ys_pos])<jupyter_output><empty_output><jupyter_text>Visualize the newly created dataset<jupyter_code>from utils import plot_x_y, plot_x_s
plot_x_s(train_xs, spy)
plot_x_y(new_train_xs, new_train_ys)<jupyter_output><empty_output><jupyter_text>Step2: Learn a Gaussian NB classifier P(y_|x_)Now, we have positive-labeled data (s=1), negative-labeled data (s=0 & P(s_|x) < P_spymin), and unlabeled data (otherwise).We learn a classifier only using positive-labeled data and negative-labeled data.<jupyter_code>from sklearn.naive_bayes import GaussianNB

clf = GaussianNB().fit(new_train_xs, new_train_ys)<jupyter_output><empty_output><jupyter_text>Predict<jupyter_code>test_xs, test_ys, test_ss, test_es = test

test_ys_hat = clf.predict(test_xs)
test_ys_prob = clf.predict_proba(test_xs)[:, 1]<jupyter_output><empty_output><jupyter_text>Evaluate the performance<jupyter_code>from sklearn.metrics import f1_score

f1_score(test_ys, test_ys_hat)<jupyter_output><empty_output><jupyter_text>Visualize the result<jupyter_code>from utils import plot_x_y_proba
plot_x_y(test_xs, test_ys_hat)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11059907834101383, 'fraction_numerical': 0.008558262014483212, 'mean_word_length': 8.83495145631068, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '116849', 'license': 'MIT', 'n_tokens_mistral': 1224, 'n_tokens_neox': 1179, 'n_words': 222, 'path': 'notebooks/two_step_spy_nb.ipynb', 'repo_name': 'hkiyomaru/pu-learning', 'size': '3038'}