<jupyter_start><jupyter_text>Element 49 is better on all criteria than element 2, thus scoring them would not improve model stability. The worse correlation between rankings returned by different models based on this comparison is close to -1<jupyter_code>model.scorePair(df.iloc[49].values, df.iloc[2].values)
model.better
df.iloc[[17, 49]]<jupyter_output><empty_output><jupyter_text>Element 49 is better than 17 on 4 out of 6 criteria and worse on the remaining 2. Scoring these two models would enrich our model with some preference information; however, it can still be better.<jupyter_code>model.scorePair(df.iloc[17].values, df.iloc[49].values)
df.iloc[[13, 6]]<jupyter_output><empty_output><jupyter_text>Element 13 is better than element 6 on 3 criteria and worse on 3 remaining ones. Knowledge about the relation between these two alternatives can significantly improve model stability<jupyter_code>model.scorePair(df.iloc[13].values, df.iloc[6].values)<jupyter_output><empty_output><jupyter_text>We provide preference information $e_6 \succ e_{13}$ and then present ranking based on this one comparison<jupyter_code>model.addBetter(df.iloc[6].values, df.iloc[13].values)
df["score"] = model.getRanking()
df.sort_values("score", ascending=False)
model.plotUtilityFunctions()
df.iloc[[17, 49]]<jupyter_output><empty_output><jupyter_text>Now the comparison between 17 and 49 is a nice addition to the model since it provides information that was not available in the first comparison.<jupyter_code>model.scorePair(df.iloc[17,:6].values, df.iloc[49,:6].values)<jupyter_output><empty_output><jupyter_text>We can also create fictive alternatives to obtain preference information. With this approach we can create a stable model in smaller number of iterations.To find the best pair to compare gaussian process is used<jupyter_code>f = lambda x : -model.scorePair(x[:6], x[6:])
start = time.time()
res = gp_minimize(f,                  # the function to minimize
                  [(-1.0, 1.0) for _ in range(12)],      # the bounds on each dimension of x
                  n_calls=50,         # the number of evaluations of f
                  n_random_starts=5,  # the number of random initialization points
                  random_state=0)   # the random seed
time.time() - start
res.fun, res.x[:6], res.x[6:]
model.scorePair(res.x[6:], res.x[:6])<jupyter_output><empty_output><jupyter_text>To compare performance we will find the best pair to compare using random search with fictive alternatives and random search using real alternatives<jupyter_code>start = time.time()
score = []
pairs = []
for _ in range(300):
    i,j = np.random.choice(np.arange(len(df)), 2)
    score.append(model.scorePair(df.iloc[i,:6].values, df.iloc[j,:6].values))
    pairs.append([i,j])
print(time.time() - start)
print(max(score))

start = time.time()
score = []
pairs = []
for _ in range(300):
    i,j = np.random.randn(6), np.random.randn(6)
    score.append(model.scorePair(i, j))
    pairs.append([i,j])
print(time.time() - start)
print(max(score))<jupyter_output>Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non zero error:  tensor(0.)
Max non [...]<jupyter_text>We provide second comparison of fictive alternatives and present ranking based on the updated model<jupyter_code>model.addBetter(res.x[6:], res.x[:6])
df["score"] = model.getRanking()
df.sort_values("score", ascending=False)
model.plotUtilityFunctions()
f = lambda x : -model.scorePair(x[:6], x[6:])
start = time.time()
res = gp_minimize(f,                  # the function to minimize
                  [(-1.0, 1.0) for _ in range(12)],      # the bounds on each dimension of x
                  n_calls=50,         # the number of evaluations of f
                  n_random_starts=5,  # the number of random initialization points
                  random_state=0)   # the random seed
time.time() - start
res.fun, res.x[:6], res.x[6:]
model.addBetter(res.x[6:], res.x[:6])
df["score"] = model.getRanking()
df.sort_values("score", ascending=False)
model.plotUtilityFunctions()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12106002810680586, 'fraction_numerical': 0.02589841397309777, 'mean_word_length': 4.168049792531121, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '342374', 'license': 'MIT', 'n_tokens_mistral': 1673, 'n_tokens_neox': 1599, 'n_words': 522, 'path': 'test.ipynb', 'repo_name': 'miepsik/UTA', 'size': '4981'}