<jupyter_start><jupyter_text>I used precomputed fingerprints from PubChem. To reproduce, you can run `python download_data_from_pubchem.py`, however it takes some time.<jupyter_code>fingerprints = pd.read_csv("pubchem_fingerprints.csv")
train_df = train.merge(fingerprints, on="SMILES", how="left")
test_df = test.merge(fingerprints, on="SMILES", how="left")
print(train_df.fingerprint.isnull().sum(), "train molecules have no associated fingerprint")
print(test_df.fingerprint.isnull().sum(), "test molecules have no associated fingerprint")<jupyter_output>33 train molecules have no associated fingerprint
5 test molecules have no associated fingerprint<jupyter_text>I use only molecules which have fingerprint available to find k nearest neighbours, that's why I filter both train and test data and use unpacked fingerprints to compute K nearest neighbours.<jupyter_code>def to_bits(x):
    try:
        unpacked = np.unpackbits(np.frombuffer(bytes.fromhex(x), dtype=np.uint8))
    except Exception as e:
        print(e)
        print(x)
        
    return unpacked


train_df = train_df[~train_df.fingerprint.isnull()]
train_fingerprints = train_df.fingerprint.apply(to_bits)#lambda fingerprint_string: [x=='1' for x in fingerprint_string])
train_fingerprints = np.stack(train_fingerprints.values)

test_df = test_df[~test_df.fingerprint.isnull()]
test_fingerprints = test_df.fingerprint.apply(to_bits)#lambda fingerprint_string: [x=='1' for x in fingerprint_string])
test_fingerprints = np.stack(test_fingerprints.values)
nbrs = NearestNeighbors(n_neighbors=5, algorithm='ball_tree').fit(train_fingerprints)
distances, neighbour_indices = nbrs.kneighbors(test_fingerprints)
for i, neighbours in zip(test_df.index, neighbour_indices):
    test.loc[i, "PREDICTIONS"] = ";".join([train.loc[train_df.index[x], "SENTENCE"] for x in neighbours])
test.PREDICTIONS.isnull().sum()<jupyter_output><empty_output><jupyter_text>We still need to fill several predictions, for this we use top-5 most common molecular scents from train dataset.<jupyter_code>train.SENTENCE.value_counts()[:5]
default_prediction = ";".join(train.SENTENCE.value_counts()[:5].index)
test.loc[test.PREDICTIONS.isnull(), "PREDICTIONS"] = default_prediction
test.to_csv("baseline_submission.csv", index=None)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11361665221162186, 'fraction_numerical': 0.004336513443191674, 'mean_word_length': 7.038327526132404, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '662067', 'license': 'MIT', 'n_tokens_mistral': 775, 'n_tokens_neox': 727, 'n_words': 167, 'path': 'baseline_solution.ipynb', 'repo_name': 'latticetower/learning-to-smell-baseline', 'size': '2306'}