<jupyter_start><jupyter_text>Creating queries that are unique (they cannot share words between them) and result in the same cluster for both of them (but a different cluster than the original 'Mood disorder due to known acute rheumatoid arthritis' query)For example, if the original 'Mood disorder due to known acute rheumatoid arthritis' query resulted in cluster 4 then our two new queries should result both in cluster 0, 1, 2, or 3.<jupyter_code>def predict_query(query):
  query = vectorizer.transform([query])
  predicted = model.predict(query)
  print('query sentence belongs to cluster %d' % predicted[0])
  
query2 = 'Morning sickness is common in first trimester'
query3 = 'glucose level soars up during pregnancy '
predict_query(query2)

predict_query(query3)<jupyter_output>query sentence belongs to cluster 1<jupyter_text>Increasing the value of true_k (number of clusters) until our two queries end up in different clusters<jupyter_code>true_k = 6
model = KMeans(n_clusters=true_k, init='k-means++', max_iter=100, n_init=1)
model.fit(X)
clust_labels = model.predict(X)

order_centroids = model.cluster_centers_.argsort()[:, ::-1]
terms = vectorizer.get_feature_names()

predict_query(query2)
predict_query(query3)<jupyter_output>query sentence belongs to cluster 5
query sentence belongs to cluster 2
{'fraction_non_alphanumeric': 0.07838660578386605, 'fraction_numerical': 0.01598173515981735, 'mean_word_length': 6.346368715083799, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '289443', 'license': 'MIT', 'n_tokens_mistral': 420, 'n_tokens_neox': 382, 'n_words': 147, 'path': 'KNNclustering_Text_Analysis.ipynb', 'repo_name': 'gsdataenthusiast/KNN-Clustering-Text-Analysis', 'size': '1314'}