<jupyter_start><jupyter_text>Recommend three strains based on strain, effects, and flavors proposed by app user. 'Documents' are constructed by string representations of **Strain**, **Effects**, and **Flavors**. - A SpaCy instance is used to create a search space by vectorizing the docs.- The docs are vectorized.- A function is defined to take a text, searches the space, and returns three closest matches.  Import code needed.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv('./data/cannabis.csv')<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis<jupyter_code>df.shape
# df.head()
# df.groupby('Rating').count()<jupyter_output><empty_output><jupyter_text>Limit search and results to quality straains.<jupyter_code>good_stuff = df[df['Rating'] >= 4.0]
good_stuff.shape<jupyter_output><empty_output><jupyter_text>Replace Not a Number values to avoid errors.<jupyter_code># replace blank flavor with ""
good_stuff = df.replace(np.nan, '', regex=True)<jupyter_output><empty_output><jupyter_text>A function to standardize string input:<jupyter_code>def clean_string(strng):
    s = strng.replace(","," ") # comma-> space
    s = s.replace("("," ") # (-> space
    s = s.replace(")"," ") # (-> space
    s = s.lower()
    return s<jupyter_output><empty_output><jupyter_text>Combine labels into one string, for vectorization.<jupyter_code># cols = good_stuff.columns
cols = ['Type', 'Effects', 'Flavor', 'Description']

for col in cols:
    good_stuff[col] = good_stuff[col].apply(clean_string)

good_stuff['text'] = good_stuff['Type'] + " " + good_stuff['Effects'] + " " + good_stuff['Flavor']
# good_stuff.iloc[0]['text']
# good_stuff.shape
# !pip install spacy
!pip install spacy download en_core_web_lg
import spacy
spacy.load("en_core_web_lg")<jupyter_output><empty_output><jupyter_text>Load Spacy to use embedding strings in a space to be used to find similar results.<jupyter_code>!python -m spacy link en_core_web_lg en
nlp = spacy.load('en')
# nlp<jupyter_output><empty_output><jupyter_text>Build a list of docs to use fitting model.<jupyter_code>def gather_docs(df):
    """ Produces List of Documents from a dataframe.
    
    df: a Pandas dataframe that has the column 'text'.
    
    Returns a list of strings. 
    """
    
    docs = list(df['text'])
    
    return docs
# docs = gather_docs(good_stuff)
# docs[:5]<jupyter_output><empty_output><jupyter_text>Instantiate Vectorizer.<jupyter_code>from sklearn.feature_extraction.text import CountVectorizer

vect = CountVectorizer(stop_words='english', max_features=1000)<jupyter_output><empty_output><jupyter_text>Fit vectorizer to docs.<jupyter_code># Fit our documents
vect.fit(docs)<jupyter_output><empty_output><jupyter_text>Make a Document-Term Matrix, and Densify.<jupyter_code>#Create a sparse document-term matrix
dtm = vect.transform(docs)

# Make a dataframe of a condensed version of the DTM, using feature names
dtm = pd.DataFrame(dtm.todense(), columns=vect.get_feature_names())
# dtm.head()
doc_len = [len(doc) for doc in docs]<jupyter_output><empty_output><jupyter_text>Make histogram of occurrence of words in 'docs'.<jupyter_code>import seaborn as sns
sns.distplot(doc_len);<jupyter_output><empty_output><jupyter_text>Instandiate and fit a nearest neighbors learner.<jupyter_code>from sklearn.neighbors import NearestNeighbors
# Fit on DTM
n = NearestNeighbors(n_neighborsn=3, algorithm='kd_tree')
nn.fit(dtm)
# inspect
nn.kneighbors([dtm.iloc[0].values])
test_text = [ """hybrid creative energetic tesla euphoric monster earthy sweet citrus"""]
# input_dtm = vect.transform(test_text)
# input_dtm
# input_dtm = pd.DataFrame(input_dtm.todense(), columns=vect.get_feature_names())
new = vect.transform(test_text)
new
nn.kneighbors(new.todense())
# Inspect Most relevant result
docs[429]<jupyter_output><empty_output><jupyter_text>Define function to use in retrieving three nearest.<jupyter_code># def clean_string(strng):
#     s = strng.replace(","," ") # comma-> space
#     s = s.replace("("," ") # (-> space
#     s = s.replace(")"," ") # (-> space
#     s = s.lower()
#     return s

def text_to_top_three(txt):
    """ Receives a string containing strain, effects, and flavors, and 
    
    returns a 2-tuple of (array of scores, array of indexes) describing
    
    the best matches among strains modeled."""
    
    clean_text = clean_string(txt)
    transformed_text = vect.transform(clean_text)
    return nn.kneighbors(transformed_text.todense())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12329370321444298, 'fraction_numerical': 0.0030823425803610744, 'mean_word_length': 5.690721649484536, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 64, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '621387', 'license': 'MIT', 'n_tokens_mistral': 1611, 'n_tokens_neox': 1537, 'n_words': 376, 'path': 'Med_Cab_1.4_Jud.ipynb', 'repo_name': 'gptix/DS--Data-Engineering-', 'size': '4542'}