<jupyter_start><jupyter_text>The purpose of this notebook is to use pubchem's online data retreival resources, in this case PUG REST specifically, in order to expand material candidate datasets by searching for other materials/compounds with similar structure. I'm sure there are much better in depth and complex methods to achieve this, but this should be viewed as a rather straight forward and quick approach for basic material screening and discovery.  The pubchempy API can be viewed here: https://pubchempy.readthedocs.io/en/latest/api.html The PUG REST documentation can be viewed here: https://pubchemdocs.ncbi.nlm.nih.gov/pug-rest According to PUG REST, there are a few methods to do chemical searches based on structure. The most appropriate one seems to be a similarity search, which simply put, searches for similarities in compounds based on their 2D structure. You can sue a cid, smiles string, or InChl to search by. They also show a "standard" and "fast" method, in which the regular method requires two subsequent requests to retrieve the data, and the fast method just instantaneously gives you the results. After playing around with both, I didn't see any difference in the results I was getting, so i'm going to stick to the very convenient "fast" method.  We can initiate a request for the data below. We will be using our trusty friend choline chloride as an initial example, searching by it's smiles string. Note you also have the option to attach constraints to the end of the request, such a max number of results returned, and more useful, a minimum threshold for the Tanimoto score, which is basically an index or score of chemical similarity. Choosing a threshold value is a little arbitrary, too low of a threshold and the resulting compounds may not be as similar as you would like. Too high of a threshold yielded a much smaller quantity of results, but they were almost "too" similar, most were just radioactive compounds of choline chloride. The default threshold value is 90, I'm going to use 80 to get a few more variety in the returned structures.<jupyter_code>request_url = "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsimilarity_2d/smiles/C[N+](C)(C)CCO.[Cl-]/cids/JSON?Threshold=80"

request = requests.get(request_url)
request_json = request.json()<jupyter_output><empty_output><jupyter_text>The result is returned as a very short dictionary, and we can parse it as such to get the list of similar compounds.<jupyter_code>request_json['IdentifierList']['CID']
len(request_json['IdentifierList']['CID'])<jupyter_output><empty_output><jupyter_text>1072 different compounds were returned, let's find out the names of those compounds.<jupyter_code>similarity_list = request_json['IdentifierList']['CID']
name_list = [] #empty list to append compound names too

for cid in similarity_list:
    #general pubchempy request to obtain a json of the entire compound data
    names_request = "https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/%s/JSON" % str(cid) 
    request = requests.get(names_request)
    request_json = request.json()
    
    names = request_json['Record']['RecordTitle'] #the compound name is contained at the very top of the json, fairly easy.
    
    name_list.append(names)
name_list<jupyter_output><empty_output><jupyter_text>Scrolling through the list, we can see some compounds that look very interesting for our application as hydrogen bond acceptors (HBA) for deep eutectic solvents. Others seem not as useful, such as the radioactive compounds I mentioned earlier, or some are just the parent compounds of the halide salts. Not sure of a great way to further screen and reduce this list, but a start would be removing results that are not halide salts, since that is our criteria for a HBA.<jupyter_code>sorted_names = []

for i in name_list:
    
    if "chloride" in i:
        sorted_names.append(i)
        
    elif "bromide" in i:
        sorted_names.append(i)
        
    elif "fluoride" in i:
        sorted_names.append(i)
        
    elif "iodide" in i:
        sorted_names.append(i)
        
    else:
        pass<jupyter_output><empty_output><jupyter_text>Keeping only the results that contain a halide seems to be the most straightforward to screen these results. The only downside is we may miss compounds that go by a common name, but may indeed by halide salts. I think there shouldn't be too many of those cases to worry about significantly.<jupyter_code>sorted_names
len(sorted_names)<jupyter_output><empty_output><jupyter_text>We have reduced the number of results from 1072 to 260, now most of these should be decent candiates to begin the screening process. Interestingly some of these results are also complexed with metals. I'm not sure about these and whether or not to include them.  Let's start writing this as a function so that we can test this on a larger dataset. Another better way to approach this is to check the smiles strings of the compounds and eliminate based on not containing the halide and the ammonium group<jupyter_code>smiles_list = []
for cid in similarity_list:
    smiles_list.append(pcp.get_properties('canonical_smiles', cid))
smiles_list
temp_list = []

for i in range(len(smiles_list)):
    
    if '[N+]' in smiles_list[i][0]['CanonicalSMILES']:
        
        temp_list.append(smiles_list[i][0])
        
#for j in range(len(temp_list)):
        
    #if '.[Cl-]' or '.[Br-]' or '.[I-]' or '.[F-]' in temp_list[j]['CanonicalSMILES']:
            
        #new_smiles_list.append(temp_list[j])
            
    
    
        
        
 
    
new_smiles_list = []

for i in range(len(temp_list)):
    if '.[Cl-]' in temp_list[i]['CanonicalSMILES']:
        new_smiles_list.append(temp_list[i]['CanonicalSMILES'])
        
    elif '.[Br-]' in temp_list[i]['CanonicalSMILES']:
        new_smiles_list.append(temp_list[i]['CanonicalSMILES'])
        
    elif '.[I-]' in temp_list[i]['CanonicalSMILES']:
        new_smiles_list.append(temp_list[i]['CanonicalSMILES'])
        
    elif '.[F-]' in temp_list[i]['CanonicalSMILES']:
        new_smiles_list.append(temp_list[i]['CanonicalSMILES'])
    
    else:
        pass
len(new_smiles_list)
new_smiles_list<jupyter_output><empty_output><jupyter_text>There is a lot of time consuming requests in this notebook, so now that we have an idea of what we want to do we will continue this on to a new notebook using one of the webscrapped datasets from nist in order to use as a n example and write a function for this whole process. Sorting by smiles string narrowed down the results much better, only things I saw still were some are strange QAS that have metal complexes.<jupyter_code>#function for finding the names of the results from their cid's

def get_names(cid_list):
    
    """This function will retrieve the chemical names from a list of cid's"""
    
    name_list = [] #empty list to append compound names too
    
    for cid in cid_list:
        #general pubchempy request to obtain a json of the entire compound data
        names_request = "https://pubchem.ncbi.nlm.nih.gov/rest/pug_view/data/compound/%s/JSON" % str(cid) 
        names_request = requests.get(names_request)
        names_request_json = names_request.json()

        names = names_request_json['Record']['RecordTitle'] #the compound name is contained at the very top of the json, fairly easy.

        name_list.append(names)
        
    return name_list
#function for sorting halide salts

def keep_salts(name_list):
    
    """This function will remove non halide salts from a list of chemical names"""
    
    sorted_list = [] #empty list to append sorted names to.

    for i in name_list:

        if "chloride" in i:
            sorted_list.append(i)

        elif "bromide" in i:
            sorted_list.append(i)

        elif "fluoride" in i:
            sorted_list.append(i)

        elif "iodide" in i:
            sorted_list.append(i)

        else:
            pass
        
        return sorted_list
#wrapper function starting from a dataframe containing smiles strings

def get_similar_structures(dataframe, smiles_column,):
    
    cid_list = [] #contains cid's of similar chemcials from results
    
    for i, row in dataframe.iterrows():
        
        smiles = row[smiles_column]
        
        request_url = "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/fastsimilarity_2d/smiles/%s/cids/JSON?Threshold=80" % str(smiles)

        request = requests.get(request_url)
        request_json = request.json()
        
        cid_list.append(request_json['IdentifierList']['CID'])
        
    name_list = get_names(cid_list)
    
    sorted_list = keep_salts(name_list)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07570361860999425, 'fraction_numerical': 0.0027570361860999423, 'mean_word_length': 3.6581059390048156, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '195599', 'license': 'MIT', 'n_tokens_mistral': 2575, 'n_tokens_neox': 2463, 'n_words': 983, 'path': 'scripts/pubchem/pubchem_similar_chemicals.ipynb', 'repo_name': 'jrodguez/des-basis-set', 'size': '8705'}