<jupyter_start><jupyter_text>After noting which cell types need to be created on wikidada, now I`ll populate a spreadsheet with the information I need. I can only reference the P31 (instance of) statement, but that is ok.As mentioned in the previous notebook, I`ll:    - Set a table for manual edits with following columns:        - Len (label)        - Den (description)        - Aen (aliases)        - P31 (instance of, referenced to PanglaoDB)        - P279 (subclass of)        - P361 (part of)        - P703 (found in taxon)    - Add the items to Wikidata    Notes:    - T follicular helper cell already existed, but as "Follicular B helper T cells"    Changes to make:    - T follicular helper cells: Q5464650    - Retinal precursor cells: Q7316768I have made the table, now I`ll proceed with the QuickStatements commands<jupyter_code>import pandas as pd

df = pd.read_csv("../data/cell_types_to_add_31_10_2020.csv")
df = df.fillna("")
df

with open("../quickstatements/add_cell_types_30_10_2020.qs", "w+") as qs:

    for i, row in df.iterrows():

        s = "LAST"

        Len = s + "|Len|" + '"' + row["Len"] + '"' + "\n"
        Den = s +"|Den|" + '"' + row["Den"] + '"' + "\n"
        Aen = s +"|Aen|" + '"' + row["Aen"] + '"' + "\n"
        if row["Aen.1"] != "":
            Aen_2 = s +"|Aen|" + '"' + row["Aen.1"] + '"' + "\n"
        else:
            Aen_2 = ""      
        P31 = s + "|P31|" + row["P31"] + "|S248|" + "Q99936939" +   "\n" 
        P279 = s + "|P279|" + row["P279"] + "\n"
        P703 = s + "|P703|" + row["P703"] + "\n"
        P361 = s + "|P361|" + row["P361"] + "\n"

        qs.write('CREATE \n')
        qs.write(Len)
        qs.write(Den)
        qs.write(Aen)
        qs.write(Aen_2)
        qs.write(P31)
        qs.write(P279)
        qs.write(P703)
        qs.write(P361)
        qs.write('\n')<jupyter_output><empty_output><jupyter_text>Now I have prepared the table and the quickstatements V1 commands. Nice. What I`ll do now is create a dict with all the manually curated PanglaoDB to Wikidata conversions. It will be overlaid upon the checked automatic matches of a few days ago. First, the previously curated matches:<jupyter_code>previous_matches = {
"Schwann cells" : "Q465621",
"Astrocyte progenitor cells" : "Q101001053",
"NK cells" : "Q332181",
"Beta cells" : "Q1767180",
"Satellite cells" : "Q101001061",
"Ependymal cells" : "Q28646122",
"Myocytes" : "Q428914",
"T cells" : "Q193529",
"Multinuclear myocytes" : "Q101003165",
"His bundle cells" : "Q101003168",
"Transient cells" : "EXCLUDE",
"Spermatocyte and Spermatozoa" : "Q101003191",
"Granulocytes" : "Q223143",
"Theca cells" : "Q101003314",
"Acinar cells" : "Q66590269",
"Neuroendocrine cells" : "Q10599959",
"Epiblast cells" : "Q101003525",
"Immature neurons " : "Q101003994",
"Cortical interneurons" : "Q101004030",
"Neural stem/precursor cells" : "Q2944097",
"Oligodendrocyte progenitor cells" : "Q7086819",
"T cell naive" : "Q6959842",
"Cardiomyocytes" : "Q1047227",
"T memory cells" : "Q2304808",
"Clara cells" : "Q645712",
"Peripheral blood mononuclear cells" : "Q736033",
"Juxtaglomerular cells" : "Q2596226",
"Cardiac stem and precursor cells" : "Q5376332",
"Thymocytes" : "Q7799635",
"Tuft cells" : "Q25325383",
"Paneth Cells" : "Q2004084",
"Parietal cells" : "Q1640093",
"Mesangial cells" : "Q3331086",
"Trophoblast cells" : "Q66589800",
"Hemangioblasts" : "Q1642030",
"Pancreatic progenitor cells" : "Q25325366",
"Cholangiocytes" : "Q5104283",
"Mesothelial cells" : "Q66562849",
"Myoepithelial cells" : "Q1476681",
"Retinal epithelial cells" : "Q66572893",
"Neuroblasts" : "Q2927095",
"Pulmonary alveolar type I cells" : "Q3296912",
"Pulmonary alveolar type II cells" : "Q3272858",
"Cajal-Retzius cells" : "Q2299284",
"Reticulocytes" : "Q1404923",
"tanycyte" : "Q1419042",
"Gamma delta T cells" : "Q3077796",
"Pinealocytes" : "Q3241787",
"Satellite glial cells" : "Q63710"
}<jupyter_output><empty_output><jupyter_text>However, I decided not to do it for the new matches. Instead, I will run the reconciliation code again, and manually check all matches, to be extra sure.As I am ultimately interested in the markers table, I have downloaded it from https://panglaodb.se/markers.html and stored it in ../data/PanglaoDB_markers_27_Mar_2020.tsv . This was the name of the file in the database, leading me to infer it was last updated in 27 March 2020.<jupyter_code>import pandas as pd

markers = pd.read_csv("../data/PanglaoDB_markers_27_Mar_2020.tsv", sep="\t")
markers.head()
from reconciler import reconcile
import pandas as pd


cell_types = pd.Series(list(set(markers["cell type"])))
reconciled = reconcile(cell_types, type_id="Q189118")
reconciled["match"].value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14678510998307953, 'fraction_numerical': 0.10173434856175972, 'mean_word_length': 3.9005181347150257, 'pattern_counts': {'":': 1, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '224155', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 2114, 'n_tokens_neox': 1716, 'n_words': 488, 'path': 'improvements/src/add_missing_cell_types_31_10_2020.ipynb', 'repo_name': 'jvfe/wikidata_panglaodb', 'size': '4728'}