<jupyter_start><jupyter_text>Adding chairpersons to NFDI consortiaOur goal here is adding information about chairpersons (spokespersons) of NFDI consortia to Wikidata.<jupyter_code>import requests
import pandas as pd<jupyter_output><empty_output><jupyter_text>Let's check previously parsed data.<jupyter_code>df = pd.read_csv("../../../data/GEPRIS_NFDI_project_pages.csv")
df<jupyter_output><empty_output><jupyter_text>The column 'Spokesperson' contains data we need: the name of a chairperson and its GEPRIS ID.<jupyter_code>for person in df['Spokesperson']:
    print(person)<jupyter_output>[('Professor Dr. Oliver  Stegle, Ph.D.', 'https://gepris.dfg.de/gepris/person/196795178')]
[('Dr. <NAME>', 'https://gepris.dfg.de/gepris/person/53182641')]
[('Professor Torsten  Schrade', 'https://gepris.dfg.de/gepris/person/40014658')]
[('Professor Dr. Christoph  Steinbeck', 'https://gepris.dfg.de/gepris/person/1438900')]
[('Professor Dr. <NAME>  Glöckner', 'https://gepris.dfg.de/gepris/person/2043896')]
[('Dr. <NAME>', 'https://gepris.dfg.de/gepris/person/92382825')]
[('Professor Dr.-Ing. Robert  Schmitt', 'https://gepris.dfg.de/gepris/person/969363')]
[('Professorin Dr. Juliane  Fluck', 'https://gepris.dfg.de/gepris/person/1817110')]
[('Professor Dr. Christof  Wolf', 'https://gepris.dfg.de/gepris/person/1610376')]
[('Professor Dr. Erhard  Hinrichs', 'https://gepris.dfg.de/gepris/person/459920279')]
[('Professor Dr. <NAME>', 'https://gepris.dfg.de/gepris/person/47182391')]
[('Professor Dr. <NAME>', 'https://gepris.dfg.de/gepris/person/2743[...]<jupyter_text>Apart of that we need mapping between GEPRIS and Wikidata. We can query the Wikidata SPARQL endpoint. Unfortunately, some of the chairpersons did not have a corresponding page at Wikidata and majority of chairperson's Wikidata pages did not have a claim with GEPRIS ID. So we created those pages and added GEPRIS IDs to all pages of chairpersons. Let's create a helper function `get_gepris2wikidata()` which for given GEPRIS ID returns a list with the corresponding Wikidata page URL and its label.<jupyter_code>def get_gepris2wikidata(gepris, person=True):
    if person:
        prop = "P4872" # GEPRIS person ID
    else:
        prop = "P4870" # GEPRIS project ID
    query = """
    SELECT ?item ?itemLabel 
    WHERE 
    {
      ?item wdt:""" + prop + """ '""" + gepris + """'
      SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
    }
    """
    r = requests.get('https://query.wikidata.org/sparql',
                     params = {'format': 'json', 'query': query},
                     headers = {"User-Agent": "gepris2wikidata"})
    r.raise_for_status
    results = r.json().get('results').get('bindings')
    try:
        wd_id = results[0].get('item').get('value', '')
        wd_label = results[0].get('itemLabel').get('value', '')
    except Exception:
        wd_id, wd_label = '', ''
    return [wd_id, wd_label]<jupyter_output><empty_output><jupyter_text>Now all chairpersons have the Wikidata pages and a claim with GEPRIS ID:<jupyter_code>gepris2wikidata = {}
for gepris_sub, person in zip(df['gepris'], df['Spokesperson']):
    gepris_id, gepris_name = eval(person)[0][1], eval(person)[0][0]
    [chair_wd, chair_name] = get_gepris2wikidata(eval(person)[0][1].split('/')[-1])
    gepris2wikidata[gepris_sub] = chair_wd
    print([gepris_sub], [gepris_id, gepris_name ], [chair_wd, chair_name])
print(gepris2wikidata)<jupyter_output>['https://gepris.dfg.de/gepris/projekt/441914366'] ['https://gepris.dfg.de/gepris/person/196795178', 'Professor Dr. <NAME>, Ph.D.'] ['http://www.wikidata.org/entity/Q41049272', '<NAME>']
['https://gepris.dfg.de/gepris/projekt/441926934'] ['https://gepris.dfg.de/gepris/person/53182641', 'Dr. <NAME>'] ['http://www.wikidata.org/entity/Q108652783', '<NAME>']
['https://gepris.dfg.de/gepris/projekt/441958017'] ['https://gepris.dfg.de/gepris/person/40014658', 'Professor <NAME>'] ['http://www.wikidata.org/entity/Q57948193', '<NAME>']
['https://gepris.dfg.de/gepris/projekt/441958208'] ['https://gepris.dfg.de/gepris/person/1438900', 'Professor Dr. Christoph  Steinbeck'] ['http://www.wikidata.org/entity/Q5111731', '<NAME>']
['https://gepris.dfg.de/gepris/projekt/442032008'] ['https://gepris.dfg.de/gepris/person/2043896', 'Professor Dr. <NAME>  Glöckner'] ['http://www.wikidata.org/entity/Q28913667', '<NAME>']
['[...]<jupyter_text>All consortia have claims with GEPRIS ID as well:<jupyter_code>for gepris in df['gepris']:
    print([gepris, get_gepris2wikidata(gepris.split('/')[-1], person=False)])<jupyter_output>['https://gepris.dfg.de/gepris/projekt/441914366', ['http://www.wikidata.org/entity/Q98380337', 'GHGA']]
['https://gepris.dfg.de/gepris/projekt/441926934', ['http://www.wikidata.org/entity/Q98380342', 'NFDI4Cat']]
['https://gepris.dfg.de/gepris/projekt/441958017', ['http://www.wikidata.org/entity/Q98276929', 'NFDI4Culture']]
['https://gepris.dfg.de/gepris/projekt/441958208', ['http://www.wikidata.org/entity/Q96678459', 'NFDI4Chem']]
['https://gepris.dfg.de/gepris/projekt/442032008', ['http://www.wikidata.org/entity/Q98380341', 'NFDI4BioDiversity']]
['https://gepris.dfg.de/gepris/projekt/442077441', ['http://www.wikidata.org/entity/Q98380336', 'DataPlant']]
['https://gepris.dfg.de/gepris/projekt/442146713', ['http://www.wikidata.org/entity/Q98380344', 'NFDI4Ing']]
['https://gepris.dfg.de/gepris/projekt/442326535', ['http://www.wikidata.org/entity/Q98380343', 'NFDI4Health']]
['https://gepris.dfg.de/gepris/projekt/442494171', ['http://www.wikidata.org/entity/Q98380340', 'KonsortSWD']]
['h[...]<jupyter_text>Wikidata SPARQL endpoint can tell us which NFDI consortia do not have yet a claim with chairperson:<jupyter_code>def get_wditems_without_chairperson():
    query = """
    SELECT ?item
    WHERE 
    {
      ?item wdt:P31 wd:Q98270496.
      FILTER NOT EXISTS { ?item wdt:P488 ?x.}
      SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
    }
    """
    r = requests.get('https://query.wikidata.org/sparql',
                     params = {'format': 'json', 'query': query},
                     headers = {"User-Agent": "gepris2wikidata"})
    r.raise_for_status
    results = r.json().get('results').get('bindings')
    items = []
    for result in results:
        items.append(result.get('item').get('value'))
    return items
chairitems = get_wditems_without_chairperson()
chairitems<jupyter_output><empty_output><jupyter_text>To create claims with chairpersons, we need information about Wikidata ID, GEPRIS ID and `start time` for chairperson. Let's use inception of a consortium as start time for chairpersons.<jupyter_code>dfi = pd.read_csv("../../../data/wikidata_manual_edits.csv")
dfi<jupyter_output><empty_output><jupyter_text>Now we can add the data to Wikidata using Wikidataintegrator library.<jupyter_code>from wikidata import wduser, wdpass
from wikidataintegrator import wdi_core, wdi_login
login = wdi_login.WDLogin(wduser, wdpass)
for wditem in chairitems:
    subdfi = dfi[ dfi['Wikidata'] == wditem ]
    inception = '+' + subdfi['Inception'].values[0]
    gepris = "https://gepris.dfg.de/gepris/projekt/" + str(subdfi['GEPRIS'].values[0])
    chair_id = gepris2wikidata[gepris].split('/')[-1]
    wd_item = wditem.split('/')[-1]
    item = wdi_core.WDItemEngine(wd_item_id=wd_item)
    qualifiers = [ wdi_core.WDTime(inception, 'P580', is_qualifier=True, precision=9) ]
    references = [[ wdi_core.WDUrl(value=gepris, prop_nr='P854', is_reference=True) ]]
    statement = [ wdi_core.WDItemID(chair_id, prop_nr='P488', qualifiers=qualifiers, references=references) ]
    item.update(data=statement)
    print(wd_item, chair_id, gepris, inception)
    item.write(login, bot_account=False)<jupyter_output>Q98271443 Q51843931 https://gepris.dfg.de/gepris/projekt/460033370 +2021-10-01T00:00:00Z
Q98276929 Q57948193 https://gepris.dfg.de/gepris/projekt/441958017 +2020-10-01T00:00:00Z
Q98380336 Q97060913 https://gepris.dfg.de/gepris/projekt/442077441 +2020-10-01T00:00:00Z
Q98380337 Q41049272 https://gepris.dfg.de/gepris/projekt/441914366 +2020-10-01T00:00:00Z
Q98380342 Q108652783 https://gepris.dfg.de/gepris/projekt/441926934 +2020-10-01T00:00:00Z
Q98380343 Q50419539 https://gepris.dfg.de/gepris/projekt/442326535 +2020-10-01T00:00:00Z
Q98380340 Q31773505 https://gepris.dfg.de/gepris/projekt/442494171 +2020-10-01T00:00:00Z
Q98380341 Q28913667 https://gepris.dfg.de/gepris/projekt/442032008 +2020-10-01T00:00:00Z
Q99534506 Q18744528 https://gepris.dfg.de/gepris/projekt/460129525 +2021-10-01T00:00:00Z
Q108327788 Q83187047 https://gepris.dfg.de/gepris/projekt/460135501 +2021-10-01T00:00:00Z
Q108542181 Q108652977 https://gepris.dfg.de/gepris/projekt/460037581 +2021-10-01T00:00:00Z
Q108542327 Q28359[...]
{'fraction_non_alphanumeric': 0.16967344142498286, 'fraction_numerical': 0.09968029230417903, 'mean_word_length': 4.862784471218206, 'pattern_counts': {'":': 2, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 45, 'lorem ipsum': 0, 'www.': 14, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '650941', 'license': 'MIT', 'n_tokens_mistral': 4043, 'n_tokens_neox': 3520, 'n_words': 541, 'path': 'docs/docs/editing_wikidata/06_adding_chairpersons_to_Wikidata.ipynb', 'repo_name': 'UB-Mannheim/NFDI', 'size': '8890'}