<jupyter_start><jupyter_text>Benchmarking organizations with the Dimensions APIThis Python notebook shows how to use the [Dimensions Analytics API](https://www.dimensions.ai/dimensions-apis/) in order to perform different benchmarking analyses of Organizations using publications data. Outline1. Quick yet effective benchmarking calculations via built-in API aggregate indicators2. Building more complex quality benchmarking indicators<jupyter_code>import datetime
print("==\nCHANGELOG\nThis notebook was last run on %s\n==" % datetime.date.today().strftime('%b %d, %Y'))<jupyter_output>==
CHANGELOG
This notebook was last run on Feb 21, 2022
==<jupyter_text>PrerequisitesThis notebook assumes you have installed the [Dimcli](https://pypi.org/project/dimcli/) library and are familiar with the ['Getting Started' tutorial](https://api-lab.dimensions.ai/cookbooks/1-getting-started/1-Using-the-Dimcli-library-to-query-the-API.html).<jupyter_code>!pip install dimcli -U --quiet 

import dimcli
from dimcli.utils import *
import os, sys, time, json
import pandas as pd

print("==\nLogging in..")
# https://digital-science.github.io/dimcli/getting-started.html#authentication
ENDPOINT = "https://app.dimensions.ai"
if 'google.colab' in sys.modules:
  import getpass
  KEY = getpass.getpass(prompt='API Key: ')  
  dimcli.login(key=KEY, endpoint=ENDPOINT)
else:
  KEY = ""
  dimcli.login(key=KEY, endpoint=ENDPOINT)
dsl = dimcli.Dsl()<jupyter_output>[2mSearching config file credentials for 'https://app.dimensions.ai' endpoint..[0m<jupyter_text>1. Quick benchmarking using the API Benchmarking by volume is reasonably straight forward if what you want to compare is volume, or one of the available aggregate indicators in the Dimensions API (see [https://docs.dimensions.ai/dsl/examples.htmlindicators-aggregations](https://docs.dimensions.ai/dsl/examples.htmlindicators-aggregations)).<jupyter_code>%%dsldf
search publications
return research_orgs[name] aggregate altmetric_median 
%%dsldf
search publications
return research_orgs[name] aggregate citations_total
%%dsldf
search publications
return research_orgs[name] aggregate recent_citations_total<jupyter_output>Returned Research_orgs: 20
[2mTime: 6.54s[0m<jupyter_text>Aside:  Recent Citations<jupyter_code>%%dsldf
search publications
return year aggregate recent_citations_total
dsl_last_results.sort_values(by='id').plot(x='id', y='recent_citations_total', figsize=(20,10))
recent_citations = dsl_last_results
recent_citations['recent_ratio'] = recent_citations['recent_citations_total']/recent_citations['count']
recent_citations['year'] = recent_citations['id']
recent_citations.sort_values(by='year').\
    plot(x='year',y='recent_ratio', figsize=(20,10))<jupyter_output><empty_output><jupyter_text>End Aside:  2. Calculating more complex 'Quality' Benchmarking indicators: Number of articles in the top X percent of research their category  Step 1.  retrieve the total volume of publications by volume. (focusing on Fields of Research)<jupyter_code>%%dsldf

search publications
where year=2018
return category_for limit 1000<jupyter_output>Returned Category_for: 176
[2mTime: 1.02s[0m<jupyter_text>Step 1.2. ... Need to filter for level 2 codes<jupyter_code>result = dsl.query("""
      search publications
      where year=2018
      return category_for limit 1000

""").as_dataframe()
result['level'] = result.name.apply(lambda n: len(n.split(' ')[0]))
result
result[result['level']==2]<jupyter_output><empty_output><jupyter_text>Step 2. calculate 1% of the total number of records by category. This will be used to retrieve the 1% boundary record..What is the boundary record?<jupyter_code>result['cutoff'] = (result['count'] * .01).astype('int')
result[result['level']==2]<jupyter_output><empty_output><jupyter_text>Step 3. Use the cutoff value to get the indicator value for the 1% boundaryNote: Here we use:'sort by' , limit, and skip!*   'sort by': return results in order of field_citation_ratio*   'limit': we are only interested in the first result returned*   'skip' we are 'skipping' to the boundary recordDouble Note: this strategy won't work when the boundary record is > 50,000...<jupyter_code>dfl = []

for r in result[result['level']==2].iterrows():
    
    result = dsl.query(f"""
      
           search publications
           where category_for.id = "{r[1]['id']}"
           and year = 2018
           return publications[field_citation_ratio] 
               sort by field_citation_ratio 
               limit 1 
               skip {r[1]['cutoff']}
      
      """).as_dataframe()

    result['name'] = r[1]['name']
    result['id'] = r[1]['id']
    dfl.append(result)   
  
cutoffs = pd.concat(dfl)
cutoffs<jupyter_output><empty_output><jupyter_text>We can only filter on integers in the DSL, so we will round up the values<jupyter_code>cutoffs.field_citation_ratio =  cutoffs.field_citation_ratio.astype('int')
cutoffs<jupyter_output><empty_output><jupyter_text>Step 4. Now get the number of publications by organisation, filtered by category that have a field_citation_ratio > the boundary score<jupyter_code>dfl = []

for r in cutoffs.iterrows():

  result = dsl.query(f"""

     search publications 
     where 
         year=2018
         and category_for.id = "{r[1]['id']}"
         and field_citation_ratio >= {int(r[1]['field_citation_ratio'])}
    return research_orgs limit 1000

  """).as_dataframe()

  result['for_name'] = r[1]['name']
  result['for_id'] = r[1]['id']
  dfl.append(result)<jupyter_output>Returned Research_orgs: 1000
[2mTime: 1.21s[0m
Returned Research_orgs: 1000
[2mTime: 1.09s[0m
Returned Research_orgs: 1000
[2mTime: 3.14s[0m
Returned Research_orgs: 1000
[2mTime: 1.30s[0m
Returned Research_orgs: 1000
[2mTime: 0.96s[0m
Returned Research_orgs: 1000
[2mTime: 1.13s[0m
Returned Research_orgs: 1000
[2mTime: 1.06s[0m
Returned Research_orgs: 1000
[2mTime: 1.16s[0m
Returned Research_orgs: 1000
[2mTime: 1.06s[0m
Returned Research_orgs: 927
[2mTime: 1.13s[0m
Returned Research_orgs: 915
[2mTime: 1.04s[0m
Returned Research_orgs: 704
[2mTime: 0.83s[0m
Returned Research_orgs: 863
[2mTime: 1.03s[0m
Returned Research_orgs: 1000
[2mTime: 1.01s[0m
Returned Research_orgs: 903
[2mTime: 0.96s[0m
Returned Research_orgs: 896
[2mTime: 1.10s[0m
Returned Research_orgs: 1000
[2mTime: 1.26s[0m
Returned Research_orgs: 1000
[2mTime: 1.07s[0m
Returned Research_orgs: 476
[2mTime: 0.81s[0m
Returned Research_orgs: 495
[2mTime: 0.71s[0m
Returned Research_orgs: 369
[...]<jupyter_text>ok, can only filter on Integrers<jupyter_code>top_insts = pd.concat(dfl)<jupyter_output><empty_output><jupyter_text>Step 5. Rank the results<jupyter_code>top_insts['rank'] = top_insts.groupby('for_name')['count'].rank(ascending=False)
top_insts[top_insts['name']=='University of Melbourne'][['for_name','rank']]<jupyter_output><empty_output><jupyter_text>We should probably control for Volume though...  Step 6. Get the total paper counts for each organisation<jupyter_code>dfl = []

for r in cutoffs.iterrows():

  result = dsl.query(f"""

     search publications 
     where 
         year=2018
         and category_for.id = "{r[1]['id']}"
    return research_orgs limit 1000

  """).as_dataframe()

  result['for_name'] = r[1]['name']
  result['for_id'] = r[1]['id']
  dfl.append(result)


all_publications = pd.concat(dfl)[['id','for_id','count']]
top_insts_all = all_publications.rename(columns={'count':'count all'}).merge(top_insts, on =['id','for_id'])
top_insts_all[['for_name','name','count','count all']]<jupyter_output><empty_output><jupyter_text>Step 7.  calculate the percentage of local papers in the top 1% of global publications (in 2018)<jupyter_code>top_insts_all['percentage top 1'] = (100 * top_insts_all['count']/top_insts_all['count all']).round(2)
top_insts_all['percent rank'] = top_insts_all.groupby('for_name')['percentage top 1'].rank(ascending=False)<jupyter_output><empty_output><jupyter_text>Now the results are going to look a little strange...<jupyter_code>top_insts_all[top_insts_all['name']=='University of Cambridge'][['for_name','percent rank']]
top_insts_all[top_insts_all['for_name']=='11 Medical and Health Sciences'][['name','percent rank']]<jupyter_output><empty_output><jupyter_text>Smaller institutions are being preferenced too much...  Need to control for size...<jupyter_code>reference_institutions = top_insts_all[['id','name','for_id','count all']].\
     rename(columns={
            'id':'reference id',
            'name':'reference name',
           'count all':'reference count all'
           })
relative_ranking = reference_institutions.merge(top_insts_all, on='for_id')
relative_ranking[relative_ranking['reference name']=='University of Melbourne']
filtered_relative_ranking = relative_ranking[relative_ranking[
                                      'reference count all'] <= relative_ranking['count all']
                                      ].copy()
filtered_relative_ranking['filtered percent rank'] = filtered_relative_ranking.\
                                                   groupby(['reference id','for_name'])['percentage top 1'].\
                                                   rank(ascending=False)
inst = 'University of Melbourne'

filtered_relative_ranking[
                          
                          (filtered_relative_ranking['reference name'] == inst) &
                          (filtered_relative_ranking['name'] == inst)
                         
                         ][['id', 'for_id', 'name','for_name','filtered percent rank']]<jupyter_output><empty_output><jupyter_text>Final step. Show me the institutions that I should be most interested in (Five above)<jupyter_code>rank_cutoffs = filtered_relative_ranking[
                          
                          (filtered_relative_ranking['reference name'] == filtered_relative_ranking['name'] ) 
                         
                         ][['id', 'for_id', 'filtered percent rank']].\
                         rename(columns={'id':'reference id',
                                         'filtered percent rank':'reference filtered percent rank'})
filtered_relative_ranking_final = rank_cutoffs.merge(filtered_relative_ranking, on=['reference id','for_id'])
filtered_relative_ranking_final['rank_difference'] = filtered_relative_ranking_final['filtered percent rank'] - filtered_relative_ranking_final['reference filtered percent rank']
inst = 'Monash University'
forname = '11 Medical and Health Sciences'

filtered_relative_ranking_final[
                                 
                                 (filtered_relative_ranking_final['rank_difference'].between(-5, 5)) &
                                 (filtered_relative_ranking_final['reference name'] == inst) &
                                 (filtered_relative_ranking_final['for_name'] == forname)
                                 
                                 ][['name','filtered percent rank']].sort_values(by='filtered percent rank')



filtered_relative_ranking_final<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12318120198825125, 'fraction_numerical': 0.027293267058291912, 'mean_word_length': 4.239583333333333, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 67, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '640671', 'license': 'MIT', 'n_tokens_mistral': 3790, 'n_tokens_neox': 3396, 'n_words': 794, 'path': 'docs/cookbooks/8-organizations/7-benchmarking-organizations.ipynb', 'repo_name': 'digital-science/dimensions-api-examples', 'size': '11065'}