<jupyter_start><jupyter_text>28 - 03 - 2021  First draft of our abstractIn our first meeting with the group, we discussed about the topic and created a first draft of our abstract. For this, everyone wrote a draft and we combined our results in the end.  30 - 03 - 2021  Preliminary Analysis: exploring the classes of errorsI started to carry some preliminary analysis, in order to butter understand the problem that we need to tackle in our research question. The dataset that we need to use to solve our problem is provided by <NAME> under public license at http://doi.org/10.5281/zenodo.4625300. This dataset contains a two-column CSV file, where the first column ("Valid_citing_DOI") contains the DOI of a citing entity retrieved in Crossref, while the second column ("Invalid_cited_DOI") contains the invalid DOI of a cited entity identified by looking at the field "reference" in the JSON document returned by querying the [Crossref API](https://www.crossref.org/education/retrieve-metadata/rest-api/) with the citing DOI. Among the column containing invalid cited DOIs I noticed some first general classes of errors that invalid the identifier:* **DOIs containing additional URLs:**  1. 10.1016/j.aca.2006.07.086.http://dx.doi.org/10.1016/j.aca.2006.07.086 → 10.1016/j.aca.2006.07.086  2. “10.1186/1735-2746-10-21,http://www.ijehse.com/content/10/1/21" → 10.1186/1735-2746-10-21* **Extra characters at the end:**  1. 10.1061/9780784480502.018] →  10.1061/9780784480502.018  2. 10.1044/1092-4388(2012/11-0316)a →  10.1044/1092-4388(2012/11-0316)* **Extra strings at the end:**   1. 10.1111/j.1099-0860.1997.tb00004.x/abstract> → 10.1111/j.1099-0860.1997.tb00004.x  2. 10.4103/0975-�-7406.163460>accessed8 →  10.4103/0975-7406.163460  3. 10.1007/s10706-019-01181-9(0123456789 → 10.1007/s10706-019-01181-9* **DOIs with wrongly encoded HTML entities:**  1. 10.1379/1466-1268(1997)002lt;0162:tmethi>2.3.co;2 → 10.1379/1466-1268(1997)0022.3.co;2Arcangelo noticed that some **DOIs contain queries to proxy servers or characters forbidden in URLs:**  1. 10.2307/2491102?uid=37380728uid=20uid=40sid=4102564553863 → 10.2307/2491102  2. 10.1016/j.envexpbot.2013.10.018doilink → 10.1016/j.envexpbot.2013.10.018  03 - 04 - 2021  First draft of Data Management Plan (DMP)In a group meeting, we created a DMP on the platform Argos (https://argos.openaire.eu/) and divided the further work on it (the description of the two datasets) between us.For the DMP, I focused on part **1: Data Summary** and **2: Reusable Data** of the dataset containing the source code used to clean a CSV list of invalid DOI names.  07 - 04 - 2021  First bites of code: Counting the invalid DOI-to-DOI citations in the CSVI started to carry some preliminary computations on our data, by counting how many pair of invalid DOI-to-DOI citations are in the CSV that we use as input data.<jupyter_code>!wget https://zenodo.org/record/4625300/files/invalid_dois.csv

# if you are running this notebook locally on Windows 10 use:
# pip install wget
# !python -m wget https://zenodo.org/record/4625300/files/invalid_dois.csv
from csv import reader

f = reader(open('invalid_dois.csv'))
header = next(f)
input_list = [(doi1, doi2) for (doi1, doi2) in f]
print(input_list[:50])
print(len(input_list))
invalid_dois = list(zip(*input_list))[1]
print(invalid_dois[:50])<jupyter_output>('10.5555/646836.708343', '10.2307/20184517', '10.1161/circ.37.4.509', '10.3748/wjg.v10.i5.707.', '10.1070/10810730903528033', '10.1161/str.24.7.8322400', '10.1111/j.545-5300.2003.42208.x', '10.3760/cma.j.issn.0366-6999.20131202', '10.1161/res.35.2.159', '10.1002/per', '10.5555/1873601.1873616', '10.3390/sym11070911www.mdpi.com/journal/symmetry', '10.13745/j.esf.2016.02.011', '10.1161/circ.85.3.1537115', '10.1901/jaba.2012.45-657', '10.1059/0003-4819-100-4-483', '10.1016/j.amepre.2015.07.017.', '10.1016/j.atmosenv.2008.0305', '10.1161/circ.66.1.7083497', '10.1080/1364253032000157166', '10.1161/str.23.11.1440702', '10.1186/s12933-015-0183-6.', '10.1037/0021–9010.93.3.602', '10.1177/0312896216656720.', '10.5555/2133036.2133123', '10.5555/2442626.2442634', '10.5555/1785594.1785635', '10.1161/res.59.2.2874900', '10.1002/smj', '10.1161/res.89.12.1216', '10.3390/brainsci7120164.', '10.5555/944919.944937', '10.1145/2838344.2856460', '10.1177/0890117116661982.', '10.1161/circ.39.1.48', '10.110[...]<jupyter_text>09 - 04 - 2021  Useful References for literature reviewHere I will list the useful references that I found in order to carry a literature review:1. <NAME>., <NAME>., <NAME>. et al. Types of DOI errors of cited references in Web of Science with a cleaning method. Scientometrics 120, 1427–1437 (2019). https://doi.org/10.1007/s11192-019-03162-42. <NAME>., <NAME>. & <NAME>. DOI errors and possible solutions for Web of Science. Scientometrics 118, 709–718 (2019). https://doi.org/10.1007/s11192-018-2980-7 3. <NAME>., <NAME>., & <NAME>. (2015). Errors in DOI indexing by bibliometric databases. Scientometrics, 102(3), 2181–2186. https://doi.org/10.1007/s11192-014-1503-4.  10 - 04 - 2021  Notes on: <NAME>., <NAME>., <NAME>. et al. (2019). Types of DOI errors of cited references in Web of Science with a cleaning method. (article: https://doi.org/10.1007/s11192-019-03162-4)This paper address the problem of DOIs errors in cited references contained in the Web of Science (WoS) database. The authors of this paper collected a set of bibliographic references in the gene editing field and deeply analysed them in order to understand which classes of errors were present in cited DOIs and which possible solution could have been attempted in order to automatically correct them. After their analysis they found that **many cited DOIs were duplicates** and they **contained various errors** which they generalized in three classes:   1.	prefix-type errors: DOIs starting with extra characters, e.g. “http://dx.doi.org/10.XXXX/XXXXXXXXXX”  2.	suffix-type errors: DOIs containing extra characters at the end, e.g. “10.XXXX/XXXXXXXXXX(EPUB)”  3.	other types of errors: double underscores, white spaces, forward slashes and XML tagsAfter this generalization, they proposed an algorithmic solution to automatically clean duplicate DOIs and join them. This solution consists of 4 steps: (1) cleaning of prefix-type errors, (2) cleaning of suffix-type errors, (3) removal of incompatible characters and other errors and finally (4) joining of compatible DOIs. The first two steps were carried by using **regular expressions**.The authors state that the **vast majority of DOI errors belonged to the first category** of prefix-type (92.39%). After applying their algorithm, the authors achieved to reduce drastically the quantity of cited reference containing two and three DOI names from 9,704 to 1,990 and from 45 to 33, respectively. However, the authors acknowledge that their solution was not able to deal with the following issues: (a) to correct similar characters confused with each other, such as “O” versus “0”, “b” versus “6” etc., (b) to distinguish the correct DOI name from multiple DOI names assigned to the same cited reference; (c) to identify DOIs that cannot be resolved by the DOI system; and (d) to identify DOIs that are resolvable, but point to some knowledge unit which is related to the Digital Object but it’s different from it.  Notes on: <NAME>., <NAME>., & <NAME>. (2015). Errors in DOI indexing by bibliometric databases.(article: https://doi.org/10.1007/s11192-014-1503-4)In this paper, <NAME>., <NAME>. & <NAME>. provided a taxonomy of errors for two bibliometric databases: Scopus and Web of Science (WoS). What they found out is that errors in these databases fall within two general categories: (1) **author made errors**, due to lack of care of the author when providing the list of cited articles, and (2) **database mapping errors**, due to poor communication between database administrator and data provider. Moreover, they found out (a) that citations obtained from certain publishers are more likely to be omitted than those from other ones, and (b) that same DOIs are often mistakenly attached to multiple publications.  Notes on: <NAME>., <NAME>. & <NAME>. (2019). DOI errors and possible solutions for Web of Science.(article: https://doi.org/10.1007/s11192-018-2980-7)In this paper <NAME>., <NAME>. & <NAME>. start with the hypothesis that some DOIs assigned to the papers indexed in Web of Science (WoS) are wrongly mapped in the database due to similar characters mistyping: e.g., confusing the number "0" with the letter "O". In order to carry their research, they queried the WoS database with special strings were the letter "O" appeared between two 0-9 numbers (e.g. "0O1"). Among the 310 records returned by the system, 119 DOIs were impossible to be resolved within the DOI system. In addition, they found out that many DOIs were invalid due to other type of character mistyping (e.g. "b" versus "6" and "Q" versus "O") and that one article had 2 DOIs attached when only one was correct. However, despite these discoveries, the paper only suggests possible solutions for WoS and does not describe any concrete process to carry and evaluate data cleaning processes on incorrect DOIs.  19 -04 - 2021  Classes of errors: suffix-type errors After a manual revision, I find this recurrent suffix errors:1.   /doi/+"www.website.com/etc/etc"2.   /doi/+"http://dx.doi.org/etc."3.   /doi/+"...........-.-.403420(2001)"4.   /doi/+",pp.2206-2222"5.   /doi/+",http://etc."6.   /doi/ + "."7.   /doi/ + ","8.  /doi/ + ">accessed8"9.  /doi/ + ">"10.  /doi/ + "/suppinfo"11.  /doi/ + "...........32,63(2006)"12.  "10.1007/s11199-012-0130-x" + ",1-16"13.  "10.1088/2053-1583/3/4/045006" + "/meta"14.  "10.1111/j.0735-2751.2004.00237.x" +"/abstract"15.  "10.1002/smi.1053"+ ")."16.  "10.1038/208365a0" + ".......208365(1965)"17.  "10.1016/j.expneurol.2009.06.012" + "uncitedrefs"18.  "10.3389/fnins.2016.00584" + ".5186786.pmid28082858.author"19.  "10.1111/j.1365-2486.1999.00252.x" + "/full"20.  "10.1111/j.1467-9671.2009.01180.x" + "/pdf"21.  "/doi/" + ",/doi/"22. "10.1101/gr.229202" + "articlepublishedonlinebeforemarch2002"23. "10.1177/2043820617738836" + "journals.sagepub.com/home/dhg" 24. "10.1177/1468794112468475" + "qri.sagepub.com" This is a temporary list of regular expressions which might be used for cleaning suffix-type errors.Regular expressions from 1 to 8 are derived from <NAME>., <NAME>., <NAME>. et al. Types of DOI errors of cited references in Web of Science with a cleaning method. Scientometrics 120, 1427–1437 (2019). https://doi.org/10.1007/s11192-019-03162-4<jupyter_code># these are the patterns
# to match suffix errors
regex1 = "\/-\/DCSUPPLEMENTAL"
regex2 = "SUPPINF[0|O](\.)?"
regex3 = "[\.|(|,|;]?PMID:\d+.*?"
regex4 = "[\.|(|,|;]?PMCID:PMC\d+.*?"
regex5 = "[(|\[]EPUBAHEADOFPRINT[)\]]"
regex6 = "[\.|(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4}"
regex7 = "[\.|(|,|;]*HTTP:\/\/.*?"
regex8 = "[\.\/](META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)>?"
regex9 = "([\/\.](META|ABSTRACT|FULL|EPDF|PDF|SUMMARY))?[>|)](LAST)?ACCESSED\d+"
regex10 = "[\.|(|,|;]?[A-Z]*\.?SAGEPUB.*?"
regex11 = "<[A-Z\/]+>"
regex12 = "\.{5}.*?"
regex13 = "[\.|,|<|>|&|(|;]"
regex14 = "[\.;,]PP.\d+-\d+"
regex15 = "[\.|(|,|;]10.\d{4}\/.*?"

regex_lst = [regex1, regex2, regex3, regex4, regex5, regex6, regex7, regex8, regex9, regex10, regex11, regex12, regex13, regex14, regex15]

# regex from 1 to 7 are from paper
# regex from 8 to 15 are added by me


import csv, re, urllib.request

url = 'https://zenodo.org/record/4625300/files/invalid_dois.csv'
response = urllib.request.urlopen(url)
lines = [l.decode('utf-8') for l in response.readlines()]
reader = csv.reader(lines)
rows_number = 0
occurrences = list()
for row in reader:
    rows_number += 1
    pattern = re.search("(.*?)(?:"+ "|".join(regex_lst) + ")$", row[1].upper())
    if pattern is not None:
        pattern = pattern.group(1)
        occurrences.append((row[1], pattern))
print(f"The wrong DOI names are {len(occurrences)} out of {rows_number}")
print(occurrences[5:15])

## TO DO:
## 1. Deal with closing parenthesis at the end
## 2. test on DOIs checked for incorrectness to see if results vary
## 3. print output csv
## 4. simplify regex to decrease computational time<jupyter_output>[('10.1177/0890117116661982.', '10.1177/0890117116661982'), ('10.1177/0959683616683260.', '10.1177/0959683616683260'), ('10.1016/j.aca.2006.07.086.http://dx.doi.org/10.1016/j.aca.2006.07.086', '10.1016/J.ACA.2006.07.086'), ('10.1007/s10548-017-0554-2.', '10.1007/S10548-017-0554-2'), ('10.1021/bi0100236...........-.-.403420(2001)', '10.1021/BI0100236'), ('10.1016/j.ecoenv.2014.11.015.', '10.1016/J.ECOENV.2014.11.015'), ('10.1016/j.jlumin.2004.10.018.http://dx.doi.org/10.1016/j.jlumin.2004.10.018', '10.1016/J.JLUMIN.2004.10.018'), ('10.3748/wjg.v20.i4.908.', '10.3748/WJG.V20.I4.908'), ('10.1016/j.bone.2013.01.022.', '10.1016/J.BONE.2013.01.022'), ('10.4324/9781315793207.', '10.4324/9781315793207')]<jupyter_text>Review  of The Leftovers 2.0 DMPhttps://doi.org/10.32388/DIA06O   23 - 04 - 2021 Efficiency of first list of suffix regex Out of the 148264 cited DOIs matched by my regular expression, only 10 were already valid. Out of these 10 DOIs:*   Most of them (7) have been matched due to the fact that they contain queries to proxy servers.*   8 out of 10 cleaned DOIs can still be resolved by the DOI Handle API. However, 1 of them ('10.11591/ijece.v9i4.pp2416-2424' --> '10.11591/IJECE.V9I4') points to a different resource.*  2 cleaned DOIs were no more resolvable by the DOI system:('10.3265/nefrologia.2009.29.5.5157.en.full'--> '10.3265/NEFROLOGIA.2009.29.5.5157.EN' AND '10.7895/ijadr.v0i0.222.' -->'10.7895/IJADR.V0I0.222')I think that this problems can be fixed by checking for DOI validity before cleaning the DOIs.  27 - 04 - 2021  Response letter and third version of the protocolWe worked and published a response letter to the peer reviews made by the team Leftovers 2.0 to our protocol and wrote and published a third version of our protocol. This is the revised protocol: https://dx.doi.org/10.17504/protocols.io.bufwntpeThis is the response letter to the protocol reviews: https://zenodo.org/record/4724007.YIhwhX0zb5Y  30 - 04 - 2021 Suffix regex update We updated the regular expressions for cleaning suffixes. More specifically:1. we added an initial slash to the "suffix_suppinfo" regex2. We removed the regular expression "[\\.;,]PP.\d+-\d+" because it actually altered a corrected DOI by making it pointing to a different resource (from article to venue)3. We removed the regular expressions for double DOIs "[\\.|\(|,|;]10.\d{4}\/.*?" because it also matched other symbols inside a DOI4. We added three regular expressions: one for cleaning years at the end of a string, one for cleaning queries to a proxy server and one for clearing fragment specification after the DOI (e.g. "10.5456/487346fragment")<jupyter_code>suffix_dcsupplemental = "\/-\/DCSUPPLEMENTAL"
suffix_suppinfo = "\/SUPPINF[0|O](\.)?"
suffix_pmid1 = "[\.|\(|,|;]?PMID:\d+.*?"
suffix_pmid2 = "[\.|\(|,|;]?PMCID:PMC\d+.*?"
suffix_epub = "[\(|\[]EPUBAHEADOFPRINT[\)\]]"
suffix_published_online = "[\.|\(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4}"
suffix_http = "[\.|\(|,|;]*HTTP:\/\/.*?"
suffix_subcontent = "\/(META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)([>|\)](LAST)?ACCESSED\d+)?"
suffix_accessed = "[>|\)](LAST)?ACCESSED\d+"
suffix_sagepub = "[\.|\(|,|;]?[A-Z]+\.?SAGEPUB.*?"
suffix_dotted_line = "\.{5}.*?"
suffix_delimiters = "[\.|,|<|&|\(|;]+"
suffix_doi_mark = "\[DOI\].*?"
suffix_year = "\(\d{4}\)?"
suffix_query="\?.*?=.*?"
suffix_hash ="#.*?"

suffix_regex_lst = [suffix_dcsupplemental, suffix_suppinfo, suffix_pmid1, suffix_pmid2, suffix_epub, suffix_published_online, suffix_http, suffix_subcontent, suffix_accessed, suffix_sagepub, suffix_dotted_line, suffix_delimiters, suffix_doi_mark, suffix_year, suffix_query, suffix_hash]
suffix_regex = "(.*?)(?:"+ "|".join(suffix_regex_lst) + ")$"
print("Suffix regex: ", suffix_regex)<jupyter_output>Suffix regex:  (.*?)(?:\/-\/DCSUPPLEMENTAL|\/SUPPINF[0|O](\.)?|[\.|\(|,|;]?PMID:\d+.*?|[\.|\(|,|;]?PMCID:PMC\d+.*?|[\(|\[]EPUBAHEADOFPRINT[\)\]]|[\.|\(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4}|[\.|\(|,|;]*HTTP:\/\/.*?|\/(META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)([>|\)](LAST)?ACCESSED\d+)?|[>|\)](LAST)?ACCESSED\d+|[\.|\(|,|;]?[A-Z]+\.?SAGEPUB.*?|\.{5}.*?|[\.|,|<|&|\(|;]+|\[DOI\].*?|\(\d{4}\)?|\?.*?=.*?|#.*?)$<jupyter_text>Checking matches for regex This is a little script for checking the occurrences of each pattern inside the dataset.**Note**: this numbers do not represent the actual errors that are corrected by our procedure in the dataset, since a string might be matched by more than one regular expression, even though in our procedure only the one which matches the longest error is used to clean the DOI.<jupyter_code>import csv, re, urllib.request

suffix_dcsupplemental = "\/-\/DCSUPPLEMENTAL"
suffix_suppinfo = "\/SUPPINF[0|O](\.)?"
suffix_pmid1 = "[\.|\(|,|;]?PMID:\d+.*?"
suffix_pmid2 = "[\.|\(|,|;]?PMCID:PMC\d+.*?"
suffix_epub = "[\(|\[]EPUBAHEADOFPRINT[\)\]]"
suffix_published_online = "[\.|\(|,|;]?ARTICLEPUBLISHEDONLINE.*?\d{4}"
suffix_http = "[\.|\(|,|;]*HTTP:\/\/.*?"
suffix_subcontent = "\/(META|ABSTRACT|FULL|EPDF|PDF|SUMMARY)([>|\)](LAST)?ACCESSED\d+)?"
suffix_accessed = "[>|\)](LAST)?ACCESSED\d+"
suffix_sagepub = "[\.|\(|,|;]?[A-Z]+\.?SAGEPUB.*?"
suffix_dotted_line = "\.{5}.*?"
suffix_delimiters = "[\.|,|<|&|\(|;]+"
suffix_doi_mark = "\[DOI\].*?"
suffix_year = "\(\d{4}\)?"
suffix_query="\?.*?=.*?"
suffix_hash ="#.*?"
prefix_regex = "\.)?(?:HTTP:\/\/DX\.D[0|O]I\.[0|O]RG\/|HTTPS:\/\/D[0|O]I\.[0|O]RG\/)(.*"
markup_regex1 = "<.*?>.*?</.*?>"
markup_regex2 = "<.*?/>"
backward_slash_regex = "\\\\.*?"
double_dot_regex = "\.\..+"
double_underscore_regex = "__.+"

regex_lst = [suffix_dcsupplemental, suffix_suppinfo, suffix_pmid1, suffix_pmid2, suffix_epub, suffix_published_online, suffix_http, suffix_subcontent, suffix_accessed, suffix_sagepub, suffix_dotted_line, suffix_delimiters, suffix_doi_mark, suffix_year, suffix_query, suffix_hash, prefix_regex, markup_regex1, markup_regex2, backward_slash_regex, double_dot_regex, double_underscore_regex]

url = 'https://zenodo.org/record/4625300/files/invalid_dois.csv'
response = urllib.request.urlopen(url)
lines = [l.decode('utf-8') for l in response.readlines()]
reader = csv.reader(lines)
output_dict = dict()
for row in reader:
    for idx in range(len(regex_lst)):
      pattern = re.search("(.*?)(?:"+ regex_lst[idx] + ")$", row[1].upper())
      if pattern is not None:
        if str(regex_lst[idx]) not in output_dict.keys():
          output_dict[str(regex_lst[idx])] = [row[1]]
        else:
           output_dict[str(regex_lst[idx])].append(row[1])
for key, value in output_dict.items():
  print(key, value[:5], len(value))<jupyter_output>[\.|,|<|&|\(|;]+ ['10.3748/wjg.v10.i5.707.', '10.1016/j.amepre.2015.07.017.', '10.1186/s12933-015-0183-6.', '10.1177/0312896216656720.', '10.3390/brainsci7120164.'] 120828
[\.|\(|,|;]*HTTP:\/\/.*? ['10.1016/j.aca.2006.07.086.http://dx.doi.org/10.1016/j.aca.2006.07.086', '10.1016/j.jlumin.2004.10.018.http://dx.doi.org/10.1016/j.jlumin.2004.10.018', '10.1186/1735-2746-10-21,http://www.ijehse.com/content/10/1/21', '10.1016/0014-4894(72)90103-8.http://dx.doi.org/10.1016/0014-4894(72)90103-8', '10.1002/app.23704.http://dx.doi.org/10.1002/app.23704'] 22197
\.)?(?:HTTP:\/\/DX\.D[0|O]I\.[0|O]RG\/|HTTPS:\/\/D[0|O]I\.[0|O]RG\/)(.* ['10.1016/j.aca.2006.07.086.http://dx.doi.org/10.1016/j.aca.2006.07.086', '10.1016/j.jlumin.2004.10.018.http://dx.doi.org/10.1016/j.jlumin.2004.10.018', '10.1016/0014-4894(72)90103-8.http://dx.doi.org/10.1016/0014-4894(72)90103-8', '10.1002/app.23704.http://dx.doi.org/10.1002/app.23704', '10.1006/jcat.2001.3474.http://dx.doi.org/10.1006/jcat.2001.3474'] 22077
\.{5}.*? [...]<jupyter_text>9 - 05 - 2021 Reusing regular expressions from (Xu et al., 2019) in our procedure In order to understand the novelty and efficiency of our approach we decided to reuse the regular expressions from the work of (Xu et al., 2019) in our procedure instead of our list of regular expressions.This procedure, as the one used in our paper, makes use of the methods of the class **Support**. For further informations see our [github repository](https://github.com/open-sci/2020-2021-grasshoppers-code) or our [protocol](https://dx.doi.org/10.17504/protocols.io.bunnnvde).**Note:** before comparing the results of our procedure with those obtained by using the regular expressions from (Xu et al., 2019) we will consider as cleaned DOIs only those which are *both* clean and valid. So we will not compare their results with DOIs which were temporarily invalid or DOIs which were cleaned but not valid.<jupyter_code>from support import Support
import re
from tqdm import tqdm

doi_logs = dict()

def clean_doi(doi):
    prefix_regex = "^(?:D[0|O]I\/?|HTTP:\/\/DX\.D[0|O]I\.[0|O]RG\/|[0|O]RG\/|[:\/]|\d+\.HTTP:\/\/DX\.D[0|O]I\.[0|O]RG\/?)+(.*)"
    suffix_regex = "(.*?)(?:\/-\/DCSUPPLEMENTAL|\/SUPPINF[0|O]\.?|[\s\.;]?PMID:[\d]+|[\.\/:]|[\s\.;]?PMCID:PMC\d+|[\(\.;]EPUB|[\(\[]EPUBAHEADOFPRINT[\)\]]|[\s\.;]?ARTICLEPUBLISHEDONLINE.*?\d{4}|[\.\(]*HTTP:\/\/.*?)$"
    tmp_doi = doi.replace(" ", "")
    prefix_match = re.search(prefix_regex, tmp_doi, re.IGNORECASE)
    classes_of_errors = {
        "prefix": 0,
        "suffix": 0,
        "other-type": 0
    }
    if prefix_match:
        tmp_doi = prefix_match.group(1)
        classes_of_errors["prefix"] += 1
    suffix_match = re.search(suffix_regex, tmp_doi, re.IGNORECASE)
    if suffix_match:
        tmp_doi = suffix_match.group(1)
        classes_of_errors["suffix"] += 1
    new_doi = re.sub("\\\\", "", tmp_doi)
    new_doi = re.sub("__", "_", new_doi)
    new_doi = re.sub("\\.\\.", ".", new_doi)
    new_doi = re.sub("<.*?>.*?</.*?>", "", new_doi)
    new_doi = re.sub("<.*?/>", "", new_doi)
    if new_doi != tmp_doi:
        classes_of_errors["other-type"] += 1
    return new_doi, classes_of_errors


def procedure(data):
    output = list()
    pbar = tqdm(total=len(data))
    for row in data:
        invalid_cited_doi = row["Invalid_cited_DOI"]
        unclean_dictionary = {
            "Invalid_cited_DOI": invalid_cited_doi,
            "Valid_DOI": "",
            "Prefix_error": 0,
            "Suffix_error": 0,
            "Other-type_error": 0
        }
        new_doi, classes_of_errors = clean_doi(invalid_cited_doi)
        clean_dictionary = {
            "Invalid_cited_DOI": invalid_cited_doi,
            "Valid_DOI": new_doi,
            "Prefix_error": classes_of_errors["prefix"],
            "Suffix_error": classes_of_errors["suffix"],
            "Other-type_error": classes_of_errors["other-type"]
        }
        if new_doi != invalid_cited_doi and new_doi != "":
            handle = Support().handle_request(url=f"https://doi.org/api/handles/{new_doi}", cache_path="",
                                              error_log_dict=doi_logs)
            if handle is not None:
                if handle["responseCode"] == 1:
                    output.append(clean_dictionary)
                else:
                    output.append(unclean_dictionary)
            else:
                output.append(unclean_dictionary)
        else:
            output.append(unclean_dictionary)
        pbar.update(1)
    pbar.close()
    return output

## our procedure stores only cleaned DOIs which are then
## validated through the DOI handle API
data = Support.process_csv_input(path="invalid_dois.csv")
output = procedure(data)

Support().dump_csv(data=output, path="./output.csv")

number_of_valids = 0
for row in output:
    if row["Valid_DOI"] != "":
        number_of_valids += 1
print(number_of_valids)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17282494206867496, 'fraction_numerical': 0.11957025489783021, 'mean_word_length': 3.983413814822591, 'pattern_counts': {'":': 15, '<': 57, '<?xml version=': 0, '>': 69, 'https://': 19, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '35275', 'license': '0BSD', 'n_tokens_mistral': 10392, 'n_tokens_neox': 8361, 'n_words': 2235, 'path': 'docs/Grasshoppers/cris_notebook.ipynb', 'repo_name': 'open-sci/2020-2021-', 'size': '23799'}