Unique identifiers (UID) are seen as an effective key to match identical publications across databases or identify duplicates in a database. The objective of the present study is to investigate how well UIDs work as match keys in the integration between Pure and SciVal, based on a case with publications from the health sciences. We evaluate the matching process based on information about coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match keys. We analyze this information to detect errors, if any, in the matching process. As an example we also briefly discuss how publication sets formed by using UIDs as the match keys may affect the bibliometric indicators number of publications, number of citations, and the average number of citations per publication.  The objective is addressed in a literature review and a case study. The literature review shows that only a few studies evaluate how well UIDs work as a match key. From the literature we identify four error types: Duplicate digital object identifiers (DOI), incorrect DOIs in reference lists and databases, DOIs not registered by the database where a bibliometric analysis is performed, and erroneous optical or special character recognition. The case study explores the use of UIDs in the integration between the databases Pure and SciVal. Specifically journal publications in English are matched between the two databases. We find all error types except erroneous optical or special character recognition in our publication sets. In particular the duplicate DOIs constitute a problem for the calculation of bibliometric indicators as both keeping the duplicates to improve the reliability of citation counts and deleting them to improve the reliability of publication counts will distort the calculation of average number of citations per publication. The use of UIDs as a match key in citation linking is implemented in many settings, and the availability of UIDs may become critical for the inclusion of a publication or a database in a bibliometric analysis. Unique identifiers (UIDs) have been introduced for more and more entities, e.g. Open Researcher and Contributor ID (ORCID) for researchers, and digital object identifiers (DOI) for research publications, etc. One advantage of UIDs is that integrations between databases potentially can be done much more efficiently. This is stressed in a recent evaluation of metrics in research evaluations  with different terms for UID: unique, identifier, DOI, PMID. This gave us an idea of which databases use UIDs as match keys, e.g. CrossRef, Mendeley, and Altmetric.com. We also included these databases as search terms and combined them with the different terms for UIDs.https://login.webofknowledge.com/), Scopus (https://www.scopus.com/), and Google Scholar (https://scholar.google.dk/). No range of years was specified. If no relevant publications were found in WoS and Scopus, we continued the search in Google Scholar. This means that not only peer-reviewed research but also a few reports were included in the literature review. In relevant publications, we manually scanned references and citations for other relevant publications. The searches were done in August and September 2015 followed by later supplementary searches based on the references found in August and September.The searches were conducted in WoS  at University of Copenhagen (UCPH). DoCM registers approximately 2,000 research publications in the UCPH Pure database per year. The majority are peer-reviewed journal publications in English. As this type of publication and the health sciences are well-covered in Scopus/SciVal  and repeated in December 2015 (n=2066). It is possible for researchers and administrative staff to make retrospective changes to the registrations in Pure; this is the most plausible explanation for the lower number of publications in December.Data software• Pure local installation at University of Copenhagen, version 4.22.1 for the August download and version 4.23.1 for the December download (data download)• SciVal June 8, 2015, and September 30, 2015 releases • Microsoft Excel 2007 Data download, merging and cleaning• Organisational unit = Department of Clinical Medicine• Publication category = Research• Publication statuses and dates > Latest > Date: Selected range = 2014• Workflow = ValidatedRaw data was downloaded from Pure in August and December 2015 using the following filters:• Access to electronic version (full text) > DOI -0• journalAssociation.title• pages• persons[0].lastName• typeClassification.typeClassification• title• id [=Pure ID]• Source[sourceId]: PubMed [=PMID]• Source[sourceId]: Scopus [=EID]• language.languageTo fit relevant data in just one worksheet in Excel and be able to create a .csv file, most of the data columns were deleted, and only the following kept:Due to an error in the Copenhagen University Pure at the time, it was not possible to download a full data report of publications with the DOI column. Instead, first an ungrouped raw data report was downloaded, then the same report grouped on DOI. The two reports were matched on Pure ID to create one list with DOI data where available.The Data set 1 DoCM Pure data August.csv and Data set 2 DoCM Pure data December.csv files comprise our "raw" Pure data – ever so slightly tidied to a) create one full data report with DOI where available, b) fit relevant columns in one worksheet to be able to create a .csv file.• Removing superfluous spaces at the end of DOIs to be able to match DOIs in the Pure data with the DOIs in the SciVal data.The Pure "raw" data was furthermore cleaned by:• Title• Authors• Journal title• Citations• Pages• DOI• Publication-type• EID [=Scopus ID]• PubMed ID [=PMID]After the Pure data was sent to SciVal for analysis, the resulting SciVal publication sets (August and December) were downloaded from SciVal with the following information:The Data set 3 DoCM SciVal data August.csv and Data set 4 DoCM SciVal data December.csv files comprise our raw SciVal data.• Removing "2-s2.0-" from the EIDs to be able to match with the EIDs in the Pure data.• • Article vs. Article in Press (Article kept in data set)• Publication duplicates • Publication vs. publication attributed wrong ID in Scopus/SciVal and not occurring in the Pure data set (Publication in Pure data set kept)• Publication registered as one publication type vs. same publication registered as another publication type (duplicate with same publication type as in the Pure data set was kept)• Author's reply (not in Pure data set) having same DOI as the publication (in Pure data set) it relates to (publication in Pure data set kept).Duplicate DOIs were identified to remove superfluous/irrelevant publications:The SciVal raw data was furthermore cleaned by:1. Sometimes SciVal imports only the Article in Press instance of an article in Scopus (instead of the published article instance), or the article is registered in Scopus only as Article in Press, although it is published.2. During an automatic update in June 2015 of the UCPH Pure, a number of validated publications were changed from published in 2015 to published in 2014, although really they were published in 2015. As such, they should not have been part of our Pure publication set to begin with.A note on some Article in Press occurrences in the SciVal data:In the Results section, characteristics for three groups of publications are shown: Publications with UID exported from Pure to SciVal and matched, publications with UID exported from Pure to SciVal and not matched, and publications without UID not exported from Pure to SciVal. The publications without UID  were extracted from the cleaned Data set 1 and 2 with Pure data. To identify publications exported from Pure to SciVal and matched, we compared UIDs  in the cleaned Data set 3 with UIDs in the cleaned Data set 1 (August download). Publications in Data set 1 with no corresponding UID in Data set 3 constitute publications with UID exported from Pure to SciVal and not matched. This was repeated for Data set 2 and 4 (December download). The EIDs attributed automatically to publications in Pure were not visible in our raw data. We found 32 publications in Data set 4 from SciVal which must have an EID in Pure and SciVal as no other UID was assigned to them. Finally we compared UIDs in the cleaned Data set 3 and 4 to identify publications matched in SciVal in December but not in August.The literature review shows two trends. Firstly, the publication year of relevant studies is 2011 or later. Older UIDs such as arXiv ID and PMID do not seem to have the same momentum as DOI. Secondly, the use of UIDs as match keys in bibliometric studies and citation indexes seems under-reported. A possible explanation is that the commercial players do not publish their methodologies in full detail , p. 3. HIf we do not focus on the matching process in citation indexes and bibliometric analysis alone, we find an increasing number of tools for handling and analyzing research publications, e.g. CrossRef’s cited-by links  and Altmet al. (2014) evaluated the quality of Mendeley metadata by a comparison with Web of Science metadata. The DOIs did not match for 15 publications (8%). Missing DOI or erroneous special character recognition were identified as error types.We included 17 studies in the literature review. Two of the studies’ main focus was to evaluate the integration of UIDs in existing systems and are in that sense comparable to our objective. For a small publication set, ZahediThe rest of the studies used UIDs as match keys for different purposes and in doing so discussed how well the matching process worked. Different from our study, the main focus regarding the UIDs was the coverage. For all the studies, UIDs did not cover all publications potentially relevant for the analyses. Some studies only included publications with UIDs, and others supplemented the match keys with other types of bibliographic information to include publications without UIDs. We provide a short summary of how UIDs were used in the studies. Keep in mind that UIDs are used in many other studies and tools, but with very limited or no documentation and discussion of the UIDs.et al., 2012; Costaset al., 2014;The vast majority of the studies (ten) analyzed altmetric scores or altmetric methods. They used UIDs  sometimes in combination with other match key elements to match a publication set from WoS, Scopus, arXiv.org, a local database, or specific journals with identical publications in Mendeley, Almetric.com, Twitter, Wikipedia, Impact Story, CiteULike, BibSonomy, Connotea, or a selection of Elsevier databases authored by the Department of Clinical Medicine (DoCM) at the University of Copenhagen, published in 2014, and registered and validated in Pure. We evaluated the matching process based on information about coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match keys to detect errors, if any, in the matching process.The share of publications matched between Pure and SciVal, or the coverage, is 85.6% in August and 89.3% in December.There are precision issues for a minor part of the publication sets. Three of the error types reported by other studies  are alsoDuplicate DOIs : An automatic report from SciVal states that 1837 publications (August) and 1876 publications (December) are matched with our Pure publication sets. These numbers are inflated due to DOI duplicates . Both are indexed by Scopus. Interestingly, we seeUgeskrift for Læger represented among the matched publications, the unmatched publications with UID, and the unmatched publications without UID. AlsoPLOS ONE publications are among both the matched and the unmatched publications, but not among publications without a UID. Three of six unmatchedPLOS ONE publications from the August publication set are matched in December. The remaining threePLOS ONE publications were still not registered in Scopus in December.Clinical Nutrition . In addition to this, 35 publications from the August and December publication sets were from journals not indexed by Scopus according to Scopus’ Content Coverage Guide.Our analysis reveals more journal publications in English not matched in SciVal than we expected. Therefore we extracted lists of the top journals according to number of publications from our publication sets. For the unmatched publications a large share is published in the two journals of the Danish Medical Association . Almost all the matched publications have a DOI and are journal publications in English. Among the matched publications, 67 (3.8%) in the publication set from August have a duplicate DOI, whereas 32 (1.7%) from December do. Other error types  were obsUgeskrift for Læger of which the indexing in Scopus is highly irregular. Our analysis indicates that journals with publications in English also suffer from similar irregular indexing but to a much lesser extent.The unmatched publications also include journal publications. Close to half of these are in Danish and published in the journalThe matching of publications across databases can be used to create publication sets for bibliometric analysis. As an example we now briefly discuss how publication sets formed by using UIDs as match keys may affect the bibliometric indicators number of publications, number of citations, and the average number of citations per publication. This is to our knowledge only discussed briefly in the two studies. They both conclude that duplicate DOIs can lead to errors in bibliometric analysis  so inclusion of the missing publications would probably increase the number of citations a little, but lower the average number of citations per publication. However, inclusion of the missing publications for other journals could potentially have the opposite effect and increase the average number of citations per publication. An example isPLOS ONE .The coverage can affect bibliometric indicators. Results from our case study indicated that the majority of the publications from Pure are matched correctly in SciVal. Yet, the difference between the August and the December publication sets and the analysis of top journals  show thaThe precision of a bibliometric indicator is distorted by the fact that some DOIs are matched multiple times in SciVal. In most cases it is due to a duplicate of the same publication, but we also observed instances of publications in our Pure publication set with a DOI duplicate in SciVal not present in the Pure set . The dupThe number of publications becomes inflated by inclusion of duplicates. In our publication sets from August and December the publication count increased by 3.8% and 1.7%, respectively. Therefore we recommend that when the number of publications is calculated, duplicates should be removed whether the duplicate publication is in the original Pure publication set or not.Before citations are counted, all duplicates not present in the Pure publication set must be deleted. For the remaining duplicate pairs we found that sometimes both duplicates were cited independently. In all instances except one there was no overlap between the citations. Citations divided between duplicates in Scopus are also reported in another study where variations of a journal name results in duplicates in Scopus. It is suggested that databases like Scopus can improve verification of DOIs to solve the duplicate problem  at University of Copenhagen (UCPH) are provided. A description of each file is provided in ‘Dataset description’.Click here for additional data file.Copyright: © 2016 Madsen HH et al.2016Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).UIDs are seen as effective keys to match identical publications across databases or identify duplicates in a database. The use of UIDs as match keys is well-implemented in many settings but only few studies evaluate how UIDs work as match keys. As DOIs are implemented in more and more settings DOI also becomes increasingly interesting as a match key. According to the publication years of the studies in our literature review we suggest that this trend took off around 2010.Our case study confirms the findings of the literature review. UIDs as match keys do not return a 100% coverage of a publication set, and include errors for a small part of the matches. It is not possible to draw conclusions on when the coverage and precision is satisfactory as this should be discussed in relation to the purpose of a matching exercise, exemplified here as a bibliometric analysis.In our case study we identified irregular indexing of journals in Scopus as the main problem for an optimal coverage. Duplicate DOIs were a particular problem for the precision of UIDs as match keys. This type of error is easy to detect while other types of errors demand a more thorough analysis of the publication sets. This analysis could be done by using a traditional match key based on title, author name, etc. Other error types also present in our case study are: incorrect UIDs in reference lists and databases, and UIDs not registered by the database where a bibliometric analysis is performed.Match keys to detect identical publications across databases are used for many purposes, but our focus is bibliometric indicators. Here the duplicate DOIs constitute a problem as both keeping them in the publication set to improve the reliability of citation counts and deleting them to improve the reliability of publication counts will distort the calculation of average number of citations per publication and the many other bibliometric indicators which combine publication and citation counts. Also the coverage of a publication set can affect bibliometric indicators. We have discussed that failing to fully cover a low or high impact journal may also lead to imprecise bibliometric indicators.Our purpose has been to contribute to the discussion on how well UIDs work as match keys for publications with a focus on preparing publication sets for bibliometric analysis. Compared to traditional match keys where bibliographic information is used, UIDs are efficient, but they also have drawbacks.The coverage of UIDs is fully dependent on whether a UID is assigned to a publication, and its representations in publication lists and databases. Here the traditional match key has an advantage as it often is dependent on basic bibliographic data and can be modified to fit different formats. The traditional match key will probably have a good chance of retrieving all publications with a UID if the representations of the publications have basic bibliographic data of a fair quality. In addition, the traditional match key can retrieve publications without UIDs.The precision of UIDs depends on how carefully a UID is assigned to a publication and its representations in publication lists and databases. Using a single UID as a match key can be fragile as no crosschecks are made on other data fields. Detection of errors requires an examination of the result of the matching process. The traditional match key often relies on more data fields and thus has a built-in crosscheck. Neither of the match keys will solve the problem of duplicates of identical publications.We recommend more studies to be done on the pros and cons of UIDs because UIDs are being increasingly introduced for more entities and adopted as efficient match keys. The availability of UIDs may become critical for the inclusion of a publication or a database in a bibliometric analysis.The data referenced by this article are under copyright with the following copyright statement: Copyright: © 2016 Madsen HH et al.Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).10.5256/f1000research.8913.d126923  and so I approve of the article being indexed.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. I found the discussion of the citation linking throughout the article confusing, in part because my expectation of what that means is quite different than what is described in the article. My understanding of citation linking is that one is following works cited in a publication via the UID. It took several reads to clarify this; it may be something that the authors wish to clarify further in the abstract or introduction.I found the study design, methods, and conclusions to be adequate for the study particularly for the first objective . It would be useful to expand this study to other disciplines to see whether there are similar issues and characteristics. In order to replicate this study in another discipline, however, more attention may need to be paid to coverage in Pure and SciVal, or use of other databases may need to be deployed.The second objective- to better understand how bibliometric analysis of the publication set formed might be skewed by issues in matching- was undermined by the size of the dataset- there seemed to be too little data from which to draw firm conclusions. However, given that, the description of the results are fine. I believe that the Future Implications section provides the clearest implications of the study - essentially that UIDs aren't panaceas for deduplication/ citation matching/ etc.- and that use of UIDs as match points should be informed by potential points of failure. This description could have clarified other pieces of the article.This article describes a small study to better understand how unique identifiers (in this case DOIs) assigned to publications work as a match key for what the authors call 'citation linking' or matching identical publications in different databases in order to form a publication set that can then be used for bibliometric analysis. The study's goals were to lay out the characteristics of publications that were matched versus those that were unmatched, and to illustrate how the publications set formed might be affected by problems in the matching. I wavered between 'Approved' and 'Approved with Reservations', and settled on 'Approved'. I believe that the revisions necessary are really clarity of language and noting throughout that the size and scope of the case study provide a limited window.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Thank you for your review. You touch upon important points. Please see our response below.Sarah L. Shreeves: “I found the discussion of the citation linking throughout the article confusing, in part because my expectation of what that means is quite different than what is described in the article. My understanding of citation linking is that one is following works cited in a publication via the UID. It took several reads to clarify this; it may be something that the authors wish to clarify further in the abstract or introduction.”citation linkingThe term: In the literature we saw an inconsistent use of terms, but settled on "citation linking" to mean the matching of representations of identical publications across databases. We got similar comments from other reviewers and have edited our article, minimizing the term "citation linking", instead using "match key" and "matching process".Sarah L. Shreeves: “I found the study design, methods, and conclusions to be adequate for the study particularly for the first objective . It would be useful to expand this study to other disciplines to see whether there are similar issues and characteristics. In order to replicate this study in another discipline, however, more attention may need to be paid to coverage in Pure and SciVal, or use of other databases may need to be deployed.”Generalization: We have focused on Pure and SciVal in this study and plan to do the same in a follow-up study where we expand the data sets to cover more research fields within health sciences and investigate a new matching process introduced by Elsevier. We did not consider including other databases as the main focus in the present study is the matching process. Thus we have selected a case where we expected the coverage to be good. If other disciplines with a low coverage in SciVal were studied, we agree that it would be important to analyze and discuss the coverage in more detail. As pointed out in your review, additional databases could be considered.Sarah L. Shreeves: “The second objective- to better understand how bibliometric analysis of the publication set formed might be skewed by issues in matching- was undermined by the size of the dataset- there seemed to be too little data from which to draw firm conclusions. However, given that, the description of the results are fine.“Bibliometric indicators and the discussion of them are based on a limited data set. We agree: Our results apply to our case alone and may serve as inspiration for other similar studies. In version 2 of the article the part on bibliometric indicators is elaborated upon. See the subsectionResults of the case study in the paper, version 2.Sarah L. Shreeves: “I believe that the Future Implications section provides the clearest implications of the study - essentially that UIDs aren't panaceas for deduplication/ citation matching/ etc.- and that use of UIDs as match points should be informed by potential points of failure. This description could have clarified other pieces of the article.”Future Implications: In the sectionFuture Implications we discuss UIDs as match key compared to a traditional match key. We hope to address this in a follow-up study, following Elsevier changing the way publications are matched between (Scopus and) Pure and SciVal: Publications in Pure are matched with publications in Scopus using traditional match keys and assigned a Scopus ID of the matched publication. The match between Pure and SciVal now happens using Scopus ID as only match key.    Sarah L. Shreeves: “I wavered between 'Approved' and 'Approved with Reservations', and settled on 'Approved'. I believe that the revisions necessary are really clarity of language and noting throughout that the size and scope of the case study provide a limited window.” Again thank you for your review. We hope that our response and the edited paper have addressed your comments. Explore the coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match key.Illustrate how publication sets formed by using UIDs as the match key may affect the bibliometric indicators: Number of publications, number of citations and the average number of citations per publication. The paper usefully lists the common UIDs used for research publications, and in so doing illustrates the problem of disjoint UID sets (intersected by other attributes of the publication) and lack of universal uniqueness but without commenting upon it. An important facility in PURE (based on CERIF) is the ability to utilise 'federated IDs' so that there can be several 'unique identifiers' for the same object thus allowing crosswalking within the metadata for a given publication. The paper mentions UIDs for publications and persons but not for organisations; the use of such UIDs is becoming more common and is useful for comparative analysis of performance by research organisational units.This paper aims to contribute to the discussion on use of UIDs for citation linking. It states two objectives: The paper states (P3) "Opposite to the other UIDs, the DOI is assigned to the publication itself and not merely to the publication’s representation in a database". There is a lack of clarity of thinking here. The DOI is a digital identifier and thus is not assigned to the publication itself (the scholarly content) but to a digital representation  of it. The representation in the database is of two parts: the metadata  and the publication content . Throughout the paper it should be made clear that the work is based on the metadata - although the citation is within the textual content of the publication referring to a reference at the end of the publication from where a link can/should be made to the full text of that (cited) article. In fact ideally the link should have a time period of validity and semantics of the kind of citation . The paper states (P4) "The matching of identical publications in different databases is called citation linking or reference linking. Matching within the same database is called deduplication. The term citation matching is also used, but often for the more specific purposes where citing and cited publications are matched". It is important that the paper clarifies the thinking here also.  Citation has nothing to do with matching or de-duplication, although 'clean' data is required for effective and efficient citation. In fact the paper does not really discuss citation itself but only the utilisation of UIDs to achieve citation - and more particularly the difficulties of obtaining 'clean' data. The paper does not reference "Citation Linking: Improving Access to Online Journals"Proceedings of the 2nd ACM International Conference on Digital Libraries, edited by Robert B. Allen and Edie Rasmussen, 1997 , pp. 115-122 available at http://journals.ecs.soton.ac.uk/acmdl97.htm  which states clearly (and as early as 1997): S. Hitchcock*, L. Carr, S. Harris, J. M. N. Hey and W. Hall Garfield (1955). Garfield's description of citation indexing as an 'association of ideas' bears remarkable similarity to Bush's 'association of thoughts' which anticipated modern hypertext. Citation linking combines the two approaches, mapping both reference data and citation index data on to the text in the form of links. Adding electronic links to the literature is introducing a new culture in many respects, but citation linking is likely to be acceptable to the academic community because it builds on practices established in other forms such as print. It also allows the community to exploit its own intellectual input in this process, recognised in Garfield's original rationale that "by using authors' references in compiling the citation index, we are in reality utilizing an army of indexers"."   Here citation linking is used - and with precedence - to mean something quite different to the use in this paper.   "The likely effect of citation linking can be gauged by recognising a direct parallel with citation indexing, which is sometimes referred to as 'forward' referencing , developed by  The literature review is appropriate. The case study is limited in organisational unit, research discipline, time period; however the results are useful and could - with advantage - be compared with similar studies in other disciplines /organisational units and time periods.a paragraph clarifying the inter-relationships of metadata and content for one publication with that for another cited from the first;a paragraph explaining citation, citation linking, citation indexing;the existing paragraph on UIDs being expanded to explain the usefulness of 'federated IDs' for crosswalking within the metadata of a given publication. Overall the paper is a useful contribution and hopefully will stimulate similar studies in other disciplines and time ranges.  but the lack of clarity of thinking should be addressed. I would suggest:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Response to Marion Schmidt and Keith G. Jeffery Thank you for the valuable reviews. Below we quote the issues raised in the reviews followed by our response and description of how we have edited the paper. Please note that we have sent the same reply to both of you so you have the same information on how we have edited the paper.Marion Schmidt: “In the introduction and literature review, it should be made clearer that the authors use a specific definition of the term citation linking (linking items between databases) for their study and it should be clarified if other studies refer to the same or other scenarios (like reference matching or deduplication).”Keith G. Jeffery: “The paper states (P4) "The matching of identical publications in different databases is called citation linking or reference linking. Matching within the same database is called deduplication. The term citation matching is also used, but often for the more specific purposes where citing and cited publications are matched". It is important that the paper clarifies the thinking here also.  Citation has nothing to do with matching or de-duplication, although 'clean' data is required for effective and efficient citation. In fact the paper does not really discuss citation itself but only the utilisation of UIDs to achieve citation - and more particularly the difficulties of obtaining 'clean' data. The paper does not reference "Citation Linking: Improving Access to Online Journals" S. Hitchcock*, L. Carr, S. Harris, J. M. N. Hey and W. Hall Proceedings of the 2nd ACM International Conference on Digital Libraries, edited by Robert B. Allen and Edie Rasmussen, 1997 , pp. 115-122 available at http://journals.ecs.soton.ac.uk/acmdl97.htm  which states clearly (and as early as 1997): "The likely effect of citation linking can be gauged by recognising a direct parallel with citation indexing, which is sometimes referred to as 'forward' referencing , developed by Garfield (1955). Garfield's description of citation indexing as an 'association of ideas' bears remarkable similarity to Bush's 'association of thoughts' which anticipated modern hypertext. Citation linking combines the two approaches, mapping both reference data and citation index data on to the text in the form of links. Adding electronic links to the literature is introducing a new culture in many respects, but citation linking is likely to be acceptable to the academic community because it builds on practices established in other forms such as print. It also allows the community to exploit its own intellectual input in this process, recognised in Garfield's original rationale that "by using authors' references in compiling the citation index, we are in reality utilizing an army of indexers"."   Here citation linking is used - and with precedence - to mean something quite different to the use in this paper. […] I would suggest: […] 2. a paragraph explaining citation, citation linking, citation indexing;”citation linkingThe term has caused confusion as citation can be understood as the reference from one paper (citing document) to another paper (cited document). Also in the literature we see an inconsistent use of the term. By citation linking we have meant the linking of representations of identical publications across databases. We have had difficulties finding an unambiguous term for this and ended with citation linking. We now have minimized the use of the term citation linking. Instead we use match key and matching process.Marion Schmidt: “The sample is a convenience sample based only on one year and one subject field , with a predominantly journal- and English-based publication culture. The authors describe their first objective as "Explore the coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match key", after introducing the more general goal to investigate "how well UIDs work for citation linking" (p.3). In order to truly answer the goals and objectives, the hypothesis should be discussed and tested that implementation of UIDs may vary across disciplines, depending on the amount of e.g. smaller, regional journals and easy data import options for a system like Pure from, e.g. PubMed. In this perspective a careful sampling strategy representative for all subjects that are, to a varying extent, covered in the target database SciVal/Scopus as well as more publication years, would have been much more fruitful in order to assess the coverage and thus usability of UIDs in both systems. Thus, data from a university-wide implementation of Pure would make up a reasonable case study. The authors deal inconsistently with this issue as they try to rectify their sample on the one hand "As this type of publication and the health sciences are well-covered in Scopus/SciVal [...], we expected the DoCM publication set to be well-fitted for our purpose, namely to explore the citation linking process, rather than how well SciVal covers publications from a department" (p.4) - which is a non-adequate argument as the coverage could be considered separately. On the other hand, they concede that their "case study" may not lead to generalizable results and that results will therefore be compared to those from a literature review (p.4). […] I would strongly suggest expanding the initial sample to other fields.”Keith G. Jeffery: “The case study is limited in organisational unit, research discipline, time period; however the results are useful and could - with advantage - be compared with similar studies in other disciplines /organisational units and time periods.”The case we work with was deliberately chosen as the best scenario and not as a sample fit for generalization. We wanted to study how well UIDs work as match key in the integration between Pure and SciVal, so it seemed beneficial to work with a sample with as many UIDs registered as possible. Because of the attributes of publications from the Department of Clinical Medicine , they seemed likely to have a high number of UIDs registered. Furthermore, the integration between Pure and SciVal is relatively new and we only had limited knowledge about how it worked and found no evaluations of the integration in the literature. We know that publication sets from health sciences are well-covered by Scopus. With the potential good coverage between Pure and SciVal we hoped that our publication set would allow us primarily to focus on the matching itself. The coverage of SciVal/Scopus is reported in the paper, but this is supporting information. In the selection of the case we also took local conditions into account. Working with Pure data validated by the University Library, that is publications from 2014, we felt confident that the sample would have fewer errors regarding registration/correct registration of UIDs than samples from previous years where no central validation routines were in place. It is not possible for us to repeat our analysis with a case covering more research areas and more publication years, as the matching process for the integration between Pure and SciVal has changed. Now only Scopus ID is used as match key. Before, DOI and PubMed ID also worked as match keys. However, we are doing a follow-up study to see how the new matching process works for our original publication set and a new larger publication set.Marion Schmidt: “Thus, title and abstract information do - in my opinion - not reflect the actual limits of the study adequately.”The objective was formulated somewhat broadly and could give the impression that the study included a thorough analysis of the coverage of SciVal/Scopus and the study yielded results which could be generalized to other research fields. As explained above, this is not the case. We have specified the objective to clearly show that we evaluate how well UIDs work as match key. Our focus is the matching process and any errors in this process. Furthermore, we do only work with one case – the integration between Pure and SciVal for a publication set from health sciences. The case cannot be generalized to other research fields or settings. As a consequence of the reformulated objective, we have changed the title and abstract accordingly.Keith G. Jeffery: “The paper usefully lists the common UIDs used for research publications, and in so doing illustrates the problem of disjoint UID sets (intersected by other attributes of the publication) and lack of universal uniqueness but without commenting upon it. An important facility in PURE (based on CERIF) is the ability to utilise 'federated IDs' so that there can be several 'unique identifiers' for the same object thus allowing crosswalking within the metadata for a given publication. The paper mentions UIDs for publications and persons but not for organisations; the use of such UIDs is becoming more common and is useful for comparative analysis of performance by research organisational units. The paper states (P3) "Opposite to the other UIDs, the DOI is assigned to the publication itself and not merely to the publication’s representation in a database". There is a lack of clarity of thinking here. The DOI is a digital identifier and thus is not assigned to the publication itself (the scholarly content) but to a digital representation  of it. The representation in the database is of two parts: the metadata  and the publication content . Throughout the paper it should be made clear that the work is based on the metadata - although the citation is within the textual content of the publication referring to a reference at the end of the publication from where a link can/should be made to the full text of that (cited) article. In fact ideally the link should have a time period of validity and semantics of the kind of citation . […] I would suggest: 1. a paragraph clarifying the inter-relationships of metadata and content for one publication with that for another cited from the first; […] 3. the existing paragraph on UIDs being expanded to explain the usefulness of 'federated IDs' for crosswalking within the metadata of a given publication.”The description of a publication and its UIDs is elaborated. We did not analyze the full content of a publication but only parts of its metadata representations in Pure and SciVal. A publication can have more UIDs assigned to it in one or both of the databases. We assumed that each of the UIDs for a publication in a database represented the same publication. SeeIntroduction and the subsectionLimitations in the paper, version 2.Marion Schmidt: “On the other hand, they concede that their "case study" may not lead to generalizable results and that results will therefore be compared to those from a literature review (p.4). This claim, however, is not really fulfilled, as the studies mentioned in the literature review use UIDs for different purposes, constellations and databases. More importantly, no real comparison takes place, as the authors only recap "but all conclude that UIDs do not cover all records in the databases" (p.6). In order to contextualize their own results, concrete settings and results of other studies should be represented and discussed. […] In the introduction and literature review, it should be made clearer that the authors use a specific definition of the term citation linking (linking items between databases) for their study and it should be clarified if other studies refer to the same or other scenarios (like reference matching or deduplication).”Keith G. Jeffery: “The literature review is appropriate.”The literature review shows the big picture and now also includes more details on the studies included. None of the studies in the review were directly comparable to our study as we analyzed how well UIDs work as match key in the matching process between Pure and SciVal. We now show that the studies in the review used UIDs for many different matching processes. Only two of the studies focused on the evaluation of UIDs. Still, based on all the studies we summarized information on error types when UIDs are used as match keys, which is relevant for our study. See the subsectionResults of the literature review in the paper, version 2.Marion Schmidt: “With regard to the second purpose "Illustrate how publication sets formed by using UIDs as the match key may affect the bibliometric indicators: Number of publications, number of citations and the average number of citations per publication", the authors do not actually calculate citation indicators including and excluding publications , but discuss the problem mostly theoretically.”The number of citations was only calculated for the publications matched in SciVal at the time we did the exports from Pure to SciVal (August and December 2015). We did not have the number of citations for the other publications. This information is added to the article. For the matched publications we have added the number of citations including and excluding duplicates. See the subsectionResults of the case study in the paper, version 2.Marion Schmidt: “Methods and data (with exceptions mentioned separately below) are sufficiently clearly documented, but the whole study lacks generalizability and, partly, elaborateness of analyses, as in case of the citation indicator perspective. Besides, given the rather manageable amounts of unmatched publications in this study, a comprehensive and more elaborate search and analysis of the causes of the missed match  would be preferable. In larger corpora, this could be done via a random sample.”Causes for missed matches for publications with a UID were not investigated when we did the exports from Pure to SciVal at publication level but only at journal level (Table 7a-7c). We have looked into the question now, but many of the publications not matched in August and December 2015 are now matched. We added to the paper that from our analysis of journals (Table 7a-7c) it is likely that the publications not matched were not indexed in Scopus. We identified threePLOS ONE publications which were not indexed in Scopus, andClinical and Translational Allergy had only four publications from 2014 indexed in Scopus. But the missed matches can also be due to missing UIDs in Scopus. For 35 publications we established that they were published in journals not indexed in Scopus. This is also added to the paper. See the subsectionResults of the case study in the paper, version 2.Marion Schmidt: “The authors write "In the integration between Pure and SciVal DOI, PMID and EID are used as match keys for citation linking. From March 2016 the integration between Pure and SciVal is based on EID alone. This will not affect the present study as we analyze publication sets downloaded in August and December 2015" (p.3). The authors should discuss what this does mean with respect of the relevance of their results. Could it mean that the matching of current publications will be probably better?”The new matching process between Pure and SciVal means that our study is a documentation of the first matching processes between Pure and SciVal and can serve as background for new evaluations. We have added this point to the paper. We do not know if the new functionality is better than the ones we have evaluated but we are looking into that in a follow-up study. See theIntroduction in the paper, version 2  Marion Schmidt: “Regarding the publication type categorization: Has the categorization been informed by some available classification or, e.g. database scheme? In my opinion, some mappings are sub-optimal and particular; especially the assignment of Doctoral Thesis to "Other" instead of "Book", Encyclopedia chapter to "Other" instead of "Contribution to Book", Editorial to "Other" instead of "Journal", Article in Proceedings to "Journal" instead of "Contribution to Book" (please compare the WoS document type and publication type classification for the third and fourth case). Including so many and different publication types in a residual category is unprofitable for the later analysis.”The categorization of publication types has been reorganized according to the categories in Pure. See Table 5a-c.Marion Schmidt: “I did not understand the fact that EIDs seem to have been attributed automatically without being visible in the raw data. How?”The EID automatically attributed to publications in Pure was introduced with Pure version 4.23.0. EIDs are added to publications in Pure as an automatic job. In the beginning the job ran every day with a sub-set of the publications in order not to overload the matching service. 90 days after installation of the new version of Pure the job had covered all publications. Now when a new publications are added to Pure they will be included in the first coming match, hence within 14 days. The automatic attributed EIDs are not shown in our raw data (excel files downloaded from Pure). Therefore it is not possible to detect which publications are matched on the automatic attributed EIDs. More publications are matched in the November dataset, and we assume they are matched on the automatic attributed EID but cannot prove it. Once again thank you for the reviews. We hope that our response and the edited paper have addressed all your comments. The sample is a convenience sample based only on one year and one subject field , with a predominantly journal- and English-based publication culture. The authors describe their first objective as "Explore the coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match key", after introducing the more general goal to investigate "how well UIDs work for citation linking" (p.3). In order to truly answer the goals and objectives, the hypothesis should be discussed and tested that implementation of UIDs may vary across disciplines, depending on the amount of e.g. smaller, regional journals and easy data import options for a system like Pure from, e.g. PubMed. In this perspective a careful sampling strategy representative for all subjects that are, to a varying extent, covered in the target database SciVal/Scopus as well as more publication years, would have been much more fruitful in order to assess the coverage and thus usability of UIDs in both systems. Thus, data from a university-wide implementation of Pure would make up a reasonable case study. The authors deal inconsistently with this issue as they try to rectify their sample on the one hand "As this type of publication and the health sciences are well-covered in Scopus/SciVal [...], we expected the DoCM publication set to be well-fitted for our purpose, namely to explore the citation linking process, rather than how well SciVal covers publications from a department" (p.4) - which is a non-adequate argument as the coverage could be considered separately. On the other hand, they concede that their "case study" may not lead to generalizable results and that results will therefore be compared to those from a literature review (p.4). This claim, however, is not really fulfilled, as the studies mentioned in the literature review use UIDs for different purposes, constellations and databases. More importantly, no real comparison takes place, as the authors only recap "but all conclude that UIDs do not cover all records in the databases" (p.6). In order to contextualize their own results, concrete settings and results of other studies should be represented and discussed. With regard to the second purpose "Illustrate how publication sets formed by using UIDs as the match key may affect the bibliometric indicators: Number of publications, number of citations and the average number of citations per publication", the authors do not actually calculate citation indicators including and excluding publications , but discuss the problem mostly theoretically. I would strongly suggest expanding the initial sample to other fields. Thus, title and abstract information do - in my opinion - not reflect the actual limits of the study adequately. Methods and data (with exceptions mentioned separately below) are sufficiently clearly documented, but the whole study lacks generalizability and, partly, elaborateness of analyses, as in case of the citation indicator perspective. Besides, given the rather manageable amounts of unmatched publications in this study, a comprehensive and more elaborate search and analysis of the causes of the missed match  would be preferable. In larger corpora, this could be done via a random sample. In the introduction and literature review, it should be made clearer that the authors use a specific definition of the term citation linking (linking items between databases) for their study and it should be clarified if other studies refer to the same or other scenarios (like reference matching or deduplication). The authors write "In the integration between Pure and SciVal DOI, PMID and EID are used as match keys for citation linking. From March 2016 the integration between Pure and SciVal is based on EID alone. This will not affect the present study as we analyze publication sets downloaded in August and December 2015" (p.3). The authors should discuss what this does mean with respect of the relevance of their results. Could it mean that the matching of current publications will be probably better? Regarding the publication type categorization: Has the categorization been informed by some available classification or, e.g. database scheme? In my opinion, some mappings are sub-optimal and particular; especially the assignment of Doctoral Thesis to "Other" instead of "Book", Encyclopedia chapter to "Other" instead of "Contribution to Book", Editorial to "Other" instead of "Journal", Article in Proceedings to "Journal" instead of "Contribution to Book" (please compare the WoS document type and publication type classification for the third and fourth case). Including so many and different publication types in a residual category is unprofitable for the later analysis. I did not understand the fact that EIDs seem to have been attributed automatically without being visible in the raw data. How?I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Response to Marion Schmidt and Keith G. Jeffery Thank you for the valuable reviews. Below we quote the issues raised in the reviews followed by our response and description of how we have edited the paper. Please note that we have sent the same reply to both of you so you have the same information on how we have edited the paper.Marion Schmidt: “In the introduction and literature review, it should be made clearer that the authors use a specific definition of the term citation linking (linking items between databases) for their study and it should be clarified if other studies refer to the same or other scenarios (like reference matching or deduplication).”Keith G. Jeffery: “The paper states (P4) "The matching of identical publications in different databases is called citation linking or reference linking. Matching within the same database is called deduplication. The term citation matching is also used, but often for the more specific purposes where citing and cited publications are matched". It is important that the paper clarifies the thinking here also.  Citation has nothing to do with matching or de-duplication, although 'clean' data is required for effective and efficient citation. In fact the paper does not really discuss citation itself but only the utilisation of UIDs to achieve citation - and more particularly the difficulties of obtaining 'clean' data. The paper does not reference "Citation Linking: Improving Access to Online Journals" S. Hitchcock*, L. Carr, S. Harris, J. M. N. Hey and W. Hall Proceedings of the 2nd ACM International Conference on Digital Libraries, edited by Robert B. Allen and Edie Rasmussen, 1997 , pp. 115-122 available at http://journals.ecs.soton.ac.uk/acmdl97.htm  which states clearly (and as early as 1997): "The likely effect of citation linking can be gauged by recognising a direct parallel with citation indexing, which is sometimes referred to as 'forward' referencing , developed by Garfield (1955). Garfield's description of citation indexing as an 'association of ideas' bears remarkable similarity to Bush's 'association of thoughts' which anticipated modern hypertext. Citation linking combines the two approaches, mapping both reference data and citation index data on to the text in the form of links. Adding electronic links to the literature is introducing a new culture in many respects, but citation linking is likely to be acceptable to the academic community because it builds on practices established in other forms such as print. It also allows the community to exploit its own intellectual input in this process, recognised in Garfield's original rationale that "by using authors' references in compiling the citation index, we are in reality utilizing an army of indexers"."   Here citation linking is used - and with precedence - to mean something quite different to the use in this paper. […] I would suggest: […] 2. a paragraph explaining citation, citation linking, citation indexing;”citation linkingThe term has caused confusion as citation can be understood as the reference from one paper (citing document) to another paper (cited document). Also in the literature we see an inconsistent use of the term. By citation linking we have meant the linking of representations of identical publications across databases. We have had difficulties finding an unambiguous term for this and ended with citation linking. We now have minimized the use of the term citation linking. Instead we use match key and matching process.Marion Schmidt: “The sample is a convenience sample based only on one year and one subject field , with a predominantly journal- and English-based publication culture. The authors describe their first objective as "Explore the coverage, precision, and characteristics of publications matched versus not matched with UIDs as the match key", after introducing the more general goal to investigate "how well UIDs work for citation linking" (p.3). In order to truly answer the goals and objectives, the hypothesis should be discussed and tested that implementation of UIDs may vary across disciplines, depending on the amount of e.g. smaller, regional journals and easy data import options for a system like Pure from, e.g. PubMed. In this perspective a careful sampling strategy representative for all subjects that are, to a varying extent, covered in the target database SciVal/Scopus as well as more publication years, would have been much more fruitful in order to assess the coverage and thus usability of UIDs in both systems. Thus, data from a university-wide implementation of Pure would make up a reasonable case study. The authors deal inconsistently with this issue as they try to rectify their sample on the one hand "As this type of publication and the health sciences are well-covered in Scopus/SciVal [...], we expected the DoCM publication set to be well-fitted for our purpose, namely to explore the citation linking process, rather than how well SciVal covers publications from a department" (p.4) - which is a non-adequate argument as the coverage could be considered separately. On the other hand, they concede that their "case study" may not lead to generalizable results and that results will therefore be compared to those from a literature review (p.4). […] I would strongly suggest expanding the initial sample to other fields.”Keith G. Jeffery: “The case study is limited in organisational unit, research discipline, time period; however the results are useful and could - with advantage - be compared with similar studies in other disciplines /organisational units and time periods.”The case we work with was deliberately chosen as the best scenario and not as a sample fit for generalization. We wanted to study how well UIDs work as match key in the integration between Pure and SciVal, so it seemed beneficial to work with a sample with as many UIDs registered as possible. Because of the attributes of publications from the Department of Clinical Medicine , they seemed likely to have a high number of UIDs registered. Furthermore, the integration between Pure and SciVal is relatively new and we only had limited knowledge about how it worked and found no evaluations of the integration in the literature. We know that publication sets from health sciences are well-covered by Scopus. With the potential good coverage between Pure and SciVal we hoped that our publication set would allow us primarily to focus on the matching itself. The coverage of SciVal/Scopus is reported in the paper, but this is supporting information. In the selection of the case we also took local conditions into account. Working with Pure data validated by the University Library, that is publications from 2014, we felt confident that the sample would have fewer errors regarding registration/correct registration of UIDs than samples from previous years where no central validation routines were in place. It is not possible for us to repeat our analysis with a case covering more research areas and more publication years, as the matching process for the integration between Pure and SciVal has changed. Now only Scopus ID is used as match key. Before, DOI and PubMed ID also worked as match keys. However, we are doing a follow-up study to see how the new matching process works for our original publication set and a new larger publication set.Marion Schmidt: “Thus, title and abstract information do - in my opinion - not reflect the actual limits of the study adequately.”The objective was formulated somewhat broadly and could give the impression that the study included a thorough analysis of the coverage of SciVal/Scopus and the study yielded results which could be generalized to other research fields. As explained above, this is not the case. We have specified the objective to clearly show that we evaluate how well UIDs work as match key. Our focus is the matching process and any errors in this process. Furthermore, we do only work with one case – the integration between Pure and SciVal for a publication set from health sciences. The case cannot be generalized to other research fields or settings. As a consequence of the reformulated objective, we have changed the title and abstract accordingly.Keith G. Jeffery: “The paper usefully lists the common UIDs used for research publications, and in so doing illustrates the problem of disjoint UID sets (intersected by other attributes of the publication) and lack of universal uniqueness but without commenting upon it. An important facility in PURE (based on CERIF) is the ability to utilise 'federated IDs' so that there can be several 'unique identifiers' for the same object thus allowing crosswalking within the metadata for a given publication. The paper mentions UIDs for publications and persons but not for organisations; the use of such UIDs is becoming more common and is useful for comparative analysis of performance by research organisational units. The paper states (P3) "Opposite to the other UIDs, the DOI is assigned to the publication itself and not merely to the publication’s representation in a database". There is a lack of clarity of thinking here. The DOI is a digital identifier and thus is not assigned to the publication itself (the scholarly content) but to a digital representation  of it. The representation in the database is of two parts: the metadata  and the publication content . Throughout the paper it should be made clear that the work is based on the metadata - although the citation is within the textual content of the publication referring to a reference at the end of the publication from where a link can/should be made to the full text of that (cited) article. In fact ideally the link should have a time period of validity and semantics of the kind of citation . […] I would suggest: 1. a paragraph clarifying the inter-relationships of metadata and content for one publication with that for another cited from the first; […] 3. the existing paragraph on UIDs being expanded to explain the usefulness of 'federated IDs' for crosswalking within the metadata of a given publication.”The description of a publication and its UIDs is elaborated. We did not analyze the full content of a publication but only parts of its metadata representations in Pure and SciVal. A publication can have more UIDs assigned to it in one or both of the databases. We assumed that each of the UIDs for a publication in a database represented the same publication. SeeIntroduction and the subsectionLimitations in the paper, version 2.Marion Schmidt: “On the other hand, they concede that their "case study" may not lead to generalizable results and that results will therefore be compared to those from a literature review (p.4). This claim, however, is not really fulfilled, as the studies mentioned in the literature review use UIDs for different purposes, constellations and databases. More importantly, no real comparison takes place, as the authors only recap "but all conclude that UIDs do not cover all records in the databases" (p.6). In order to contextualize their own results, concrete settings and results of other studies should be represented and discussed. […] In the introduction and literature review, it should be made clearer that the authors use a specific definition of the term citation linking (linking items between databases) for their study and it should be clarified if other studies refer to the same or other scenarios (like reference matching or deduplication).”Keith G. Jeffery: “The literature review is appropriate.”The literature review shows the big picture and now also includes more details on the studies included. None of the studies in the review were directly comparable to our study as we analyzed how well UIDs work as match key in the matching process between Pure and SciVal. We now show that the studies in the review used UIDs for many different matching processes. Only two of the studies focused on the evaluation of UIDs. Still, based on all the studies we summarized information on error types when UIDs are used as match keys, which is relevant for our study. See the subsectionResults of the literature review in the paper, version 2.Marion Schmidt: “With regard to the second purpose "Illustrate how publication sets formed by using UIDs as the match key may affect the bibliometric indicators: Number of publications, number of citations and the average number of citations per publication", the authors do not actually calculate citation indicators including and excluding publications , but discuss the problem mostly theoretically.”The number of citations was only calculated for the publications matched in SciVal at the time we did the exports from Pure to SciVal (August and December 2015). We did not have the number of citations for the other publications. This information is added to the article. For the matched publications we have added the number of citations including and excluding duplicates. See the subsectionResults of the case study in the paper, version 2.Marion Schmidt: “Methods and data (with exceptions mentioned separately below) are sufficiently clearly documented, but the whole study lacks generalizability and, partly, elaborateness of analyses, as in case of the citation indicator perspective. Besides, given the rather manageable amounts of unmatched publications in this study, a comprehensive and more elaborate search and analysis of the causes of the missed match  would be preferable. In larger corpora, this could be done via a random sample.”Causes for missed matches for publications with a UID were not investigated when we did the exports from Pure to SciVal at publication level but only at journal level (Table 7a-7c). We have looked into the question now, but many of the publications not matched in August and December 2015 are now matched. We added to the paper that from our analysis of journals (Table 7a-7c) it is likely that the publications not matched were not indexed in Scopus. We identified threePLOS ONE publications which were not indexed in Scopus, andClinical and Translational Allergy had only four publications from 2014 indexed in Scopus. But the missed matches can also be due to missing UIDs in Scopus. For 35 publications we established that they were published in journals not indexed in Scopus. This is also added to the paper. See the subsectionResults of the case study in the paper, version 2.Marion Schmidt: “The authors write "In the integration between Pure and SciVal DOI, PMID and EID are used as match keys for citation linking. From March 2016 the integration between Pure and SciVal is based on EID alone. This will not affect the present study as we analyze publication sets downloaded in August and December 2015" (p.3). The authors should discuss what this does mean with respect of the relevance of their results. Could it mean that the matching of current publications will be probably better?”The new matching process between Pure and SciVal means that our study is a documentation of the first matching processes between Pure and SciVal and can serve as background for new evaluations. We have added this point to the paper. We do not know if the new functionality is better than the ones we have evaluated but we are looking into that in a follow-up study. See theIntroduction in the paper, version 2  Marion Schmidt: “Regarding the publication type categorization: Has the categorization been informed by some available classification or, e.g. database scheme? In my opinion, some mappings are sub-optimal and particular; especially the assignment of Doctoral Thesis to "Other" instead of "Book", Encyclopedia chapter to "Other" instead of "Contribution to Book", Editorial to "Other" instead of "Journal", Article in Proceedings to "Journal" instead of "Contribution to Book" (please compare the WoS document type and publication type classification for the third and fourth case). Including so many and different publication types in a residual category is unprofitable for the later analysis.”The categorization of publication types has been reorganized according to the categories in Pure. See Table 5a-c.Marion Schmidt: “I did not understand the fact that EIDs seem to have been attributed automatically without being visible in the raw data. How?”The EID automatically attributed to publications in Pure was introduced with Pure version 4.23.0. EIDs are added to publications in Pure as an automatic job. In the beginning the job ran every day with a sub-set of the publications in order not to overload the matching service. 90 days after installation of the new version of Pure the job had covered all publications. Now when a new publications are added to Pure they will be included in the first coming match, hence within 14 days. The automatic attributed EIDs are not shown in our raw data (excel files downloaded from Pure). Therefore it is not possible to detect which publications are matched on the automatic attributed EIDs. More publications are matched in the November dataset, and we assume they are matched on the automatic attributed EID but cannot prove it. Once again thank you for the reviews. We hope that our response and the edited paper have addressed all your comments.


One of the major causes of damage to column-supported concrete structures, such as bridges and highways, are collisions from moving vehicles, such as cars and ships. It is essential to quantify the collision damage of the column so that appropriate actions can be taken to prevent catastrophic events. A widely used method to assess structural damage is through the root-mean-square deviation (RMSD) damage index established by the collected data; however, the RMSD index does not truly provide quantitative information about the structure. Conversely, the damage volume ratio that can only be obtained via simulation provides better detail about the level of damage in a structure. Furthermore, as simulation can also provide the RMSD index relating to that particular damage volume ratio, the empirically obtained RMSD index can thus be related to the structural damage degree through comparison of the empirically obtained RMSD index to numerically-obtained RMSD. Thus, this paper presents a novel method in which the impact-induced damage to a structure is simulated in order to obtain the relationship between the damage volume ratio to the RMSD index, and the relationship can be used to predict the true damage degree by comparison to the empirical RMSD index. In this paper, the collision damage of a bridge column by moving vehicles was simulated by using a concrete beam model subjected to continuous impact loadings by a freefalling steel ball. The variation in admittance signals measured by the surface attached lead zirconate titanate (PZT) patches was used to establish the RMSD index. The results demonstrate that the RMSD index and the damage ratio of concrete have a linear relationship for the particular simulation model. Collisions from moving vehicles can cause severe damage to column-supported concrete structures. From 1981 to 1990, 2418 bridges in the United States were hit by vessels .y/RxR to predict the changes in amount of damage in reinforced concrete slabs, yet the values are small compared with that of RMSD. Therefore, most of the research utilizes RMSD-based tools to estimate the damage degree of concrete structures under different load types.In recent years, lead zirconate titanate (PZT), as a smart material, has been widely used in the area of structural health monitoring (SHM) owing to its low cost, quick response, high stability, and dual sensing and actuation capacities. PZT sensors have been successfully applied in concrete structures to monitor the strength change, damage development, crack propagation, and impact location . Based oPractical applications of the RMSD index for damage evaluation require an understanding of the influence of the damage on the signal response of bonded PZT sensors. Narayanan and Subramaniam  decoupleThis paper presents a novel method in which the impact-induced damage to a structure is simulated in order to obtain the relationship between the damage volume ratio to the RMSD index, and the relationship can be used to predict the true damage degree by comparison to the empirical RMSD. Collision damage of a bridge column by moving vehicles is simulated by using a concrete beam model subjected to continuous impact loadings by a freefalling steel ball. A series of numerical analyses are implemented to monitor the damage growth of the investigated model under impact loadings. The damage volume ratio obtained from the numerical results is utilized to quantify the damage degree of the numerical model before and after impact poundings. A linear relationship is found between the numerically-obtained RMSD index and the damage volume ratio for the particular simulation model. The proposed method could help better understanding of the damage degree for real structures by comparing the empirically-obtained RMSD index to the numerical results.The EMI theoretical model was proposed by Liang et al.  and subsk =In the single degree-of-freedom mass-stiffness-damping system, the electrical admittance equation :(1)Y=IoVHowever, the above mentioned one-dimensional electro-mechanical system considers only the one-dimensional extensional actuation, ignoring the longitudinal actuations. Annamdas and Soh ,43,44 prAs Equations (1) and (2) show, the admittance signature gauged by a PZT patch is related not only to the mechanical impedance and material properties of the PZT patch, but also to the mechanical impedance of the host structure. Assuming that mechanical impedance and material properties of the PZT patch are constant, any changes in the mechanical impedance of the host structure are reflected in the admittance measurement. When concrete is damaged, the Young’s modulus of the concrete decreases, causing the modification in the structural stiffness. According to Equation (1), measurements of electrical admittance or impedance before and after damage can be qualitatively assessed.The three-dimensional finite element model is illustrated in 3. In order to simulate the contact behavior between the bottom point of the falling ball and the top surface of the concrete specimen, as well as the interfaces between the concrete beam and rigid supports, a general constraint algorithm that was frictionless in the tangential behavior and hard contact in normal behavior was used.The finite element model consists of 62,416 nodes and 56,970 elements. The minimum element size is 1 mm and the maximum element size is 10 mm. The middle zone of the concrete beam was meshed with refined grids to simulate the damage procedure accurately. The supports, ball, and concrete beam are modeled using C3D8R elements, which are eight-node linear brick, reduced-integration elements for three-dimensional cases. The PZT patches were simulated using the eight-node linear piezoelectric elements (C3D8E) which are three-dimensional multi-physics coupled-field finite elements that couple electrical conduction with stress analysis. The supports and ball were modelled as steel. The steel material has Young’s modulus of 210 GPa, a Poisson’s ratio of 0.3, and a density of 7800 kg/mThe concrete material of the specimen utilized the plastic-damage constitutive model embedded in the software ABAQUS, which is a product of Dassault Systèmes Simulia Corp., Providence, RI, USA. The concrete-damaged plasticity model implemented in the software ABAQUS was based on the constitutive relationships proposed by Lublinear et al. . To simud. Accordingly, the relationship of Cauchy stress Concrete experiences damage when the stress level reaches the failure stress. The damage causes the softening of the concrete material stiffness. Considering that concrete destruction is mainly caused by tensile damage, only the role of tensile damage was considered in this section. The damage can be expressed by the damage factor The plastic flow potential function and yield surface make use of two stress invariants of the effective stress tensor, namely the hydrostatic pressure stress As for as effective stresses, the yield function equation takes the following form:The concrete damage plasticity model presumes non-associated potential plastic flow:G used for this model is the Drucker-Prager hyperbolic function:The flow potential 3, average compressive strength The main mechanical properties of concrete used in this paper were: average density Damage factor is a commonly-used index to present the damage of concrete material, which is defined as the scalar stiffness degradation in ABAQUS software. The impact damage mode of the specimen using the concrete plastic-damage constitutive model was derived from the numerical simulation, as shown in In order to ensure the detection sensitivity of the EMI-based technique, the conductance  was scanned with a wide sweep range from 1 kHz to 1 MHz to confirm a suitable frequency range at the beginning of the numerical experiment. It is obvious that the conductance signature has a more direct relation to the resonant frequency than admittance signature and susceptance signature. This is the reason why conductance was chosen for this scan. The test results of the conductance spectra are plotted in The admittance signatures of PZT-R (the one bonded on the right surface) are shown in d > 0.7 was used to depict the local damage of the concrete beam, which can be observed in To quantitatively depict the emergence and development of the damage of a concrete beam under impact loading, the damage volume ratio (DVR), as an evaluating indicator of the concrete structure, is proposed, which can be calculated by the weighted average method given as:ith measurement point, and ith measurement point for each kth load case.The RMSD index is used to calculate the difference between values of baseline measurement of the admittance signature and the signature in the different damage status. As mentioned above, it can be used as a damage indicator to quantitatively estimate the difference in the admittance signatures before and after the damage. The mathematical expression of the metric in terms of the real part of the electrical admittance given in  as the fAll of these parameters’ variations and analyses in the previous section demonstrate the impact damage evolution to some extent. However, the results from the conductance signatures are, so far, qualitative. To account for the variations in the admittance curves during the repeated impact loading, the RMSD index is applied to evaluate the impact damage quantitatively, as shown in In order to further associate the RMSD of admittance signatures with the damage level of the concrete beam, the RMSD-based damage volume ratio is proposed. The two kinds of damage volume ratio, global damage, and local damage, are chosen to build the relationship with the RMSD of the admittance signals of PZT-F, as depicted in d > 0.7 was chosen to build relationships with the RMSD of admittance signatures of PZT-F, as depicted in To further understand the relationship between the RMSD of the admittance signatures and the damage level of the concrete beam, the RMSD-based damage volume ratio is proposed. As discussed above, the damage value of 0.7 is a crucial threshold which demonstrates the emergence of slight damage and the initial visible crack of the concrete. Thus, the damage volume ratio for To verify the accuracy of the fitted equation in The verification results are found to be in good agreement with the fitted curves from In this paper, the applicability and feasibility of an EMI-based damage monitoring technique for impact damage detection in the concrete structure was investigated. Numerical simulations of impact events on a concrete beam using the concrete damaged plasticity constitutive model implemented in ABAQUS was performed. There is a general agreement that the damage volume ratio of concrete can be estimated using the RMSD index based on changes of the electro-mechanical impedance signatures. Thus, the alterations in the admittance or impedance spectra at different impact times are denoted to be an indicator of the impact damage. All the numerical results demonstrate that the statistical damage metric RMSD-based damage volume ratio can be used to quantify the variations in the signatures of admittance and evaluate the damage degree of the concrete beam. According to the linear relationship between the damage volume ratio and the RMSD index, the EMI-based approach shows promising results for estimating the impact damage of a concrete structure through inverse computation based on the RMSD index. It has remarkable potential to assess the damage level and health condition of a concrete structure. In future works, the influence of the distance between the sensor and damage zone on the sensitivity and accuracy of the results will be studied. In addition, the concepts developed herein will be experimentally investigated to the monitoring of the concrete piers in a real environment.