British Medical Journal and eLife to study the content of review reports, editorial decisions, author responses, and open reader comments. It finds no clear differences between the review processes of articles not related to Covid-19 published during or before the pandemic. However, it does find notable diversity between Covid-19 and non-Covid-19-related articles, including fewer requests for additional experiments, more cooperative comments, and different suggestions to address too strong claims. In general, the findings suggest that both reviewers and journal editors implicitly and explicitly use different quality criteria to assess Covid-19-related manuscripts, hence transforming science’s main evaluation mechanism for their underlying studies and potentially affecting their public dissemination.The global Covid-19 pandemic has had a considerable impact on the scientific enterprise, including scholarly publication and peer-review practices. Several studies have assessed these impacts, showing among others that medical journals have strongly accelerated their review processes for Covid-19-related content. This has raised questions and concerns regarding the quality of the review process and the standards to which manuscripts are held for publication. To address these questions, this study sets out to assess qualitative differences in review reports and editorial decision letters for Covid-19 related, articles not related to Covid-19 published during the 2020 pandemic, and articles published before the pandemic. It employs the open peer-review model at the  In fact, several studies demonstrated these efforts to be highly effective, with average time between submission and publication being reduced by nearly 50% effectiveness of certain drugs to cure—or reduce negative effects of—Covid-19.However, new and even louder concerns have been raised during the current pandemic, among others after the quick spread of news about several high-impact studies, in well-reputed journals, which had to be retracted following evidence of research misconduct or the unavailability of underlying data . ExampleIn this article, we will build on earlier work regarding the peer-review process in times of the global pandemic . Rather Manuscript peer review serves many purposes, several of which have been in flux over the past decades with new expectations of the system appearing. Traditionally, the system of journal peer review has been expected to serve primarily as a gatekeeping mechanism, filtering fit from unfit manuscripts for the limited space available in journals’ print copies, as well as a mechanism to improve the quality of a manuscript based on reviewers’ feedback . ThroughTied to these changing expectations, the scholarly community has witnessed a diversification of review models. By now, a host of different ways of performing manuscript peer review is installed at different journals and publishers . Most noStudies of peer review, both as a formal process or system and as a daily scholarly practice, have a long tradition . ScholarWhile some evidence is gathered on the procedural level, documenting review procedures and formal regulations, little study has gone into the actual practice of review, nor into the content of individual review reports. Part of this has certainly been due to a lack of data availability; with review commonly being an opaque practice, performed by individual researchers and not shared with the wider public or research community. This has changed due to the establishment of open peer-review models and increasingly many journals adhering to systems in which review reports are publicly shared. Such systems now warrant studies of peer review’s content that were previously particularly challenging to perform. In addition, the collaboration between researchers and academic publishers has allowed some to peak into the review process of certain journals e.g. .Taking advantage of these open review formats, some studies have aimed to classify reviewer comments, using quantitative and qualitative methods to distinguish between various types of content in review reports . The majTwo notable exceptions to this are the recent studies by British Medical Journal (BMJ) and the online journal eLife, for three distinct categories. Both journals constitute prominent journals in their field, with the BMJ having a long tradition in publishing in Medicine, and eLife being a relatively young life science journal. While BMJ was one of the first journals to install a formal system using external reviewers at the end of the 19th century  not related to Covid-19. Again, relatedness to Covid-19 was based on the classification of the journal itself, in case of BMJ and eLife before 23rd April 2019. In the remainder of this text, this category will be referred to as the pre-Covid.Articles published before the Covid-19 pandemic emerged. For this category, we sampled the most recently published articles as of exactly 1 year before the sampling date, that is, the most recently published articles in BMJ’s and CORD19’s classification, rather than assessment by the authors, to increase transparency and replicability of the sampling strategy.For all categories, five articles per journal were included in the sample, that is, 30 articles in total. Because only four articles were published on Covid-19 in BMJ at the sampling date, the fifth article was added to this category once it was published on 5th May. Only articles classified as ‘Research articles’ by the respective journals were eligible for inclusion in the sample. The classification of Covid-19 relatedness of the articles was based on the BMJ, we also gathered the commentaries published as readers’ responses to the articles. These commentaries were sampled on 29th April 2020 (and at 12th May for the article added on 5th May to the sample of Covid-papers). It should be noted that eLife does not publish the full review reports but merely summaries of the most essential comments and feedback.For all sampled articles, we subsequently gathered all available information and documentation regarding the review process. This particularly included: bibliographic information of the articles, the articles’ dates of submission and publication, the review reports, editorial decision letters based on these review reports, author responses to review reports, the number of peer reviewers involved, the number of review rounds a paper went through, the average length of review reports, and whether articles had gone through previous rounds of review at other journals. The latter information was retrieved from the editors’ decision letters. For the An overview of the sampled articles including some basic information on their review process can be found in BMJ were coded in the same way, as were the editorial decision letters based on review reports. Following up on this coding, the materials and their codes were read through yet another time to fine-tune coding and the increase consistency across coding. Subsequently, comparisons between article categories were conducted and apparent differences across categories were further scrutinized. No dedicated coding software was used. As this manuscript is primarily interested in the qualitative differences in peer-review processes brought about by the global pandemic, we refrain from quantitative analyses of our codes. In fact, sample sizes are too small to infer any meaningful statements from such statistical analyses, with characteristics of individual reviewers or manuscripts likely accounting for a substantial share of any variation found.Upon collecting the material, all material was read and manually coded using an inductive coding scheme loosely inspired by the classification of We start off this section with some brief numerical characteristics of the review process. The number of reviewers per paper is largely consistent between the Covid and the other categories, with on average 3.3 reviewers per paper for the Covid sample and 3.45 reviewers per paper for the other two categories. Some further characteristics regarding the duration of the review process as well as the average length of review reports are provided in Even though review reports were hence delivered much faster, they, on average, seem not to be less elaborate. The length of review reports, in number of words, is fairly stable across categories. In fact, as manuscripts in the Covid-category tend to be shorter than those in other categories and longer manuscripts may be expected to yield longer review reports, the average length of reports per article page is even much higher for the Covid-category than for the other categories.In this section, we will dive deeper into the qualitative content of the review reports. In particular, four aspects showing remarkable differences between the categories will be highlighted: (1) requests for additional experiments, (2) suggestions for alternative manuscript structures, (3) demands to tone down conclusions, and (4) the general nature of comments made. First and foremost however, we note that the general structure of review reports is largely stable across the three sample categories . As was noticed before, amongst others by eLife, non-Covid-category)‘Regarding selectivity of MtTMEM175, the authors arrive at a mechanistic conclusion that differs completely from the earlier conclusion by the Jiang group, but they do not test the effect on selectivity of mutating the residue (L35) proposed to play a key role in the earlier study. Selectivity of the L35A mutant should be studied and the results included in the interpretation’.  experiments. A typical example of such a comment pointing to the need for additional control experiments is:BMJ, Covid-category)‘We would like you to address this concern in greater depth, and consider what if any additional measures could be (quickly) incorporated to improve confidence in the medical history .’ In a nutshell, gathering clinical data during an outbreak of a new pathogen is of high importance. The manuscript should benefit from a revision before publication. If data are available at the time of the revision and if the authors chose to review their paper, the paper could be updated with new available data on the outcome of the patients in their cohort. Our conclusion is that it does expand on your previous work, perhaps not to the extent that we usually like to see in such papers. However, in light of the importance of the topic vis-vis the ongoing COVID-19 epidemic, we intend to retain your paper for publication in In some cases, the editors or reviewers even specifically admit that they lower their standards due to the importance of communicating novel findings that might be relevant in the pandemic. This is for instance the case, when Another way in which comments relating to the need of additional data or experiments were differently expressed in the Covid-category compared to the other two categories, was through a larger share of comments asking for additional discussion, toning-down of conclusions, or acknowledging limitations. We will return to this topic in Section 4.2.3.BMJ, Covid-category)the main limitation of this manuscript is the limited information that is provided on the medication used in the different patient groups. There was a very high usage of steroids and antiviral drugs, which both could have dramatic effects on viral dynamics and clinical outcomes. The authors do mention this in their limitation section, but in my view, they need to include more data on the different drugs used in their results section. The authors only gave brief description of different characteristics. Statistical analysis should be performed with statistics calculated and p values listed in each table. Although the whole-cell currents of MtTMEM175A recorded from HEK293 cells appear large compared to mock-transfected cells and a point mutation (T38A) has changed ion selectivity of the channel, it would still be interesting to know whether the authors have ever confirmed expression of the protein on a biochemical levelInstead of requesting for additional data or experiments, reviewers and editors seemed to be just as likely to ask for clarification of existing methodology and data collection. For all three categories, such requests were among the most prevalent comments in review reports and editorial decision letters. These requests also include the more elaborate reporting for  variables and collected data. Illustrative examples include:eLife, non-Covid-category)We believe that more efforts are necessary to communicate the results and interpretations to the general reader who is less of an expert in the details of the specific technology and methodologies used here. A substantial revision of the manuscript structure and style is required. While the study includes an impressive volume of detailed, quantitative analyses, the manuscript suffers from poor organization and lack of structure, which makes it challenging for the reader to tease apart the real significance of the authors' contributions. You present a large number of laboratory parametres [sic] in your text. Given the large volume, I would encourage you to report only on the parametres [sic] that are significantly different between the two groups or that are clinically surprising, and refer your reader to the table that captures the rest. He [one of the statistical reviewers] and others also wondered whether this should also be a ‘Living Review’ - you would need to include some methodology about this, e.g. when you would update. The timing and types of drugs used must affect the interpretation of viral load and shedding times. Severe patients were more likely to be on steroids and have higher viral load; which came first? If you can't describe and analyse this in more detail then you should discuss this more.  experiments in the non- or pre-Covid categories. However, in this case of a Covid paper, the reviewer suggests the topic to be ‘clearly addressed in the Discussion’:eLife, Covid-category)There are several pieces of data in the manuscript that indicate that EsxA may not be the only factor, or that there may be additional receptors for EsxA on M cells. For example, […]. This possibility should be clearly addressed in the Discussion. We also note the low quality of most of the studies in the review. Many editors were concerned that the studies may not translate well to other settings. Perhaps that is one of the important conclusions of this review, and you should be more forceful in urging scepticism [sic] and caution. In general, the manuscript suffers unclear and complex wording; the introduction is inflated, providing many dissociated ideas and concepts without a clear narrative. The results section is overly long and detailed, while some analyses are missing and some could be moved to supplementary section (for the benefit of experts).  review processes might have changed during the Covid pandemic. A first, and by far most prominent difference, can be found in the number of such responses posted. At the moment of data collection, the articles in the Covid-category yielded an order of magnitude more responses since their publication (ranging from a week to two months before data collection) than the articles in the pre-Covid-category (being published more than a year ago). In comparison, the non-Covid category received hardly any responses at the time of data collection, with only two out of five articles receiving at least one commentary. Also, responses tended to be much more elaborate in the Covid category, commenting on a wide range of issues, rather than pointing out single concerns or potential mistakes. Even though reader comments were coded in the same way as review reports and editorial decision letters, we will refrain from presenting any quantitative data on this coding, due to the small number of reader commentaries available, especially for the non-Covid category. Instead, the coding will be used to inform a general discussion of some overall patterns in the post-publication response letters.The open reader responses to published articles in A particularly prominent feature of reader responses to Covid-19-related articles was the connection they tend to make to the broader literature. In their responses, readers commonly linked a study’s findings to other research, potentially published after the study being commented on. By doing so, they actively discuss the study findings, provide alternative interpretations, or point to a study’s wider relevance. Other frequently occurring comments in readers’ responses include requests for further clarification or suggestions for future research building on the current study. As such, reader responses for this category tend to be rather neutral in their tone. A last prevalent type of comment among responses to Covid-19 papers comprises more general discussions or thoughts on the pandemic, apparently only loosely inspired by the manuscript responded to.In contrast, responses to articles in the pre- and non-Covid categories are strongly characterized by a much more critical tone, with readers pointing out potential flaws in the methodology or  analyses, or highlighting inconsistency either internally in the manuscript or compared to other study’s findings. Herewith, reader responses mainly take the role of an extension of the review process, whereas responses to articles in the Covid-category commonly take a role in extending the discussion within the article itself.This article has aimed to shed light on shifting practices of manuscript peer review under the global pandemic caused by Covid-19. Previous studies have indicated review processes in medical journals for Covid-19-related content have been considerably faster than review processes before the pandemic, or for content not related to Covid-19 . This raThe results of our analyses suggest that, even though Covid-19-related content was reviewed substantially faster, review reports do not seem to be less thorough, judging by their length and the nature of comments made by reviewers. In fact, reviewers seem to be at least as engaged with the manuscripts’ content, providing detailed comments and suggestions for improvement, rather than merely pointing out flaws or gaps.Nevertheless, several important differences in the content of the review reports and editorial decision letters were observed. Most notably, Covid-19-related content attracted fewer requests for additional experiments or data gathering, less requests for major structural revisions of manuscripts, and more suggestions to address potential shortcomings in alternative ways. The latter may include suggestions to simply acknowledge limitations or rephrase conclusions, to only add additional data if this does not require substantial (time) investments, or even to state the lack of high-quality data as one of the study’s main conclusions. Some of these observed changes in review content constitute plausible explanations for the earlier findings that Covid-19-related content passes the peer-review process faster than other manuscripts.BMJ tended to be milder for articles in the Covid-category compared to pre- and non-Covid-articles. This was reflected in their generally more neutral tone, connecting studies to the wider literature, rather than questioning or criticizing methodology or analyses. Alternatively, instead of characterizing the observed differences as deviations from ideal review practices, one could argue that the differences indicate the ability of the review process to adequately respond to changes in context and demand. It arguably indicates an agility and adaptability of a system that is usually typified as conservative, traditional, or even ‘out-dated’.This suggests that reviewers and editors use different criteria for Covid-19-related papers compared to other manuscripts submitted to their journals. Both actors seem to be somewhat milder regarding Covid-19-related manuscripts, accepting shortcomings that would usually not pass an editorial evaluation or the journal’s selection mechanism. In several cases, reviewers and editors specifically refer to the Covid-19 pandemic and the difficulties it raises for data collection or the necessity it creates to disseminate novel findings to account for their decisions. However, most changes were not explicitly attributed to the pandemic, suggesting that actors either did not feel the need to stress this explicitly or that shifts in quality standards occurred unconsciously. Similar to comments in reviewer reports and editorial decision letters, the open reader responses to the articles in We stress however that these are all merely general patterns to which several exceptions exist. Indeed, requests for additional data were also found in the Covid-category, suggestions to damper conclusions or acknowledge additional limitations were surely not exclusive to this category, and readers have posed critical comments to Covid-manuscripts just as they have done to others. Nevertheless, all patterns described here were prominent across the sampled materials and are hence likely to be indicative of wider patterns in pandemic review practices.This study may have suffered from various limitations. First, even though it analyses a substantial amount of documents, these documents were related to a relatively small set of manuscripts being published in just two different journals. Individual characteristics of either the manuscripts, the reviewers involved, or the journal , might have impacted the study findings. Caution should hence be exercised when generalizing the current findings to other contexts and the wider Covid-19 literature.BMJ and eLife—has been extremely fast during the pandemic. Indeed, both policy and clinical decisions had to be made fast. Therefore, content published in the early phases of the pandemic is particularly likely to have had policy and clinical implications.Second, the data used for all analyses in this study were collected in the relatively early phases of the pandemic. This might have impacted on the analyses, for example, because Covid-19 articles that had to go through a lengthy review process could not be included in the study sample. In addition, it is unclear whether the differences found in our study indicate a lasting effect or are rather signs of a temporary shift in review attitudes. Nevertheless, the differences reported here seem to be important, especially because knowledge uptake from novel scientific articles—particularly those published in well-reputed journals such as Third, our study relies on the analyses of accepted manuscripts only. Due to the unavailability of relevant documents for manuscripts that were rejected by journals, we were not able to include these manuscripts in our study. Similar concerns have been raised in the past and constitute a major hurdle in research on academic peer-review practices . In partIn evaluating scholarly manuscripts dealing with content related to Covid-19, journal editors, peer reviewers, and readers alike seem to be somewhat milder in their opinions, using different quality criteria and being satisfied with potentially lower standards. In particular, they seem to acknowledge that gathering additional data, performing additional experiments, or extensively restructuring a manuscript requires considerable efforts. With researchers, policymakers, and society at large direly looking for additional knowledge about the virus and ways how to fight it effectively, ‘There is no time for that now’.Lowering the bar on the number of (control) experiments conducted or data sets included can have multiple implications for the quality of the science conducted. For several decades now, multiple scholars have claimed science to be in a ‘reproducibility crisis’ . While aInterestingly, changes in the review process and editorial evaluation seem to concern not merely a loosening of standards but also involve a shift in quality criteria used. For example, the case of the updated version of a clinical model presented in Section 4.2.1 reflects a shift from the importance of novelty towards a focus on clinical or societal relevance. Therewith, the Covid-19 pandemic seems to have, at least temporarily, influenced fundamental characteristics of research evaluation and impacted traditional notions of what is perceived as ‘high quality’ research .As mentioned before, a lack of clear differences between the pre- and non-Covid categories, suggests that any changes regarding review practices for Covid-papers, are not likely to be attributable to general cooperative attitudes that might have emerged during or due to the global pandemic . InsteadBMJ’s open reader responses, also the wider community of readers and researchers in their post-publication commentaries to papers, seem to comment in qualitatively different ways to Covid-19-related content. Hence, the Covid-19 pandemic seems to have influenced the peer-review process across its full range of stakeholders. It will be interesting and important for future research to keep monitoring these changes, to assess whether they have a lasting effect on science’s central mechanism of research evaluation.As a final remark, we note the observation that changes in the review and editorial practices seem to occur across a range of different actors involved. Particularly, peer reviewers seem to be at the heart of most changes, initiating decisions about whether to make specific requests or give suggestions for manuscript improvement. Subsequently, journal editors commonly follow suit, but they seem to take a less active role in the changes described. Lastly, as indicated by Research Evaluation Journal online.rvaa037_Supplementary_DataClick here for additional data file.


Brain Communications. While 2020 has been a tumultuous and difficult year, our journal has luckily continued to flourish. We are proud of the breadth of rigorous translational neuroscience studies that are being submitted to our journal. We have continued to publish papers on a wide range of conditions from neurodevelopmental disorders to neurodegeneration and studies ranging from fundamental neurobiology to diagnostic tools and results of clinical trials  was successful, meaning that all of our papers now automatically appear in PMC and are searchable in PubMed. We are also now indexed in the Emerging Sources Citation Index and the Directory of Open Access Journals, which are important steps for a new open access journal to build a reputation in the scholarly publishing space.https://academic.oup.com/braincomms/pages/Editorial_Board (last accessed 16 December 2020).Our wonderful team of editors have been working hard to keep your papers moving without too much delay despite the difficulties finding reviewers with time to help review papers during the pandemic. Our average time to first decision on papers is around 30 days. Over the past 6 month, we have said goodbye to several editors who have stepped down due to retirement or other commitments, and I extend heartfelt thanks to all of them for helping to launch the journal. Thank you Richard Ribchester, Asya Rolls and Kevin Talbot! We are thrilled to welcome several new editorial board members to the team . For a list of all of our team members and information about their areas of expertise, please see Brain as former Editor in Chief Dimitri Kullmann has finished his term and handed over the reins to Masud Husain. Masud and his new editorial team have refreshed Brain’s style including changing the format of references in papers. To make transfers from Brain to Brain Communications as seamless as possible, we will be changing our referencing style to match.There are also big changes happening this year with our sister Journal The cover image for this issue comes from the review by Jonathan Lifshitz and colleagues, which reminds us of the importance and beauty of rod morphology adopted by some microglia after injury, which had been largely ignored for almost 100 years (Thank you all for continuing to support our journal by sending in papers, reading the work here and spreading the word. Wishing you all a healthy 2021 full of vaccines and eventually, in-person conferences and even hugs!


Objectives: Peer review is a powerful tool that steers the education and practice of medical researchers but may allow biased critique by anonymous reviewers. We explored factors unrelated to research quality that may influence peer review reports, and assessed the possibility that sub-types of reviewers exist. Our findings could potentially improve the peer review process.Methods: We evaluated the harshness, constructiveness and positiveness in 596 reviews from journals with open peer review, plus 46 reviews from colleagues’ anonymously reviewed manuscripts. We considered possible influencing factors, such as number of authors and seasonal trends, on the content of the review. Finally, using machine-learning we identified latent types of reviewer with differing characteristics.Results: Reviews provided during a northern-hemisphere winter were significantly harsher, suggesting a seasonal effect on language. Reviews for articles in journals with an open peer review policy were significantly less harsh than those with an anonymous review process. Further, we identified three types of reviewers: nurturing, begrudged, and blasé.Conclusion: Nurturing reviews were in a minority and our findings suggest that more widespread open peer reviewing could improve the educational value of peer review, increase the constructive criticism that encourages researchers, and reduce pride and prejudice in editorial processes. Peer review is a very influential type of feedback. It educates authors by telling them why their work is not deemed publishable or by recommending improvements, upon which acceptance for publication depends. Peer review can make or break PhDs-by-publication and influence academic career progression. It shapes the field by determining what it is acceptable for authors to say in print and is an arbiter of academic quality. Given its formative place in the education of academically inclined doctors, it is important to evaluate peer review and consider whether there is scope for reshaping this aspect of our field.Practice pointsPeer review is an influential type of feedback, but not without flaws.We identified several factors, unrelated to research quality, that may influence sentiment and language in peer review.Using a data-driven approach, we identified three types of reviewers: nurturing reviewers, begrudged, unhelpful reviewers, and blasé, indifferent reviewers.More widespread open processes could improve the educational value of peer review.The reliability of peer review has been questioned , in which Hitler abreacts to his generals telling him Germany is about to be defeated, was spoofed as a rant over one peer reviewer. It would be disturbing if ‘a 19th century gloves off approach’ , collect and disseminate the finest examples. The Facebook group, ‘Reviewer 2 Must Be Stopped!’, shares the poor experiences with the peer review process of nearly 13 K members; and a famous scene from the WWII drama h’ Walsh  was stilh’ Walsh .The purpose of this study was to help medical researchers and those who educate them, appraise the formative value of peer review. We set out to evaluate the content of peer reviews systematically and objectively, examining factors that influence the language of peer review reports. We took a data-driven approach to identify different types of scientific reviewers in the biomedical domain. Peer review reports were downloaded from 10 randomly selected BioMed Central (BMC) journals that operate an open peer review policy. The 10 journals spanned the fields of medical education, medicine, biology, biomedical data mining, public health and infectious disease see . SelectePublished in 2017Research articlePeer review reports were available onlineWe randomly selected papers from across the 10 BMC journals , with a given paper included in the analysis if it met the following criteria:We aimed to collect an even distribution of reviews from papers published in each of the calendar months.Supplementary File 1.A full description of the data collection process is available in To collect additional review reports from journals without an open peer review policy, we contacted colleagues and collaborators (past and current) to request reviewers’ reports from their own manuscripts. As before, only reports of papers that were eventually published and in the fields of medicine or biology (or related fields) were accepted. We did not restrict the collection to papers published in a specific year, instead we asked for reviews of recently accepted published work. To minimize bias in report selection by our colleagues, the request did not reveal our hypothesis and only gave a general description of the study. A total of 46 reviews from 18 papers were included in this ‘anonymous report’ set; this sample size is based solely on response rates from our colleagues and collaborators.Paper title, number of authors, original submission date and subsequent publication dateDate of the 1st reviewReviewer numberLevel of detail (from 0 to 5)Number of comments (from 0 to 5)Request for an additional analysis (yes/no)Constructiveness (from 0 to 5)Positiveness (from 0 to 5)Harshness (from 0 to 5)Number of rounds of review and which reviewer requested theseAny quotes that included recurring phrases across peer reviews.We randomly allocated a subset of the 10 BMC journals among ourselves to evaluate. Each evaluator randomly selected a paper from the journal’s website (according to the aforementioned criteria) and extracted the following information from the original (i.e. 1st round) reviewer report for each reviewer individually:This process was repeated for the anonymous peer reviews.To assess degree of agreement between evaluators, reviewers’ reports from an additional 10 randomly selected papers  were evaluated and scored by the evaluators. Evaluators’ agreement, for each of the ‘level of detail’, ‘constructiveness’, ‘positiveness’ and ‘harshness’ measures, were assessed by calculating the intra-class correlation coefficient (ICC). ICC estimates and their 95% confidence intervals were calculated based on an absolute-agreement, two-way random-effects model, to indicate the scoring agreement between the four evaluators. The between-evaluator agreement on ‘request additional analysis’ was evaluated with Fleiss’ extended Kappa statistic.To study the risk of getting a harsh review, we performed a regression analysis in which the outcome was defined as the harshness score assigned to each review. Predictors included: the season, the day of the week on which the review was received by the editor, and the number of authors of the manuscript. To control for imbalances of variables, we added evaluator and journal terms in the model.To identify polytomous classes of reviewers, we applied latent class analysis. This is an unsupervised statistical method for identifying unmeasured class membership among subjects using observed categorical and/or continuous variables Formann , used inSignificant differences between the identified latent classes were tested using Chi-squared tests for categorical or binary variables  and analysis of variance for continuous variables (number of comments and days between submission and review). A chi-squared test was also used to test for significant differences in how reviewers were represented between classes.p < 0.001), and lowest for the ‘constructiveness’ measure , with ‘positiveness’ and ‘harshness’ showing fair agreement . Similarly, the between-evaluator agreement for the ‘request additional analysis’ indicated fair agreement (Fleiss’s Kappa: 0.336), which was significantly better than no agreement (p < 0.001).We evaluated a total of 596 reviewers’ reports from 278 peer-reviewed papers, across 10 BMC journals. Additionally, we included 46 reviews from 18 papers that were published in journals with an anonymous peer review policy. R = –0.476, p < 0.001) and with constructiveness . On the other hand, constructiveness strongly correlated with the level of detail within a given review . Unsurprisingly, the level of detail, number of comments and length of review all positively correlated with each other (R > 0.4).Examination of correlations between the different factors assessed by our evaluators revealed that, intuitively, harshness negatively correlated with positiveness .To examine the effect that open peer review policy may have on the sentiment of reviewers’ reports, we included an analysis of data collected from anonymous reports. As described above, a total of 46 anonymous reports were obtained from 18 papers. Overall, anonymous reports tended to be harsher, less positive and not as constructive as those published alongside the manuscripts . The next most prevalent category of reviewer was the ‘begrudged’; academics lacking positive regard towards fellow scientists. Whilst their criticisms may have been valid, they would have been far more useful if worded more constructively and less discourteously than: ‘this paper strikes me more as a review than original research - there is no set hypothesis tested or original findings.’; ‘I am concerned that this paper may be more confounding than informative’; ‘The first major issue is that the manuscript was poorly written’; and ‘In the Discussion the authors describe that… What did they expect?’. One reviewer wrote ‘… so broad as to be superficial and meaningless: instead of “narrative”, we could just say “pink cloud of general nice stuff” and be just as meaningful’. Our analysis even found one self-identified begrudged reviewer: ‘The A novel strength of this work was to apply a data-driven unsupervised learning method to identify and stratify subtypes of peer reviewer with different characteristics similar to reviewer types identified anecdotally and from and qualitative studies Duncan .The main limitation stems from the likelihood that open peer review policy influences reviewers to be more constructive and to tone down harsh, cynical or very negative comments. We addressed this potential confounder by including analysis of anonymous review reports of papers published in journals without an open peer review policy, collected from our colleagues. Another limitation is the potential bias introduced by focusing only on BMC journals in our main data collection. To mitigate this, at least in part, we collected reviews from different journals in the BMC family, from different fields of research , and with different levels of Impact Factor. However, a more comprehensive assessment of journals across different fields of research is warranted. Similarly, we were only able to study published articles, meaning comments for rejected manuscripts (which may differ from those of accepted articles) are missed from this study, perhaps leading to an underestimation of harshness in peer review reports. As with many text evaluation studies, there were some differences between our evaluators in the way reviewers’ reports were scored. This was particularly apparent given the subjective nature of the measures by which each evaluator scored the reviews.Our findings have implications for scholarly practice. We suggest that editors should remind reviewers that authors have invested time and effort in their manuscripts and deserve fair, transparent and constructive criticism. It may also be possible that journal editors themselves fall into the three types identified here, or some other groupings; and while this would require a separate investigation, editors should be mindful of their own attitudes when assessing peer review reports. Open peer review policies could help reduce biases and related problems by removing the blanket of anonymity that allows reviewers to abuse their power. Being more accountable might encourage reviewers to carry out more meticulous evaluations and be more careful in their choice of words. It is plausible that reviewers shift between the types we have described on different occasions. Editorial boards might consider mechanisms to encourage and educate reviewers to adopt the ‘nurturing’ type in preference to the other two.To conclude, our findings underpin the urban legend of three types of reviewer with empirical evidence – a data-driven classification of Nurturing, Begrudging and Blasé reviewers. We also identified the potential of winter blues to generate harsh reviews, but this requires further investigation by exploring the geographical locations of reviewers in larger datasets. We urge editors to adopt open peer review policies to avoid the cloak of anonymity offered by blinded reviewer comments.


Preprint manuscripts, rapid publications and opinion pieces have been essential in permitting the lay press and public health authorities to preview data relating to coronavirus disease 2019 (COVID-19), including the range of clinical manifestations and the basic epidemiology early on in the pandemic. However, the rapid dissemination of information has highlighted some issues with communication of scientific results and opinions in this time of heightened sensitivity and global concern.Rapid publication of COVID-19 literature through expedited review, preprint publications and opinion pieces are important resources for the medical scientific community. Yet the risks of unverified information loom large in times when the healthcare community is desperate for information. Information that has not been properly vetted, or opinion pieces without solid evidence, may be used to influence public health policy decisions. We discuss three examples of unverified information and the consequences in this time of high anxiety surrounding COVID-19.In an era when information can be widely and swiftly disseminated, it is important to ensure that the scientific community is not an inadvertent source of misinformation. This will require a multimodal approach, with buy-in from editors, publishers, preprint servers, authors and journalists. The landscape of medical publications has changed, and a collaborative approach is required to maintain a high standard of scientific communications. When the WHO shared reports of a novel coronavirus with pandemic potential on the 31st of December 2019, the world watched the development of the outbreak in China warily and anticipated how it would affect the rest of the world. Hospitals and public health authorities were anxious to gather information to guide clinical, logistical and public health decision-making at this fast-moving early stage. The lay press was eager to bring breaking information and up-to-the-minute discoveries to the public. Preprint servers  and the rapid publication of articles were instrumental in bringing data from China and other early-affected countries, to the global community.Recognizing the need for rapid information on coronavirus disease 2019 (COVID-19), reputable journals published case reports quickly, through expedited review, and made them freely available, while simultaneously encouraging preprint publication as a means in which to bring data to a wide audience expeditiously [New England Journal of Medicine (NEJM) receives up to 40 COVID-19-related publications per day [Journal of the American Medical Association (JAMA) has been quoted as saying he has received up to 235 COVID-19 manuscripts in a single day [There has been a surge in the submission of COVID-19 manuscripts: One recent article noted that the  per day , and theNew England Journal of Medicine (NEJM) paper by Rothe et al. which was published online at the end of January 2020, entitled “Transmission of 2019-nCoV Infection from an Asymptomatic Contact in Germany” [Publishing timeframes have also become increasing tight, as illustrated by a Germany” . This caGermany” . The texGermany” . We lateGermany”  that theGermany” , 7. In tThere was considerable interest in asymptomatic transmission at this time, leading to extensive speculation on twitter and from the scientific community and public health bodies like the WHO . While ahttps://www.medrxiv.org/content/about-medrxiv) is one such preprint server; authors upload completed scientific work, yet to undergo peer review, for general readership. Authors in theory benefit from an informal review via reader comments, although there is no mechanism in place to ensure that this occurs, while the scientific community benefits from early access to unpublished data. Authors are required to submit a reporting checklist, and manuscripts undergo a brief review, including review by a medical editor, to ensure (as stated by the website) that they do not contain material that may put public health at risk.Preprint publications often go hand in hand with expedited publication and have been encouraged by journals offering expedited peer review for COVID-19 manuscripts [As of the 27th of April 2020, 1784 manuscripts on COVID-19, or the virus that causes it, have been posted on the MedRxiv website. On the 6th of February, a MedRxiv preprint garnered much attention because it described the clinical characteristics of the largest cohort of COVID-19 cases at that time: 1099 cases from more than 500 hospitals in China. In the preprint, the authors state that the “median incubation period was 3 days ” , a shortThe Straits Times) reported that the incubation period for COVID-19 could stretch out to 24 days [Based on the preprint article, on the 12th of February, the national newspaper in Singapore , where possibly as a consequence of the peer review process, the reported data has changed; it now reports a median incubation period of 4 days, with an interquartile range of 1 to 7 days [The same manuscript has since been accepted by the o 7 days . Some miHowever, it is also important to update readers if changes to data are made; as of May 13, 2020, the original preprint article remains available on the MedRxiv server, without an update of the results, or any mention that the median incubation period has been corrected by the authors; and the NEJM publication provides no clarity as to why the results were changed. A Google search of the title “Clinical Characteristics of Coronavirus Disease 2019 in China” pulls up both the NEJM publication and the MedRxiv preprint in the top ten search results, as of May 2020. This makes it likely that a member of the public or the press could pull up either version, and although disclaimers are present, it may be difficult for a lay person to grasp the difference between a preprint manuscript and a peer-reviewed paper.The Lancet published a letter to the editor by Fang et al. entitled “Are patients with hypertension and diabetes mellitus at increased risk for COVID-19 infection?” [Even hypotheses briefly mentioned in opinion pieces can be controversial in this time of heightened anxiety. On the 11th of March, 2020, ection?” . It is uection?” .In this brief correspondence piece, the authors postulate that patients with hypertension and diabetes treated with angiotensin-converting enzyme (ACE) inhibitors or angiotensin II type-I receptor blockers (ARBs) are at increased risk of developing severe and fatal COVID-19. There is pathophysiologic plausibility since these drugs increase expression of angiotensin-converting enzyme 2 (ACE2), the receptor used by SARS-CoV-2 to infect cells, but the studies from which Fang et al. drew their hypothesis did not collect data of which antihypertensive agents, if any, these patients with poorer outcomes were taking .There was one line in this commentary that garnered particular attention: “ACE2 can also be increased by thiazolidinediones and ibuprofen”. Perhaps as a result of this commentary, and despite an absence of supporting data, on the 14th of March, the French Health Minister, Olivier Véran, tweeted  that peoThree days later, a WHO spokesperson told reporters in Geneva the agency’s experts were “looking into this to give further guidance”, but advised against using ibuprofen in the meantime . The folThe confusion precipitated by one sentence in a short letter to the editor may have ultimately contributed to actions and retractions at the highest levels of international public health organizations, and there are currently over 200,000 Google search results for COVID-19 AND NSAIDS. Such confusion in the international lay press, and worse still from scientific agencies, can lead to uncertainty for both medical professionals and the public, and ambiguous messaging can erode confidence in public health systems.In an era when information can be widely and swiftly disseminated, it is important to ensure that the scientific community is not an inadvertent source of misinformation. This will require a multimodal approach, with buy-in from editors, publishers, preprint servers, authors and the media. The landscape of medical publications has changed, and a collaborative approach is required to maintain a high standard of scientific communications. We outline suggestions for the various players as follows:In order to ensure the quality of scientific journals, editors should continue to rapidly reject articles that do not meet their standards. Rapid rejections give authors a chance to improve and resubmit manuscripts, while maintaining the timeliness of their message. Maintaining rigorous journal standards cements the role of academic journals as a distinct entity from preprint servers. More stringent guidelines for reviewers would better ensure the quality of those papers selected for publication. Many subject matter experts are unavailable for reviews at this time, and it is reasonable to tap on the next generation of journal reviewers who may be more likely to provide rapid and thorough reviews, with the help of set guidelines.Publishers should clearly label pieces as “opinion” or “rapid review” as appropriate, with highly visible headings, and a comment about what that means, to make these designations clear to both the medical community and the lay press. Editors and publishers should be especially cognizant of eye-catching or sensationalistic titles and try to limit these when dealing with COVID-19 in particular. The expected timeline for manuscript review, and whether expedited review is available, should also be clearly stated on the journal’s webpage.Preprint servers could go as far as limiting access to subscribers with an academic affiliation, allowing for feedback and input from peers, but at the same time keeping the circle of access limited, until the peer-review process is completed. However, this could be problematic, as some clinicians on the front lines of the COVID-19 response could be denied access based on the criterion proposed.Alternatively, preprint servers could continue to make manuscripts widely accessible, but also include updates as a paper goes through the peer-review process. This could include a reminder to authors to post an updated version of the manuscript to the website after peer-review. For more complete transparency, the manuscripts could be updated to include feedback from reviewers and editors, with any changes to the manuscript highlighted clearly.Preprint websites like MedRxiv currently include an alert stating that the reports included are not peer-reviewed, but these statements, are much less noticeable on the manuscript itself. A clearer “note to the reader” should be included on each manuscript, and readers should be urged to state that the article has not been peer-reviewed when referencing the manuscript in an article, tweet or blog post; watermarking each page maybe a solution.As tempting as it may be to weigh on this pandemic that has grabbed headlines around the world, authors should be aware of the heightened attention, particularly around COVID-19, and avoid statements that are speculative or could have public health implications, especially when they are outside the author’s realm of expertise. If speculation is required, it should be clearly spelled out, with a statement about the lack of supporting data.Many in the academic world have years of training in scientific methodology and scientific writing, but no formal training in communicating with the lay press or on social media. We should look to the available guidance from academic organizations and scientific publishers on how to communicate with the press via press releases , social In recent years, news outlets have faced significant financial cutbacks and reductions in staff, resulting in reductions in experienced science writers , thus wiRapid publication of COVID-19 literature through expedited review, preprint publications and opinion pieces are important resources for the medical scientific community. Yet the risks of unverified information are amplified at a time when the healthcare community is desperate for information on which to base clinical and policy decisions, and when media outlets are consumer-driven. Insightful medically trained readers as well as the lay press must consider the content of these articles for what they are, and apply additional scrutiny and scepticism while acknowledging that expedited publications and preprint servers are a valuable resource. In these times, we must remember that thoughtful review is the essence of what transforms data into meaningful and generalizable conclusions.


Transparent and accurate reporting is essential for readers to adequately interpret the results of a study. Journals can play a vital role in improving the reporting of published randomised controlled trials (RCTs). We describe an RCT to evaluate our hypothesis that asking peer reviewers to check whether the most important and poorly reported CONsolidated Standards of Reporting Trials (CONSORT) items are adequately reported will result in higher adherence to CONSORT guidelines in published RCTs.Manuscripts presenting the primary results of RCTs submitted to participating journals will be randomised to either the intervention group  or a control group . The primary outcome will be the mean proportion of the 10 items that are adequately reported in the published articles. Peer reviewers and manuscript authors will not be informed of the study hypothesis, design or intervention. Outcomes will be assessed in duplicate from published articles by two data extractors (at least one blinded to the intervention). We will enrol eligible manuscripts until a minimum of 83 articles per group  are published.https://osf.io/c4hn8).This pragmatic RCT was approved by the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001). If this intervention is effective, it could be implemented by all medical journals without requiring large additional resources at journal level. Findings will be disseminated through presentations in relevant conferences and peer-reviewed publications. This trial is registered on the Open Science Framework ( Pragmatic randomised controlled trial (RCT) with individual randomisation of real manuscripts describing RCTs submitted to a variety of journals.Main outcomes will be assessed from publicly available sources .If this simple intervention is effective, it could be implemented by journals without requiring large additional resources at journal level.The intervention could not be included within the email from journal with the link to the manuscript for review, risking peer reviewers will potentially ignore the separate email containing the CONsolidated Standards of Reporting Trials reminder.There is substantial agreement that well conducted and reported randomised controlled trials (RCTs) generate the most trustworthy evidence when evaluating newly developed or existing clinical interventions.To improve the quality and transparency of clinical and epidemiological research, the EQUATOR  Network was founded in 2006 and officially launched in 2008.The CONsolidated Standards of Reporting Trials (CONSORT) is perhaps the most prominent reporting guideline, designed to help improve the transparency and quality of reporting of RCTs.14Despite some improvement in reporting following the endorsement of the CONSORT statement, there remain major reporting deficiencies in published RCTs.Journals can play a vital role in improving the reporting of published RCTs. For example, a survey of journals’ ‘Instructions to Authors’ in 2014 found that 63% (106 of 168) of biomedical journals mentioned CONSORT25A scoping review conducted in 2017 by Blanco and colleagues summarised different interventions aimed at improving adherence to reporting guidelines.27The role of peer reviewers and expectations of them is varied.The objective of this study is to evaluate the impact of giving peer reviewers, during the standard peer review process, a short version of the CONSORT checklist (C-short) together with a brief explanation of the items and asking them to check if they are adequately reported in the manuscript.This study is a multicentre superiority RCT with submitted manuscripts as the unit of randomisation .323210.1136/bmjopen-2019-035114.supp2Supplementary dataThe population will be defined on two levels: included journals and included manuscripts.Included journals must: (i) endorse the CONSORT statement by mentioning it in the journals’ Instruction to Authors; (ii) have published primary results of at least five RCTs in 2017 (identified using a PubMed search). To be efficient, we plan to contact (via email) the editors of eligible journals from specific publishers ) instead of separate journals. A description of the requirements for participation and a short summary information sheet will be included as part of the email invitation sent to journal editors. If a journal is eligible, and the editor agrees to take part, the editor will need to provide access to their editorial system  to enable the external researcher (BS) to screen and randomise eligible manuscripts. In cases where this is not possible, we will explore with individual journals if it would be possible to grant limited access  or to handle the different steps without access to the editorial system  and that the emails for the intervention would be sent by a member of the editorial team.All new manuscript submissions reporting the primary results of RCTs, which the journal editor has decided to send out for external peer review. Since the 10 chosen CONSORT checklist items (C-short) are applicable to different study designs, we will include all manuscripts reporting the primary results of RCTs regardless of study design .Manuscripts clearly presenting secondary trial results, additional time points, economic analyses or any other analyses.Manuscripts which are clearly labelled as a pilot or feasibility study or animal studies.Manuscripts not sent for peer review.Details of journal manuscript submission and peer review processes, including consent and potential confidentiality issues, will be discussed in detail with each journal by teleconference and/or face to face prior to the journal agreeing to take part to ensure that randomisation of manuscripts is feasible.In participating journals, the external researcher (BS) will check at least twice a week (by screening automated submission lists) all research manuscripts that are sent out for external peer review. As soon as the first invited peer reviewer accepts the invitation to review, the manuscript will be randomised to the intervention or control arm (see ‘Randomisation’ for more details). It is possible that this process might be slightly different among various included journals .After accepting to review a manuscript, peer reviewers will receive the automated, journal specific standard email with general information as per each journal’s usual practice .After accepting to review a manuscript, peer reviewers will receive the automated, journal specific standard email with general information . In addition, peer reviewers will receive an additional email from the editorial office that includes a short version of the CONSORT checklist (C-short) together with a brief explanation of the items either as a table within the email or as an attachment based on the preferences and possibilities of the journal . Peer re10.1136/bmjopen-2019-035114.supp1Supplementary dataFor the development of C-short we chose the 10 most important and poorly reported CONSORT items as identified by a group of CONSORT experts in a previous study conducted by Hopewell and colleagues.The primary outcome of this study will be the difference in the mean proportion of adequately reported C-short items in published articles between the two groups.Secondary outcomes will include the following:Mean proportion of adequately reported C-short items in published articles considering each item separately.Difference in mean proportion of adequately reported C-short items in published articles considering each sub-item (see ‘Assessment of outcomes’) as a separate item.Time from assigning an editor to the first decision (as communicated to the author after the first round of peer review).Proportion of manuscripts rejected after the first round of peer review.Proportion of manuscripts that will be published in the journal under study.Exploratory analysis of available peer reviewer comments .For journals where peer reviewers’ comments are subsequently published alongside the published article, we will examine the peer reviewers’ comments for any reference to CONSORT and trial reporting. We will contact those journals which do not make peer reviewers’ comments publicly available, to see if reviews could be provided for such analyses under the condition that only anonymised data will be published.The outcomes will be assessed independently by two  outcome assessors with expertise in the design and reporting of clinical trials. Any disagreement will be resolved by consensus or if necessary by consulting a third assessor. To ensure consistency between reviewers, we will first pilot the data extraction form; any disparities in the interpretation will be discussed and the data extraction form will be modified accordingly.Adequate reporting of items will be assessed in duplicate from published full-text publications following the same instructions as provided by the CONSORT C-short checklist.Outcomes (item 6a): (i) define primary outcome, (ii) how it was measured, (iii) at what time point and (iv) the analysis metric .Sample size (item 7a): (i) the estimated outcomes in each group, (ii) the α (type I) error level, (iii) the statistical power (or the β (type II) error level), (iv) for continuous outcomes, the SD of the measurements.Blinding (item 11a): Is the blinding status clear for the following persons: (i) healthcare provider, (ii) patients and (iii) outcome assessors.Funding (item 25): (i) the funding source, and (ii) the role of funder in the design, conduct, analysis and reporting.All items will be judged as either ‘yes’ meaning adequately reported, ‘no’ meaning not adequately reported or not reported at all, or ‘NA’ meaning that this sub-item is not applicable for this RCT. Items with different sub-items will only be judged as adequately reported if all relevant sub-items were adequately reported.The outcomes ‘time from assigning an editor to the first decision’, ‘proportion of manuscripts rejected after the first round of peer-review’ and ‘proportion of manuscripts that will be published in the journal under study’ will be extracted directly from the journal’s editorial system or provided by the journal.The overview of the study schedule, including enrolment, intervention and assessments, is presented in For the sample size calculation, we hypothesised in a first scenario  that thePLOS ONE (IP and AC), one of the participating journals, pointed out that 3 out of the 10 assessed items  should always be implemented in submissions to their journal given their policy requirements for clinical trials. Assuming that this journal will recruit a high proportion of manuscripts, and that also other journals might update their templates, we increased the sample size in a second scenario, in which all these three items would have an overall adherence of 90% in the control arm , to avoid loss of power due to potential imbalance between arms. Recruitment will be stopped as soon as both arms reach the sample size of 83. After recruitment has stopped we will wait 3 months so that manuscripts, which are still in production, can be published. Manuscripts which are published after the 3-month period will be excluded.Manuscripts meeting the eligibility criteria and sent out for external peer review by the journals will be randomised into one of the two groups . The randomisation list will be created by the Study-Randomizer systemAuthors will be blinded to the intervention. Editors will not be actively informed about the randomisation (possible exception listed under ‘Interventions’). To avoid potential bias, peer reviewers and manuscript authors will not be informed of the study hypothesis, design and intervention.Outcomes will be assessed in duplicate (see ‘Assessment of outcomes’). At least one outcome assessor will be blinded. Due to restricted resources the investigator conducting the randomisation (BS) might be involved in the data extraction from published manuscripts.All quantitative variables will be described using means and SD, or medians and interquartile ranges in case severe departures from a normal distribution are identified. Data distributions will be inspected visually  instead of performing formal statistical tests for normality. Categorical variables will be described using frequencies and percentages. For the primary and secondary outcomes, we will estimate the mean difference between the two groups and report them with respective 95% CI. No interim analysis will be conducted.The main population for analysis will be all manuscripts randomised and accepted for publication in the participating journals. In contrast to RCTs conducted with patients, where losses to follow-up need to be carefully considered , we are only interested in the reporting adherence of RCTs that are published. As such, we will exclude randomised manuscripts that were not published from the main analysis. All outcomes will be calculated based on the main population. The secondary outcome ‘Time to the first decision’ will additionally be calculated considering all randomised manuscripts (including the ones which were not published). For all analyses a p-value of 0.05 (5% significance level) will be used to indicate statistical significance. Exact p-values will be presented up to three decimal places. We anticipate there will be no missing data in this study, neither at the individual C-short items, nor at the manuscript level. This is due to the study design, which will include only the randomised manuscripts that are accepted for publication. We will analyse if the rate of manuscripts rejected after the first round of peer review and if the proportion of manuscripts that will be published differentiate among the two study arms (both secondary results).The effect of the intervention will be estimated as the mean difference in the proportion of C-short items adequately reported between the study arms. If the data on the primary outcome are normally distributed, groups will be compared using an unpaired Student’s t-test. If the data are not normally distributed, comparisons will be performed using a non-parametric equivalent test .To investigate the effect of the intervention on the secondary outcomes, mean differences with respective 95% CI will be reported. If normality is not observed for any of the continuous secondary outcomes, the same strategy adopted for the primary outcome  will be used.No formal subgroup comparative analysis is planned for the primary or secondary outcomes. However, the effect of the intervention on the primary outcome within subgroups will be presented using forest plots to visually examine whether it may differ according to some variables such as: (i) journals that actively implement the CONSORT Statement  versus journals that are not actively implementing the CONSORT statement, (ii) sample size of included RCTs (n<100 vs n≥100) and (iii) impact factor  as there is evidence that higher impact factor and higher sample size are associated with higher adherence to reporting guidelines.Outcomes from publications will be assessed and extracted in duplicate. Since this information is not confidential, we will use freely available online forms  for data extraction from published RCTs. Data entered will be validated for completeness.Data from the journal’s editorial system ) will be extracted , anonymised and entered in password protected files which are saved on a server from the University of Oxford. Data will be managed and curated according to University of Oxford regulations, which includes regular back-up (on a daily basis) of the virtual drives where the data are stored. No auditing or data monitoring is planned .The raw data extracted from the included published manuscripts can be made openly accessible in an anonymised way . Derived/aggregated data, including anonymised information generated from the journal’s editorial system, will be stored and made available to the research community when the project ends . Where appropriate, the researcher who has access to the journal’s editorial system (BS) and anyone else who will see the identifiable data will sign a confidentially agreement with the participating journals, confirming that they will not share identifiable data with any other party. Publishers such as the British Medical Journal (BMJ) state in their Company Privacy Statement that reviews and manuscripts may be used for quality improvement purposes and that is the nature of this research. Furthermore, peer reviewers for all BMJ journals receive the following statement in their invitation letter ‘We are constantly trying to find ways of improving the peer review system and have an ongoing programme of research. If you do not wish your review entered into a study please let us know by emailing […] as soon as possible.’https://osf.io/c4hn8). The first manuscript was randomised in July 2019. We expect that recruitment will be finished in summer 2021.This trial was denied registration on ClinicalTrials.gov as the study is not a clinical study that assesses a health outcome in human subjects. Instead we registered the trial on the Open Science Framework  or a member of the journal staff . The likelihood of contamination due to peer reviewers being invited to assess several RCTs and therefore becoming exposed to both intervention arms was judged small and therefore we do not plan to adjust for clustering by journal. Originally we planned to implement the intervention within the original instruction to peer reviewer email which is sent out as soon as a peer reviewer accepts the invitation from the journal. However, as these emails are sent automatically by the journal’s editorial system we would have needed to modify the software from each journal to make sure that only half of the manuscripts administered the intervention. After our first discussion with journal editors and journal staff, we realised that this approach is not feasible and therefore decided to implement the intervention in the form of a separate email. We intended to conduct this RCT in a pragmatic way so that results ‘would also be relevant to […] people who decide whether to implement the intervention on the basis of its results’.A selection of CONSORT items was chosen instead of the entire CONSORT checklist as we did not want to put too high a burden on peer reviewers, which could increase the risk that peer reviewers ignore our reminder.Should the proposed intervention be successful in improving the reporting quality of published RCTs, as measured by the adherence to CONSORT, the intervention could be implemented at the journal level without requiring a large amount of additional resources. In addition, very similar interventions for other article types  and corresponding guidelines  could be easily implemented too.Ethical approval has been obtained from the Medical Sciences Interdivisional Research Ethics Committee of the University of Oxford (R62779/RE001). The original approved study protocol is available in The results from this study will be published in a peer-reviewed journal irrespective of the study results. Authorship of publications will be granted according to the criteria of the International Committee of Medical Journal Editors. We plan to make the anonymised data set, including the data from the published articles, available as a supplementary file of the main publication.


BDJ Open ‘opened’ in 2015 and as editor I have had the genuine pleasure of seeing a huge range of articles submitted and published for anyone with an internet connection to read.a vastly expanded scope (as no confines of publishing space).relatively short publication timeframes (no need to wait for a space in the print copy).and the obvious advantage that anyone can access an open access article without the need for subscriptions or log-ins.The benefits of open access publishing are relatively easy to describe and quantify:The benefits to readers, and the whole dental/academic community, of this form of publishing are perhaps less easy to quantify but are where the real excitement in an open access journal lies. Subscription journals naturally need to publish articles targeted to their subscribers, and the space constraints of a printed journal compound this. The range of study topics and methods submitted and published in BDJ Open has exceeded expectations. To date we have published:Randomised Control Trials:Microstream capnography during conscious sedation with midazolam for oral surgery: a randomised controlled trialService evaluations:A service evaluation of the “Open Wide and Step Inside” school oral health programmeEconomic analysis:Pharmacoeconomic analysis of antibiotic therapy in maxillofacial surgeryAnd some exciting developments from researchers in industry:Stain control with two experimental dentin hypersensitivity toothpastes containing spherical silica: a randomised, early-phase development studyReviewers sometimes recommend rejection as the subject matter is ‘not of interest’ or there is ‘no reason to publish this’. The beauty of open access publishing is that, provided an article is of acceptable quality, there is really no reason not to accept it. It’s up to the world to decide if they want to read the articles. A journal with a narrower scope and more space constraints might find it difficult to justify publications on the experience of dental professionals from anywhere across the globe such as:Public Dental Service personnel facing a major health care reform in FinlandFortitude and resilience in service of the population: a case study of dental professionals striving for health in Sierra LeoneOpen access allows these authors to publish in the BDJ portfolio for anyone to compare their practices and conditions with the worldwide dental community. This international element is a particular benefit of open access publishing. Techniques and instruments, which wouldn’t be used in higher income countries due to higher cost alternatives, might be of interest to dentists in lower income countries. BDJ Open makes this possible. This includes research instruments such as the Oral Health Impact Profile:Oral health-related quality of life in patients with upper gastrointestinal and hepatic disorders in Pakistan: validation of the Oral Health Impact Profile-14 in the Urdu language.Authors from low income countries can also apply for reduced fees or waivers, meaning anyone can learn about dental students in Nepal and their knowledge of implants:Knowledge and perception about dental implants among undergraduate dental students.In terms of the quality assurance of the publishing process, it’s the same process as the print BDJ, often the same reviewers, and the same Editor-in-Chief (Stephen Hancocks) who has the final say. But is the quality of the actual research the same? With open access publishing anyone at all is free to answer that question. Our brief to reviewers is as follows:“To be published in BDJ Open, a paper must be scientifically valid and technically sound in methodology.”When open access publishing began there was a common concern that open access journals could offer a ‘you pay, we publish’ service. Whilst any topics related to oral and dental health are within scope, the peer review process means that this has been far from true for BDJ Open. Importantly, challenges to articles can be dealt with through published correspondence for anyone to access. The published correspondence on the use of LED light curing instruments in lower income countries exemplifies many of the points raised in this editorial.Reply to: Relevance of investigating light transmittance through red protective shields in dentistryIt’s been exciting to publish in more niche areas  such as a genetics paper examining links between hypodontia and colorectal cancer:AXIN2 predict risk and prognosis of colorectal cancerVariations in We were also really interested to publish research on a situation lots of dental professionals will recognise but which has not, to our knowledge, been published elsewhere:What reward does a child prefer for behaving well at the dentist?In summary, BDJ Open continues to bring oral health research to as wide an audience as possible. The breadth and depth of research published in BDJ Open to date shows that authors across the world value the opportunity to publish in a BDJ journal. These are exciting times for publishing in general and we are very proud of the contribution BDJ Open is making to the global dental community.https://www.nature.com/bdjopen/authors-and-referees or contact us at  bdjopen@nature.com.For anyone interested in publishing in BDJ Open or reviewing submissions to BDJ Open please see the author and referee instructions:


The publication of a paper on mentorship, now retracted, led us to reflect on our editorial processes and strengthened our determination in supporting diversity, equity and inclusion in research. Mainly, these were directed to claims of a negative impact of both female mentors and mentees and suggestions that opposite-gender mentorship may help to elevate the status of women in science. Following a journal investigation, the paper has now been retracted by the authors as explained below.On November 17We selected the paper for publication because we believe that mentorship in academia is an important and understudied area of research. Quantifying institutional biases in science publication, citation and “impact” is a necessary first step towards recognising and addressing them. This paper analysed a large dataset to approach the question of collaboration and co-authorship among junior and senior researchers in an attempt to study informal mentorship, and we considered this to be of interest to the community.th.The criticisms from readers revolved around the validity of the conclusions in light of the available data, assumptions made and methodology used. In particular, readers criticised the use of co-authorship as a measure of mentorship, and citations as a measure of success of the mentoring relationship. Some of these concerns were also raised during the first round of peer review, which involved four reviewers with expertise ranging from science of science, network analysis and mentoring relationships. In light of these concerns, a number of changes, including the addition of a survey that was aimed at elucidating the type of mentorship between pairs of junior and senior co-authors, were added to the revised version by the authors and the manuscript was peer reviewed again. Upon publication, it became clear that the concerns had not been sufficiently addressed, and we started an investigation. We alerted readers with an Editor’s Note published on November 19We followed our established editorial processes, which involved recruiting three additional independent experts to evaluate the validity of the approaches and the soundness of the interpretation. They supported previous criticisms and identified further shortcomings in relation to the use of co-authorship as a measure of informal mentorship. They also noted that the operationalisation of mentorship quality, based on the number of citations and network centrality of mentors, was not validated.paper.According to these criticisms, any conclusions that might be drawn on biases in citations in the context of co-authorship cannot be extended to informal mentorship. As such, the paper’s conclusions in their current form do not stand, and the authors have retracted the During the investigation, we also received further communications from readers highlighting issues with the paper and are grateful to all the researchers who have contacted us and who have invested their time in reviewing the work.Simply being uncomfortable with the conclusions of a published paper, would and should not lead to retraction on this basis alone. If the research question is important, and the conclusions sound and valid, however controversial, there can be merit in sharing them with the research community so that a debate can ensue and a range of possible solutions be proposed. In this case, the conclusions turned out not to be supported, and we apologise to the research community for any unintended harm derived from the publication of this paper.information for authors on how we approach this type of paper. As part of these guidelines, we recognise that it is essential to ensure that such studies are considered from multiple perspectives including from groups concerned by the findings. We believe that this will help us ensure that the review process takes into account the dimension of potential harm, and that claims are moderated by a consideration of limitations when conclusions have potential policy implications. We will keep developing our guidelines for manuscripts with sensitive research in the social and behavioural sciences, and in areas with significant societal and public policy impact.As part of our investigation, we also reviewed our editorial practices and policies and, in the past few weeks, have developed additional internal guidelines, and updated Sense about Science, we launched as a pilot a peer review programme for early career researchers, consisting of a webinar and a hands-on phase which we plan to extend next year. We will share the results of these efforts in editorials in 2021.This experience has reinforced our commitment to equity and inclusion in research. We have been working to strengthen our ongoing efforts to reach out to a diverse pool of reviewers and commissioned authors. In collaboration with mentorship that characterises the Nature journals, we intend to highlight, recognize and support mentorship by women academics. We are actively discussing within the editorial team what further initiatives we can launch or support and we will finalise our plans and commitment early next year.As part of the longstanding commitment to


Academic publishing is increasingly threatened by unscholarly forces, including false and/or predatory entities. One serious false element are individuals that masquerade as fictitious authors by creating fictitious personalities, emails, and affiliations, sometimes misappropriating the entities of real scholars. One objective of fictitious papers is to target “predatory”  as well as legitimate  journals or publishers, possibly to expose flawed peer review. Some fictitious papers are retracted, but retractions are insufficient since the real individuals who created fictitious authors and papers suffer no ethical or legal consequences.The International Journal of Biochemistry & Cell Biology (IJB&CB). This is not a sting operation or hoax paper  that created “Bo Liu” and this false paper.There are four retractions in ted once , and accted once . CitatioIJB&CB paper. Since the ORCID mission failed in this case because this tool could not disambiguate real from actual or potentially false authors, ORCID may also thus begin to represent an academic threat.The author disambiguation tool, ORCID, had  over 340 “Bo Liu” or “Liu Bo” registered, most of which are “single-use” accounts with no identifying information that could shed light on the real identity of the lead author or confirm if any of them is the “Bo Liu” of the now retracted Ethics groups and organizations such as Committee on Publication Ethics, International Committee of Medical Journal Editors, Open Access Scholarly Publishers Association, and Directory of Open Access Journals, which collectively represent thousands of publishers and journals, need to take action and assume a clear and resolute stance regarding the creators of fake elements such as false authors or emails, or individuals who misappropriate institutions because these actions cause reputational and financial damage to valid authors, journals, and publishers. In addition to labeling such actions as profoundly unethical, instituting life-time bans in affected journals and publishers, stronger detection and prevention methods need to be implemented .


Productivity in science has increased and it is becoming more important for scientists to publish, to publish frequently, and to accumulate citations to their work. However, the peer review system may not only promote and advance but also hinder, prevent, or delay publication. In this personal perspective, confirmatory, consensual, competitive, and controversial publication strategies are described that they may meet with various degrees of approval or disapproval from the author’s peers. The selected publication strategy may impact on the development of a career. Resolving controversies helps science advance efficiently. Therefore, controversies should be sought and addressed, although preferably not at the start of a career. The productivity in science has increased logarithmically over the past decades. The mass of scientific information that has come available until now and the relatively short “cited” life of the majority of papers may dissuade some scientists to contribute new knowledge. Yet, it has become very important to publish, to publish regularly, and to obtain citations. This is particularly pertinent for biomedical research and physiology. Both fields are characterized by their high potential impact on society but also by the fierce competition between its workers for the available funding. To publish regularly is a sign of active involvement in research and helps in the acquisition of funding. In addition, being cited is considered an indication of scientific quality. Publication of scientific reports can be helped, promoted but also hindered, delayed, or even prevented by one’s peers. Scientists, especially when they are at the start of their career, may want to spend some thoughts on their publication strategy and on how to be successful in publishing. In the below personal thoughts, I will offer some unrequested and likely unwanted advice, like others before me .Eventual publication of the scientific reports depends on the gatekeeper function of the peer review system. This selection procedure stems from a time when printed pages were expensive and only a limited amount of papers could be printed. Hence, only the best papers could be accepted and a quality assessment by several of the author’s peers was the preferred selection method. Various forms of peer review exist, but they all involve this assessment of the scientific report by peers. In other sections of science, other forms of peer review are used whereby a paper can appear in a “pre-print” version and to which the entire community can add their comments.In biomedical sciences and physiology, the author’s peers are often also competitors for the same funding or contestant in the race of publishing data first. Authors and their peers, therefore, form a social network in which publication of the results of scientific research is regulated. A new member of the group is admitted by agreement of the existing members (co-optation). This has implications. One is that a scientist has to please the network and earn his/her place in the network before he or she can challenge it. Another one is that the allowed increment of scientific knowledge in scientific publications is limited, because the author is not permitted to exceed the limits of the allowed “niche” and because it is in the interest of the network to spread the new information over as many publications as possible. Because a similar peer review system is operative for the distribution of funding of research, failure to publish, therefore, often implies the failure to acquire funding. It is, therefore, important for a – beginning – scientist to think about a publication strategy before publishing a highly competitive or controversial paper. One strategy is to defer a controversial publication until the social network of peers is “primed.”Awareness of various publication types may help the – young – scientist to choose a publication strategy and to eventually obtain a position in which he/she is respected and able to publish even the most controversial ideas. Each of the following four publication types can be equally valuable and helpful for the advancement of science, but they differ in their effect on the readership and peers. As with all classification system, combinations of these classes occur and other classifications are possible.status quo and will not overhaul existing systems of distribution of funding. It is friendly to the peers and it can be conceived as “doing what is expected.”The consumers of a scientific publication are represented by the readership and the peer reviewers. It is in the interest of these consumers to be informed of research that does encroach on their own field of research and that does not challenge the existing scientific ideas too much. A consumerist, confirmatory, publication will thus maintain the existing scientific A confirmatory publication is characterized by testing an existing hypothesis, by expanding an existing hypothesis in a reductionist or integrative fashion , by application of this hypothesis to other species/models, or by describing associated phenomena without an overruling new hypothesis. I surmise that the majority of publications are of the confirmatory consumerist type.My own 1988 publication (my first as a first author) about the change of extracellular potassium concentration during acute myocardial ischemia can be seen as a confirmatory publication . It builConsensus reports are often published to guide clinical practice and to standardize diagnosis and/or treatment of patients. Scientists from various countries and disciplines often confer and reach consensus about a disease, including its pathophysiological mechanisms. Even when there is disagreement about some aspects of the mechanism or about treatment, an effort is made to reach consensus or to reach consensus on which points to disagree. Consensus is often reached by voting on various theses and, therefore, involves a democratic  process. The ensuing report is often published in various journals at the same time in order to emphasize the broad – network wide – support of the agreement reached. A consensual publication is an expression of the idea that scientific research is a group effort and, thus, supports the notion that the scientific community is a social network.Usually, a consensual publication is typified by a large number of authors, most of whom are authorities on the topic of interest. The paper usually contains a review of the literature and includes citations to publications by each of the co-authors. The publication is often supported or endorsed by one or more international professional societies. Other types of consensual publications occur when authors agree to publish on the same subject and explore their differences. They agree to disagree.Examples of consensus reports are the ones on Brugada syndrome, a potentially lethal syndrome occurring in relatively young patients , 2017. TThis is (to the best of our knowledge) the first (description of)…” , it emphasizes the idea that an individual or a single group can stand out and excel in science. Indeed, a single paper may change the way we think about a subject and the first person to publish it can acquire eternal fame. The publication that comes second is then considered confirmative and not as ground breaking as the first. A publication of the competitive type usually contains a claim that the authors are the first to describe the particular scientific content, often in the summary, in the introduction, or in the discussion: “on of)…” .It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material”  of the highly controversial “On the origin of species” and the separate publication of the “The descent of man” by Darwin may be examples of a publication strategy in which Darwin let his peers become acquainted with his ideas on evolution prior to extrapolating these ideas to the human species , 1871. OA special case is formed by papers in which a failure to replicate earlier findings is published. These papers are often associated with the same strong emotions and eventually may lead to controversy, if the difference is not resolved. Publication of the “negative results” is difficult, because the exclusion of all alternative explanations has provided. Publications of this type are rare, although probably very valuable to bring science forward.If we accept that the scientific community is a social network in which co-optation plays a role, acceptance of the scientist by his/her group of peers is important for his/her professional career. Because publication success is dependent of the gatekeepers of the peer review process, scientists should think about how to publish their research before embarking on the study. In the above, I have mentioned four possible – arbitrary – publication modalities with increasing chance of confrontation with the peer group. Of course, more classifications of publication modes exist, and the categories mentioned are not mutually exclusive. It is not easy to define whether a particular publication type is better for a career than another. If a controversy can be resolved  or a major advance is published in a competitive paper, it may bring personal success and a favorable position for acquiring funding to the authors of the paper. However, this may take more time than is permitted for a scientist during his/her PhD-trajectory , during a post-doctoral fellowship , or for junior faculty. These beginning scientists may consider to concentrate on confirmatory or consensual publications in their own research, while participating in research groups to obtain more “risky” publications . Publishing a controversial paper as a first author and as a first publication may be disadvantageous. It should rather be recommendable to publish a confirmatory or a consensual paper before a competitive or controversial paper and entering into battle with the peers. Postponing the battle until enough critical mass – support from others – has been acquired may be preferred for a starting career.The advantages and disadvantages of the various publication strategies thus depend on the peer review system and we should consider changing this widely accepted system of quality assessment. However, changing it into other forms  will not change the essential characteristic of the scientific world as a social network where co-optation is the norm. Open peer review, as practiced by Frontiers, increases transparency about the potential biases of the peers, because the names of the peer reviewers are disclosed but does not change the essential method of co-optation. Controversial publications bring science forward most efficiently. Publishing houses may promote controversial publications and offer platforms for public debate .Consensus in the form of a compromise is generally not good for science, because truth usually is not amenable to bartering. If a controversy exists, consensus may be reached on what to agree on and on what to disagree on. Eventually, resolving controversies helps science advance efficiently. Therefore, scientific controversies should be sought and resolved, and their publication promoted.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.The author confirms being the sole contributor of this work and has approved it for publication.The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.