Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes. But what is meant by `open peer review', and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for? Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues. We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community. Peer review is heralded as the bedrock of quality assurance in scholarly communication, used to scrutinise, select, and improve manuscripts for publication . However there are differences in the way various models of peer review are implemented. What is often termed ‘traditional’ or ‘conventional’ peer review is generally 1) partially or completely anonymous, with either the reviewer unknown to the author (single-blind review) or both author and reviewer unknown to each other (double-blind review); 2) selective, with reviewers invited by editors; and 3) opaque, with neither the review, editorial process nor the review reports themselves ever made public. Large-scale surveys continuously show that researchers hold peer review to be beneficial, but that processes are potentially often sub-optimal and open to criticism for being, for example, biased or slow , and “open reports”, where review reports are published alongside the relevant article. These traits can be combined, but need not be, and may be complemented by other innovations, such as “open participation”, where the wider community are able to contribute to the review process, “open interaction”, where direct reciprocal discussion between author(s) and reviewers, and/or between reviewers, is allowed and encouraged, and “open pre-review manuscripts”, where manuscripts are made immediately available in advance of any formal peer review procedures .So what is OPR? OPR means different things to different people and communities and has been defined as “an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of open science”All these features aim towards either increased transparency, rigour, or inclusivity in research processes, as well as recognizing reviewers’ contribution to published research literature, driven by a wide range of considerations. In this sense, this article sets out the following 10 items that outline how to apply OPR in a way such that it becomes a valuable exercise for you as an author, reviewer or editor.4).The results of this papers have been derived from a review of a broad range of research studies (in some the authors have been involved), as well as practices and experiences at publishing houses that have implemented OPR . In the review process, authors and reviewers typically collaborate on the improvement of a manuscript, which is in principle submitted as ready for publication. A notable exception here is theregistered report article format in which the rationale for a study and the proposed methodology — the “study protocol” — are pre-registered with the journal and submitted for peer review before the actual gathering of data and research takes place7. During the review process, authors and reviewers may subjectively agree or disagree on how to present the results and/or what needs improvement, amendment or correction. Now imagine all this happens with readers able to see the process ‘live’ or after the article has been published! It is therefore essential that reviewers ensure that they communicate their points in a clear and civil way, to maximise the chances that it will be received as valuable feedback by the author(s). Authors should also be able to respond in kind , accepting comments and critique as a process of constructive collaboration in ensuring their work is of the highest quality for publication, and refrain from anything which could be interpreted as a vengeful action.Whatever the degree of openness in a peer review process, as a standard form of academic best practice, it is essential to act with an attitude of charity and in accordance with the highest moral principles8. Although some early studies found no overall change in quality between single-blind versus identities revealed to the authors (i.e. “open identities”)9, other studies have indicated that there may be an improvement of the overall quality of review reports under OPR, particularly that comments are more constructive11.In situations where manuscripts are made available as a preprint and review reports are disclosed, all steps of the scientific quality assurance process can be traced and examined. Experiences gained since the early 2000s have shown that submissions that are posted publicly for interactive public peer review start off with a higher quality compared to those submitted in closed peer review processeseLife takes this approach), which is an efficient way of providing feedback to the authors. Consistent feedback saves the authors’ time and is far easier to take into account than contradictory reports. Signed reviews enable direct communication between authors and reviewers and thus may also enhance constructive quality assurance, although there is a risk that non-anonymous reviews will be less critical.Some OPR approaches rely on a consensus-building process between the reviewers agreements  were addressed and how the final version emerged . This is especially the case when the authors’ responses to the reviewers’ comments are also published. An OPR process also makes the reviewers accountable for their comments, and indeed the editor accountable for their choice of reviewers and the final acceptance decision. On this basis, the research community and wider public can assess all comments made by authors, reviewers and editors, and may even participate in the discussion. By fostering such transparency, OPR reports can help dispel persistent concerns about the rigour of peer review processes13. However, how to best archive and preserve published review reports and related comments has yet to be addressed comprehensively.In fact, were OPR the standard, it could help distinguish journals, authors, editors, and reviewers who follow good practices from those that do not. Furthermore, the growing body of openly available publications together with review reports also enables the mining of content and perceptions as well as directions of research, including the assessment of “quality and quantity of contributions to the peer-review process alongside publication record as an additional measure of a researcher’s impact in his or her field”8  consensus building between reviewers and authors. Finally, as an author, you may cite reviews and comments in your revised version, and thus acknowledge these contributions.If the process is opened up to the wider community, additional constructive input (in addition to the reviewer reports) can help to further enhance the quality of a manuscript. Examples for journals with such discussions are the15. Yet, traditionally there have been few obvious incentives for reviewers, beyond a quid pro quo status of mutualism. A recent survey of almost 3,000 reviewers found that 4 in 5 agreed peer review is currently insufficiently recognised, and that reviewers would invest extra effort if review activities were formally acknowledged in research assessments, promotion processes and funding applications16. OPR can facilitate this by making review activities visible, open to inspection, and formally citable (e.g. by assigned DOIs to review reports). Traditionally, as a reviewer, it was only possible to indicate the journals that one had reviewed for — now with open reports linked to reviewer identities, reviews become creditable research outputs in their own right. Crossref has recently adopted a metadata schema that allows DOIs to be assigned to reviews, enabling peer review reports to be persistent research outputs, which can be listed on CVs and ORCID profiles18.Peer review, done well, is hard work — usually taking between 5 hours (median) and 8.5 hours (mean) per person per review13 and in recent optional trials, the majority of authors are willing to publish the full peer review history if given the opportunity . However, as some journals consider posting on a preprint server as pre-publication you need to check beforehand if your preferred journal permits posting on preprint servers .As an author there is less room for direct interaction in the traditional journal setting but with the emergence of preprint servers in many fields you have new opportunities to collect feedback and comments from the scientific community and wider public, both in public and private. If you want to crowd-source additional comments and reviews for your manuscript via open participation, this is still possible even if the venue you submit to does not practice such peer review — simply post your pre-review manuscript to a preprint-server simultaneously with submission and invite feedback through commenting features, e.g. services for collaborative review of preprintsIt is well recognized that the diverse practice of peer review is not without its flaws . In the context of this paper, we have refrained from introducing an “opt out” related to any of the traits of OPR as the option to decline to review seems sufficient.Are there any rules missing? Possible “Opt out if there is a sound reason to do so” could be considered. For example, in pilots with OPR experimentsOne further open question is who benefits most from OPR, e.g. is it the case that there are more positive reviews for well-established researchers or for papers which tackle more trendy topics? Which biases play out in OPR? Are language skills a factor? Are there country, subject or gender biases? How can such effects be controlled/mitigated? For the latter there certainly is a role for e.g. editors, who can amend policies and provide additional instruction and education. Is “open reports” the best we can have given the strong competition in academia?OpenUp project. In addition to exploring workflows and the behavior of all actors involved, OpenUp proposes to collect and aggregate data across publishers in order to evaluate the efficacy of OPR processes33.Some of these questions are currently being explored, e.g. in the context of the EU-funded34. An investigation based on Wellcome Open Research from the first year of operation  showed that reviewers took only a few days longer to review papers of female first authors 35. Further investigation would be desirable, in order to monitor undesirable behavior and identify opportunities for editor intervention.However, the good news is that in OPR settings biases can be monitored and inform interventions. E.g. a recent study revealed that all-female economics papers remain six months longer in peer review than all-male papers, and the author expressed that hope that open peer review may prevent such behaviorOPR is an innovation in scholarly communication that deserves further attention. As we have outlined in the 10 items above, it places a research work in the context of a discussion, and gives authors, readers and others a chance to better understand the process from the initial manuscript submission to final published version. As such it provides excellent learning opportunities and the potential to improve scholarly communication and research towards a more transparent, collaborative and participative undertaking.No data are associated with this article. This is an nice introductory article to a reader who wishes to learn about open peer review. Authors introduce the main concepts and lay down ten items to be considered when dealing with open peer review. It will be useful to early stage researchers and  general readership new to the issue. This is not a research article, but a non-systematic review of existing literature with recommendations from experienced researchers and professionals in the field. In my opinion, the article can be indexed without revisions. pp 4 bottom right says "Thing 8" - should be "Item 8" (?) Minor point:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. I'm glad that this article is already available to be read by those who want to know more about open peer review, particularly young researchers. The article is a useful guide. Ironically, I think that little value will be added by me peer reviewing it openly ot otherwise. If the article had been submitted to a more traditional journal, including many that review openly, then it would not be available for people to read. It might have languished for months while it progressed slowly through the bureaucracy of science journals. Indeed, it would probably be rejected because there is little or nothing that is new in the article. The article would then work its way down the food chain of journals before appearing in an obscure journal where few people would read it, especially if it appeared in a journal behind a paywall. Why have the authors submitted the paper to a journal? Why haven't they simply posted it as a blog and used social media to promote the article? They might easily achieve a higher readership, and presumably it is readers that they want. Or is it? I think that these authors are primarily interested in readers, but sadly the main motivation for many authors is simply to add to their CVs and increase their chances of raising more grants and getting promoted. I am letting my cynicism about the whole process of science publishing shine through, and I would have liked this article more if it had been less accepting of the status quo and more questioning. But that is probably not the role of a simple guide. I don't have any major criticisms of the article, and it will be fine if it published and indexed as it is. But here are some comments in descending order of importance.https://breast-cancer-research.biomedcentral.com/articles/10.1186/bcr2742 and scientists who would not advance string views on their own subject without evidence are quite happy to make statements on peer review without  evidence. For example, in Item 3 the authors write: “Although some early studies found no overall change in quality between single-blind versus identities revealed to the authors (i.e. “open identities”), other studies have indicated that there may be an improvement of the overall quality of review reports under OPR, particularly that comments are more constructive.” They do not make clear that the first study was a randomised trial, a much more reliable method than the later studies, one of which was simply a survey and the other a retrospective study. 1. I think that the authors could have done a better job of discerning the quality of the evidence they quote. As I've argued elsewhere peer review is more faith than evidence based, 2. This is not a scientific study, but the authors do outline what might be  described as their methods in the fourth paragraph. They might usefully expand this, providing more information and explaining how this paper builds on their systematic review.1, which is more up to date and contains more data. They might also have discussed more the objectives of peer review. 3. This is my bias, but I'd have liked to see a short section pointing out that there is little empirical as opposed to anecdotal evidence to support peer review but lots demonstrating its problems. At the end the cite my article in the JRSM, but they would do better to cite my article in Breast Cancer ResearchRoyal Society Open Science that allows reviewers to choose whether or not to be named. This is a tangential point, but I think that offering this option is probably a bad idea. I've experienced a case where a journal rejected a paper on the basis of an unsigned review while also sending the authors a glowing signed review. You can imagine that this seemed unfair to authors. The other strategy the authors suggest--of consulting a mentor--is a much better idea. 4. The authors suggest that tyro reviewers might start with journals, like2. All publishing should be about credit and accountability, and closed review is weak on both. 5. Perhaps the authors might say more about the ethics of open peer review. We introduced open peer review at the BMJ when I was the editor largely on ethical grounds as our studies did not show that open peer review was superior to closed peer review 6. I, like most readers, find it annoying when authors introduce things I've never heard of without saying more. I'd like them not just to give a reference to Open Science Peer Review Oath but tell me a bit more in this paper. Similarly they refer to "the PRO initiative." Is PRO  a reference to the Peer Review Oath? I guess that it probably is, but I don't want to have to guess. 7. I wish the authors had not succumbed to the academy disease of acronyms. I'd lime them to spell out OPR, and there is something particularly silly about referring, as they do, to "the process of OPR."F1000Research. 8. I hadn't heard of the process the authors mention adopted by Niko Kreigeskorte, but I think it a great idea. Like him I will post these comments on my blog as well as on Conclusion As I said at the beginning, I’m sure that some researchers and others will find this guide useful, and I’m glad that it’s already published. The authors might want to revise the paper in the light of my comments, but I don’t think that it’s essential. I support the paper being indexed whether or not they revise the paper.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.


Open access policies have been progressing since the beginning of this century. Important global initiatives, both public and private, have set the tone for what we understand by open access. The emergence of tools and web platforms for open access  have placed the focus of the discussion on open access to knowledge, both for academics and for the general public, who finance such research through their taxes, particularly in Latin America. This historically unnoticed discussion must, we believe, be discussed publicly, given the characteristics of the Latin American scientific community, as well as its funding sources. This article includes an overview of what is meant by open access and describes the origins of the term, both in its philosophical sense and in its practical sense, expressed in the global declarations of Berlin and Bethesda. It also includes the notion of open access managed (or not) by some reputable institutions in Chile, such as CONICYT  and higher education institutions reputed nationally, such as the Universdad de Chile and Pontificia Universidad Católica de Chile. Various Latin American initiatives related to open access  are described, as well as the presence of Chilean documents in those platforms. The national institutional repositories are listed, as well as their current status and a discussion about what open access has implied in Latin America and its importance for the replicability of the investigations carried out locally. Finally, we describe some governmental initiatives (mainly legislative) at the Latin American level and propose some recommendations regarding the promotion and implementation of repositories for the access to scientific data (for access and replication purposes) of the national research. Well known are national level OA policies and regulations from Peru10 and Unpaywall11, and the controversial academic social networks12 that allow self-archiving and distribution of publications such as academia.edu13 and researchgate14. This latter is currently under legal scrutiny for distributing academic publications15.Another important element in the promotion of OA is the development of tools that can accelerate this process, that can direct to publications stored in OA repositories, such as Science Open, 1Science and web browser extensions such as Canary Haz17. According to recent publication statistics, there is a linear increase in publications in OA journals; this is, however, rarely contrasted with the increase in the number of publications in non-OA journals in the region during 2013–2017, a trend that appears to be growing, at least according to indexes such as Web of Science. Therefore, the growth of OA has not had an effect on the number of publications in paywalled journals; instead, we see an increase in both and especially the latter.In the case of Latin America, while there has been important work in pushing some initiatives, some of which have shown progress, results do not seem very encouraging18. The former refers  to OA articles for which publication is associated with article processing costs (APCs), while the latter makes reference to self-archiving, in which case a version of the peer-reviewed article is made available through an online repository or website, and archiving will depend on the policies that each journal imposes19.In this analysis, it is necessary to take into account that OA refers only to documents that have been made available through ’Gold’ and ’Green’ OA modalities20, Gratis OA20, Gold OA22, Green OA23, Hybrid OA24, Delayed OA:25 and Bronze OA26.OA has been characterized by various authors, who have described its different varieties, among which we can identify: Libre OA27.In addition, we should consider the following two non-traditional alternatives: Academic Social Networks (ASN), for-profit social networks that allow academics to share their publications, with more than half of their content being shared illegally; and Black OA, articles shared on illegal pirate sites such as SciHub. The data that Sci-hub has provided with respect to downloads from Chile (February 2016) show that these download concentrate mainly in the capital Santiago, where 273,834 articles were downloaded, followed by Concepción , Valdivia , Valparaíso  and Viña del Mar Academic Social Networks (ASN): corresponds to for-profit social networks that allow academics to share their publications. Although some include OA definitions12, others consider that the content shared through this platforms is not OA since, in contrast to Green OA repositories, these do not check copyrights and therefore, more than half of the its content is stored and shared illegally28, causing controversy29.Black OA: corresponds to articles shared on illegal pirate sites, mainly Sci-Hub and LibGen29.27. Access to content through this kind of OA is common, as if all academic publications worldwide are considered, only 25% are disseminated through any existing form of OA, not including ASNs and black OA30.Related to Black OA, the data that Sci-hub has provided with respect to downloads from Chile (February 2016) show that these download concentrate mainly in the capital Santiago, where 273,834 articles were downloaded, followed by Concepción , Valdivia , Valparaíso  and Viña del Mar 31. However, obstacles in the interoperability between different search engines such as SciELO, Redalyc and LA Referencia31 hinder analyses of the necessary metrics that would allow a clear perspective on the advancement of the OA movement in the region. We must consider that, at least in Chile, the cost of collection subscriptions to publishers such as Elsevier, Springer-Nature, Wiley, American Chemical Society, Annual Reviews, Oxford University Press and AAAS from 2008 to 2017 was $95,754,011 USD for universities, government and other educational institutions, according to Consorcio para el Acceso a la Información Científica Electrónica,32, making it possible to reduce the cost of access to each article from $20 USD to $3.In Latin America, historical OA movements such as SciELO, CLACSO, Redalyc and, more recently, LA Referencia, form the basis of what is understood by OA in the region. It has been argued that what “characterizes the Latin American flavour of OA” is that journals that constitute it are supported by universities, research institutes and other academic organizations without APCs1), Argentina2, México: Decreto Ley3), while others such as Colombia, Brazil and Chile have focused mainly on the management of national systems of digital repositories, despite the absence of mandatory policies for OA in national research.Despite these initiatives, out of the 20 countries considered part of Latin America, only Argentina, Perú and México have national laws that promote OA, especially through the development of institutional repositories , Karl Popper (1902–1994) and George Soros (1930), that propose fostering values such as freedom, progress, equality, fraternity, tolerance, rejection of tyranny, censorship and the exercise of power as a form of control.34, the Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities (2003)35 and Bethesda Statement on Open Access Publishing 36 emerged.The creation of institutions such as the Open Society Institute by George Soros , provided the context in which initiatives such as the Budapest Open Access InitiativeIn general, OA terms and (maybe even more importantly) access to research outputs, generally in the form of scientific publications, mean that social agents besides researchers are able to interact with such outputs, both in sciences and humanities. This democratization of knowledge creates a problem of access: it is not enough to make just the content available, it should also be interpretable by any of those who see it. It is maybe at this point that knowledge moves from being ’visible’ to being ’accessible’.Although this might seem obvious, it is of particular relevance if we consider that many of our individual decisions, as well as our collective political decisions, are many times based  on the knowledge that we have about certain phenomena. This point will be discussed later, when we reflect about what is understood by ’public good’, a fundamental element that often justifies the adoption of OA policies.In order to have a clear perspective of what OA entails at national level, we will take as reference the definitions of OA used by institutions associated with research in sciences and the humanities, taking as an example CONICYT (the largest funding body of science in Chile), Universidad de Chile and Pontificia Universidad Católica de Chile .The first of these organizations is the main agency that funds and regulates academic research in Chile, and the other two are the most prestigious universities in terms of academic and research performance in the country, (highest QS ranked Chilean Universities in 2019).37:To our knowledge, neither Conicyt nor Pontificia Universidad Católica de Chile have a clear definition of what is considered as OA; however, Universidad de Chile states on its website“The Open Access Movement is an initiative that promotes free access to digital materials derived from scientific or academic production, beyond copyrights, that these materials may hold.”“The idea is to enable people to read, download, copy, distribute, search or link these resources without needing to register or pay. This access is often performed through the internet and for this reason, the authors of these materials will receive a higher dissemination of their work. These digital contents can include articles published in online journals, images, data, audiovisual material, and any other digital content whose author wishes to give free access”.38.To comply with these objectives, they propose options to publish in OA journals, among which they include the 5000 magazines present in the Directory of Open Access Journals (DOAJ), and institutional repositories such as Universidad de Chile’s repository CAPTURA, CONICYT’s repository and SHERPA/RoMEo39 and Fischman40 propose 10 different ’flavours’ of OA that correspond to different strategies oriented towards the diffusion and promotion of academic knowledge. Those correspond to 10 different “flavours” of Open Access are: Home page (websites in which a profile of the academics and their publications are made available), E-print , Author fee , Subsidized , dual mode , Delayed , Partial (a part of the content is open access whereas the rest require a subscription). These flavours are specially used by publishers from developed countries to promote OA. In contrast, developing countries as Chile usually adopt Per capita subscriptions  and Indexing . Finally, a certain amount of the academic production in Chile is supported by academic institutions affiliated to SciELO Chile. This last modality is known as ’Cooperative’.WilliskyLatindex : a system for online dissemination that gathers information about scientific research journals, professional magazines and magazines dedicated to scientific and cultural outreach, from countries in Latin America, the Caribbean, Spain and Portugal. It emerged as an initiative from the National Autonomous University of Mexico (UNAM) in 1995, becoming a network for regional cooperation since 1997. It offers data related to print or online journals that comply with academic criteria (in a broader sense) of quality that Latindex has established, as well as access to full academic journals and articles in the different languages used in Latin America. It is the most inclusive source41, since from its origins it used files present in the platforms CLASE, PERIODICA and LILACS, although currently it has a broader coverage. For Latindex, the identification, registry and update of its entries has been difficult due to the different criteria and standards for publication. However, it has been able to gather fundamental data about publications in the affiliated regions. In 2018, Latindex registered 26,010 journals, of which 15,948 are Latin American . Of those, 2,147 are Chilean, which represents 8,251% of the total; among these, 2,038 are still being issued, 105 are not being published anymore and four have an unknown status41.https://www.latindex.org/latindex/regRev): The journal must be at least two years old, item The official or institutional website of the journal will be assessed and must have free access to all its contents, non one pdf journal, among others.Latindex provides users access to a directory that takes into account traditional journals with an international distribution as well as newer journals with limited distribution. The criteria to be included in the directory are the following (SciELO (Scientific Electronic Library Online): a scientific library with a digital database of academic articles belonging to different disciplines, freely available in full text, which allows for preparation, outreach, storage and evaluation of academic literature in electronic format. It works under a set of common criteria of publication and software, and its operation is based on national collections of academic journals with editorial committees and is peer reviewed. One of its main purposes is to make visible academic articles in local languages (Meneghini 2006). By 2018, it indexed 1,285 journals, 145,182 academic articles and 16,943,454 citations (confirma SciELO en números). This initiative was developed by São Paulo Research Foundation (Fundação de Amparo à Pesquisa do Estado de São Paulo FAFESP) and the Latin American and Caribbean Center on Health Sciences Information (BIREME). SciELO Chile indexes 121 journals, of which 107 are still being issued and 14 are no longer published. Each SciELO website is responsible for the costs and responsibilities of obtaining citation data of the indexed articles. In some cases, this is performed by the academic journals and in others, the organization in charge of SciELO42.Redalyc: a more recently created indexing and publication platform. This network publishes more than 1,278 academic journals from Ibero America and the Caribbean. It stores about 47,056 individual issues and 609,283 full text articles and documents of diverse nature, being the database that provides the largest amount of metadata associated to authorship and co-authorship in Latin America. With respect to Chile, RedALyC indexes 93 journals, 37,921 full articles and 110 instalments. The catalogue of RedALyC selects journals in the social sciences and humanities based on a set of criteria : a non-governmental international organization created in 1967 in which 47 countries participate  with a total of 616 research centres in the social sciences and the humanities. It has generated initiatives that promote the development of OA in different regions through the publication of books, journals and other formats through its virtual library45, the Latin American and the Caribbean Library of Social Sciences and CLACSO TV46.LA Referencia: The Federated Network of Institutional Repositories of Scientific Publications, LA Referencia, was created on the 29 of November, 2012, after the signature of a cooperation agreement in Buenos Aires47, with the purpose of providing OA to academic research financed through public funds in the region. It is a network of OA repositories present in nine countries . This platform aims to generate an interoperable system, in which it would be possible to share and make visible academic outputs generated in higher education institutions and scientific research organizations. It is operated by national nodes, storing and hosting academic articles and post-graduate theses. It is based on agreements between public agencies of science and technologies  of the member countries, together with red-CLARA (Latin American Cooperation of Advanced Networks), whose objective is to connect academic computational networks in Latin America49. Its purposes include (1) to realize regional agreements to define common standards and policies, (2) to ’harvest’ the registries of metadata obtained from each national node, (3) to validate the quality of the data obtained, (4) to generate a unified search service (federate) and (5) to facilitate access to full text documents and bibliometric data50. The national node corresponding to Chile has a council of directors belonging to the National Commission of Scientific and Technological Research (CONICYT), and its function is “to strengthen and ensure the access to national and international scientific information with the purpose of research, education and innovation”, by managing the national infrastructure of access to STI information. The policy and regulation that supports its actions is the’proposal for open data’ generated by the government of Chile in 2014, which introduces seven relevant aspects that recipients of CONICYT grants should consider, such as: publish data (non mentioning raw data) and other products in institutional repositories, provide the location to CONICYT, publish before one year from final report, CONICYT “should” publish final report not longer than 3 months after the approval.51. As a platform, it is managed by Web SIC (Scientific Information System) that hosts CONICYT’s Digital Repository, in which it is possible to store and access the results of research, productivity and instruments financed by this agency. The Chilean institutions that participate in LA Referencia can be found on SciELO-Chile and their Institutional Repository.In this respect, CONICYT provides recommendations for OA and preserving scientific information and data in the ’Manual for open data’52.Standardization, regional and international interoperability, regional collection of data and training are some of the main objectives of this initiative, that work together with The Confederation of Open Access Repositories (COAR)The Chilean academic heritage is published and preserved through national efforts of different kinds of institutions, such as universities and government organizations. The efforts have been directed towards the construction of institutional repositories (IRs) and journals that feature OA licenses (OAJ). These store, share and provide access to academic products from institutions and their researchers. This research can be made available in various formats, from theses, monographs and preprints to published documents .Three important initiatives have emerged as a way of organizing and making visible global OA content: Directory of Open Access Repositories , Directory of Open Access Journals  and Registry of Open Access Repositories .9, Chile registers 22 entries, of which 16 belong to higher education institutions, one is SciELO Chile, one the Digital Library of the Museum of Memory and Human Rights and one the digital repository of CONICYT. The rest of the repositories correspond to duplicated or inactive entries.In ROAROf the 22 repositories, 13 correspond to institutional repositories of Chilean universities, four correspond to institutional repositories of government bodies and 13 operate using the DSpace software for the creation and management of digital repositories.In DOAJ, Chile registers 113 OA journals, most of them corresponding to publications related to the social sciences. Among these, 109 do not charge APCs. With respect to licenses, 39 use the licence CC-BY, 26 the licence CC BY-NC-ND, 17 use the license CC BY-NC-SA, 15 the licence CC BY-NC, 12 the licence CC BY-SA (12), and only three use specific editorial licenses . In Latin America, there are various interpretations of the meaning of OA and the consequences of accepting some of these meanings, due to different motivations for participation in this movement58. Whatever its meaning for the organizations in charge of this initiative  in Latin America, it is clear that currently, two ways of OA are recognized: ’Green OA’, in which the papers that have been published under a traditional model or that are in the process of being published are made available in repositories; and ’Gold OA’, in which articles are published in journals that have ’opening’ mechanisms in their publications, generally with publications costs for the researcher or its founders. In either case, the method of access is through the internet.As mentioned above, and according to the Budapest Declaration, OA is defined as an editorial model where access and use of scientific literature is free59, making this knowledge practically inaccessible to researchers and institutions that do not have large funding sources and even more difficult in the case of access for educational purposes and the general public.The impact of both methods has been recently tested and it was been concluded that only 13% of the articles published between 2008 and 2015 are available through Green OA, while 9% were available through Gold OA, showing that most of what is being published is being hidden under a ’paywall’64 is the possibility to critically analyze the corrections of scientific declarations and the conclusions that other scientists have reached and their data analysis, both at systematic and statistical levels65). To fulfill this task, it is necessary that other researchers can reuse, replicate and reproduce the findings reported in scientific publications. This terminology has been a source of confusion  proposes the following definitions of reproducibility:While so far we have stressed the importance of OA for citizens, an important aspect, and probably essential for science itself (and the humanities)Methods reproducibility: provide sufficient detail about procedures and data so that the same procedures could be exactly repeated.Results reproducibility: obtain the same results from an independent study with procedures as closely matched to the original study as possible.Inferential reproducibility: draw the same conclusions from either an independent replication of a study or a reanalysis of the original study.68 proposes that the minimum combination of data would include not only the data presented in the publication itself, but also the unprocessed numerical data that underlie the graphs and quantitative evaluations, the independent repetitions of representative experiments that provide the support for reproducibility in the results and sets of large scale data generated by the study69.To achieve reproducibility, it is necessary that researchers share materials, protocols and data that sustain findings, with the purpose of confirming previous studies or becoming the basis for new studies. For example, Nature Cell Biology70 or Dryad71. This databases can also be complemented, depending on its appropriateness, with tools like Zenodo72, Github73, Gitlab74 or OpenScience Framework75, among others, as platforms for publication, as well as for preprints, data and supplementary code resulting from a research project.These data should be available with the published document . However, larger datasets could be stored in public databases and their location specified in the corresponding document. In the absence of specific OA databases , Nature recommends to store data in general data repositories, such as Figshare76 journals found that, as reflected in the instructions for authors and their editorial policies, only a small percentage (11.9%) declare explicitly that sharing data is a necessary condition for publication, 9.1% requested shared data (without this being an explicit condition for publication), 23.3% of journals promote sharing data but not as mandatory, 9.1% indirectly mention the relevance of sharing data and 14.8% propose to share data of proteins, proteomics and genomics. The 31% left does not mention data sharing practices. Although, as the authors mention, 65,7% of the journals require data sharing as a criteria for replicability, they do not provide specific policies or guidelines about these practices that could ensure that the data is available and reusable.Despite the relevance of OA databases, a study performed in 318 biomedical research77. The article examines the social agreements that regulate the behaviour of academic researchers through an opinion survey in which 185 active researchers participated .In Latin America, and in particular in Chile, little emphasis has been given to the importance of access to data or the development and promotion of platforms to store this type of research output with the purpose of promoting reproducibility. A Latin American study analyzed whether there are differences in the beliefs that regulate research between natural and social scientistsThis study proposes four fundamental  agreements that constitute scientific work, including:The world has laws or regularities that are understood through observationYou must have the ability to analyze in an objective, impartial, verifiable and systematic way the information provided by reality .Have mastery of technical aspects of the work, the correct use of the recording devices, the baseline calibration for each experiment, the design of appropriate controls , among others.The results should be communicated in an open way, that is, verifiable or replicable.The most remarkable element of this work in terms of OA (as a means of communication of science) is precisely that expressed in the last point, since the interviewees  mention that the importance of ensuring the veracity and verifiability of the data reported by an investigation are necessary for the advancement of knowledge.79 discusses what has been going on with the so-called ’reproducibility crisis’ of certain scientific disciplines, but makes no mention to what this implies for the future of Latin American science. The only claim that CONICYT has made on this topic was to define what is understood as scientific knowledge, including in this definition the idea of reproducibility (without a deeper discussion of the concept):Although this research does not discuss how to achieve reproducibility through the use of OA databases, it is one of the few precedents in this topic in Latin America. Additionally, in some blogs on the SciELO website, the problem of replicability has been superficially discussed. In these blog entries, Lilian Nassi-Caló80.“Scientific knowledge, generated in this way, has the qualities of reproducibility and objectivity. Precisely herein resides an essential part of its enormous utility, since its predictive value applies to all situations in which the established conditions are reproduced, despite the subjectivity of the observer”As mentioned, Chile does not have clear perspectives or statements about public databases and their relevance, not only for OA itself, but also for reproducibility of Chilean and international science, since it is possible that in the near future, what we today call impact factor will be highly conditioned by the reproducibility of the results of research.In this light, it seems evident that the exchange of data and its accessibility is fundamental to the advancement of knowledge in the natural sciences, social sciences and humanities, since it allows validation and to contribute with new knowledge about information already published. This supports the development of research and innovation at national and international levels, together with democratization of access to such data or registries in many cases. All this will happen only if this information is available, in an organized and coherent way, according to the declarations of data availability. This can be observed both in prescriptions of governmental mandates, as well as editorials in charge of access and dissemination of the outputs of research. To learn more about this, please see84. Most of these policies have been developed by research organizations. Despite Latin American efforts, to date (March 2019), Chile is not present in this database.As mentioned above, OA and the availability of resources are important, not only for the development of new knowledge, but also for social, economic and political advancement in a democratic society. If we only analyze South American countries, according to ROARMAP, only 49 policies have been adopted in this region, of which seven correspond to Argentina, one to Bolivia, six to Colombia, eight to Perú and four to Venezuela. Brazil leads the list with 23 policiesAs we discussed in this paper, although there are some Latin American initiatives that support OA, mainly in the use of licenses and the development of international databases , it is important to note that there are still important challenges for interoperability and access to documents .85, among others.It is important to highlight that university repositories are an essential element in the diffusion and communication of knowledge. The number of digital documents that universities systematically produce, considering academics, undergraduate and postgraduate students, are elements that can help us define the profile of the university, regional, national and international research, and its opportunities for collaboration. This goes far beyond simple university outreach and public engagement activities, and should be treated as a research process in itself. This is why it is an essential element of the academic cycle, implying a responsibility for the administration in the development of technological capacities, as well as human resources within the institution . This is not a new revelation; in fact, it has been mentioned by the Association of Research LibrariesA fundamental role of universities is the dissemination of knowledge and the recognition of intellectual capital and knowledge produced in them. For this, they must develop strategies that guarantee the distribution of the generated content, which implies that they must recover the ability to manage their intellectual capital in order to promote the resolution of local  problems. Therefore, it is important that local governments promote (and finance) these types of initiatives.85 point out various points of relevance when considering the current status of Plan S, important elements to be considered by Latin American countries for approval by researchers in Latin American financing agencies, including: (1) quality versus number of publications ; (2) increases in the costs associated with publication that could only be financed by governments and richer institutions, generating a gap of inequality; (3) segregation of research quality, diversity and isolationism ; (4) would compromise the peer review process ; and (5) a reproducibility crisis . In particular for this last point, we point out that the use of models and knowledge management platforms that combine being reader-friendly with a depth of detail that allows critical reading by specialists and non-specialists is required. It is evident that the countries of Latin America will be at a disadvantage in the face of these type of measures, considering that most of the research funding is through public funds. This is particularly important if we think in the little funding (or lack in some cases) to promote open access in South America and the redundancy of initiatives trying to ’harvest’ publications and their metadata and trying to make different platforms ’interoperable’. We propose instead that the development and maintenance of institutional repositories and the use of already existing platforms should be prioritized.The latter is of particular relevance considering the current state of OA worldwide and the recent emergence of Plan S, an initiative developed by cOAlition in 2018 and revised in 2019 and that proposes 10 principles so that in 2021, publicly funded research is published exclusively in OA journals. Several of the principles are ambiguous and inaccurate and some controversial, such as the impediment of researchers to publish in hybrid journals, preventing researchers from publishing in about 80% of academic journals, including Nature, Science, The Lancet, etc. In response to this, more than 1500 researchers, including two Nobel Prize winners, signed a letter arguing against the plan. Agostini and BerkOur proposal aims to promote and strengthen the use of institutional repositories as disciplinary , in parallel to the efforts made to increase the quality of research in the region mainly in strengthening the reproducibility and/or replicability of the investigations developed.? ], facilitates research uptake and understanding. Furthermore, when implementing an institutional repository in Latin America and, in particular, in the Chilean context, it is necessary to explore and focus efforts on the prescriptions provided by institutions with a well-defined trajectory, such as The Repositories Support Project (RSP), an initiative created to “contribute to the creation of capacities, knowledge and abilities in higher education institutions of the United Kingdom”84. RSP has developed a repertoire of considerations to select the best platform to be implemented, taking into account technical aspects86, metadata, repository management, outreach and user engagement and a checklist87.We should also consider what is indicated in the document OpenUP – 7107220 Deliverable D4.3 Good practices and lessons learned. Briefly, research dissemination, as defined by Wilson  reference in page 9subtitles require to be more systematic E. Check details of form: I approve this paper under the condition that the authors follow the corrections indicated.I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.


Transparency lies at the heart of the open lab notebook movement. Open notebook scientists publish laboratory experiments and findings in the public domain in real time, without restrictions or omissions. Research on rare diseases is especially amenable to the open notebook model because it can both increase scientific impact and serve as a mechanism to engage patient groups in the scientific process. Here, I outline and describe my own success with my open notebook project, LabScribbles, as well as other efforts included in the openlabnotebooks.org initiative. This Research Matters article proposes that open notebooks present an opportunity for researchers to showcase their work, push the boundaries of open research practises and rapidly advance science in their field by making an immediate impact in their communities. The term “open notebook science” was first coined in 2006 by Professor Jean-Claude Bradley, a Canadian chemistry researcher at Drexel University. In defining the practice, Bradley said, “It is essential that all of the information available to the researchers to make their conclusions is equally available to the rest of the world” , meaningStarting my own open notebook for my postdoctoral research project was appealing for a number of reasons. I am a postdoctoral fellow at the Structural Genomics Consortium (SGC), where open science is a critical part of the laboratory ethos. SGC scientists not only make their work as open as possible through extensive data and material sharing but have also recently implemented an open publication strategy, in which all manuscripts are submitted systematically to open access preprint servers. Piloting innovative open science strategies is well supported and is encouraged for scientists working within the SGC.My particular research focus is Huntington disease (HD), a devastating inherited neurodegenerative disease. Although scientists mapped the causative mutation 25 years ago . Open scThe notebook format I have adopted is inspired by the groundwork laid by other open “notebookers”: it’s designed to be discoverable, accessible, clear, and detailed in its presentation, and to permit dialogue between readers and me, and to pave the way for collaborations. After I finish an experiment, I write up all the details of the materials and methods as well as all the data and analyses, ensure they’re in suitable file formats, and upload them to Zenodo as a data set licensed with Creative Commons Attribution 4.0. I then write a lay summary of the experiment on my LabScribbles blog, linking to the data set deposition, with the aim of reaching a wider nonspecialist audience. In this, my aim is to give context to this experiment within my broader research project, as well as provide guidance on points for improvement and next steps. Links to the Zenodo and blog post are then shared via a Twitter post.Sharing my work via the open notebook has helped both my professional and personal development. The availability of my work has attracted collaborations with more than 10 international laboratories on my HD research. Data sets, materials, and instrument access are being freely shared back and forth, and I have received excellent feedback and advice from more senior researchers. Professor Ray Truant, an established HD researcher at McMaster University, is now one of my mentors for my Huntington’s Disease Society of America (HDSA) Berman-Topper Family career development fellowship. Professor Susan Lea, an expert in protein structural biology and director of the Central Oxford Structural Molecular Imaging Centre (COSMIC EM) facility at the University of Oxford, is a collaborator for electron microscopy work, work funded by a successful joint grant application to the Huntington Society of Canada. My writing and communications skills are also improving rapidly, particularly my ability to communicate with both scientific and general audiences. My professional network is also growing beyond the scientific sphere to include different stakeholder groups, including numerous HD patient organizations. I have also found that writing the notebook has improved my project planning skills and facilitated my progress through regular written reflection of my research.Science is a competitive field, but I have been pleasantly surprised by the community’s response. Although many scientists might fear being scooped if they adopt open notebooks, my notebook has served as an opportunity to showcase my research project in the HD field, which has in turn protected me from being scooped by stamping my preliminary work and findings with a digital object identifier (DOI). The success of the project has subsequently led to generous funding support from HD research funding agencies, including the CHDI Foundation, the Huntington Society of Canada, and a career development fellowship from the Huntington’s Disease Society of America.These successes, however, depend on commitment. The biomedical open notebook community has launched many efforts with enthusiasm only to lose steam and interest. In an effort to catalyze a sustainable community, earlier this year, the SGC launched a wider open notebook initiative with the inception of openlabnotebooks.org , a platfIn an age when sharing daily aspects of our lives online through social media and other platforms is normalized, it’s a logical step to apply similar practices to scientific research. Open notebooks present an opportunity for researchers to showcase their work, push the boundaries of open research practices, and rapidly advance science in their field by making an immediate impact in their communities.


Open peer review (OPR) is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. As more journals move to implement and experiment with the myriad of innovations covered by this term, there is a clear need for best practice guidelines to guide implementation. This brief article aims to address this knowledge gap, reporting work based on an interactive stakeholder workshop to create best-practice guidelines for editors and journals who wish to transition to OPR. Although the advice is aimed mainly at editors and publishers of scientific journals, since this is the area in which OPR is at its most mature, many of the principles may also be applicable for the implementation of OPR in other areas .The online version of this article (10.1186/s41073-019-0063-9) contains supplementary material, which is available to authorized users. BMJ and BMC  for almost 20 years, it has gained ground recently, spurred by the Open Science agenda for increased transparency and participation in scientific processes. Many publishers and journals already run some form of open peer review, including BMC (owned by Springer Nature), BMJ, Copernicus, eLife, EMBO Press, F1000Research, Nature Communications, Royal Society Open Science and PeerJ. In 2018, an open letter was published in Nature [Openness in peer review has been labelled an ‘increasing trend’ . AlthougOpen peer review (henceforth OPR) can take place in different stages of the review process—pre- or post-publication—offering extended communication and knowledge exchange between researchers. Platforms and publishers implement OPR tools to encourage wider and more transparent discourse within the review process. Yet, the openness of these systems often differs in terms of what is revealed to whom and when. To bring clarity to how the term ‘open peer review’ is used, a systematic analysis by one of the current authors analysed definitions of OPR in the literature, identifying seven core traits which were used in 22 distinct configurations . Across F1000Research authors rated transparency of the peer review system, including revealing reviewer’s names, as important or very important reasons for publishing with that venue by the majority of respondents [Attitudes to the elements of OPR vary greatly among researchers. A recent study by one of the current authors  found thpondents .how to ‘open’ peer review, it becomes urgent to offer clear guidelines for those publishers and editors interested in taking up such practices.OPR is attracting increasing attention, but there are a diverse cluster of interrelated but distinct innovations, which can be combined in a myriad of combinations, that fall under this term. Hence, any publisher wishing to move in this direction faces crucial choices about which elements of openness to embrace, and these decisions will in their turn expose them to potential advantages and disadvantages for the quality of their peer review systems. Which OPR system is optimal, for which communities and in which circumstances? How should these systems be implemented, and what opportunities and pitfalls should be recognised? Given such growing interest in open peer review, combined with such a proliferation of options for This article addresses this need. It presents a series of structured guidelines for the introduction of the various open peer review traits for publishers and editors. The guidelines were produced in close collaboration with a group of experts in peer review, and especially open peer review, from publishing and research. Methods included via background research, expert interviews and an expert synthesis/validation workshop. The guidelines seek to be of use to those who oversee the peer review of manuscripts for publication who are considering introducing more transparency or inclusivity to their peer review processes by implementing any of the innovations grouped under the term ‘open peer review’. The paper first gives general advice which cuts across all OPR elements, before going on to detail specific advice for each of the OPR elements. Although the advice is directed mainly at editors and publishers of scientific journals, since this is the area in which OPR is at its most mature, many of the principles may also be applicable for the implementation of OPR in other areas .Peer review is a method of scholarly quality assurance that serves to validate the soundness, substance and originality of a work, to assess and help improve it until it meets required standards for these criteria, as well as sometimes to select for ‘appropriateness’ or ‘fit’ for certain venues. Peer review, as it relates to scholarly publishing, is here understood as the formal scholarly process where an editor sends copies of a manuscript to neutral third-parties judged knowledgeable enough to be able to comment on its quality and suitability for publication. Standard peer review is understood to be typically:Anonymous: either reviewer identities are kept from authors (single-blind) or author and reviewer are unknown to each other (double-blind). In some cases, authors identities are also hidden from editors (triple-blind)Confidential: the process takes place behind closed doors  and reviews are not publishedSelective: reviewers are chosen by the editorAccountability: The increased transparency offered by open identities and reports could increase accountability and make reviewer conflicts of interest more apparent. Open participation could reduce possible problems with biases or elitism associated with editorial selection of reviewers [eviewers ; on the eviewers .Bias: Open reports allow the scientific community to examine how publication decisions were made. However, open identities remove the anonymity for reviewers (single-blind) or authors and reviewers (double-blind) which has traditionally been used to counteract such biases [h biases , 10.Inconsistency: Open identities and open reports could improve the quality of reviews, encouraging reviewers to be more thorough in their assessments  [he case) . Open pahe case)   .Time: Peer review often takes a long time. Publishing manuscripts online in advance of peer review, either as pre-prints or as part of the publisher workflow, speeds up dissemination and (in disciplines like Physics) enables researchers to claim priority in a finding [ finding . Open pl finding , and ope finding .Incentive: If review reports were published alongside reviewer names, it would be easier for researchers to claim credit for these activities, thus incentivising review [g review . Open paWasted effort: Rather than hiding the useful contextual information contained in peer review, open reports would make this available [vailable .As has been said, open peer review is a complex phenomenon, which actually represents a range of possible innovations to standard peer review processes. Each of these innovations addresses different issues with standard peer review. Following the summary presented in , we offeBMC (part of Springer Nature), BMJ, Copernicus Publications, eLife, Elsevier, F1000Research, Hindawi, MDPI, Nature (part of Springer Nature), PLOS, Royal Society Open Science, Taylor & Francis, Wiley), along with representatives from Publons and the PEERE research consortium. All data was collected and analysed by the current authors: Dr. Tony Ross-Hellauer  and Dr. Edit Görögh , researchers in Open Science. The authors have rich previous experience in performing qualitative analysis in these areas. Seven of the participants were already known to the authors, but otherwise, no relationships were established prior to study commencement. Prior familiarity with some participants means, however, that the authors were to varying degrees aware of the broad stance of some participants towards open peer review before the study commencement.The guidelines were created in close consultation with a group of experts. The views of expert participants were sought via an interactive meeting, a brief pre-meeting questionnaire and subsequent sharing of drafts of the guidelines with all participants for feedback. Participants were chosen for their expertise in peer review—especially open peer review—and, although care was taken to include conservative voices, the topic and aim of the meeting attracted people ‘open’ to open peer review. The aim was to have most large publishers represented. The list of 15 experts who took part in the workshop, and others who contributed to the pre-meeting questionnaire and commented on drafts of the guidelines, is listed in the ‘Acknowledgements’ section of this article. All consented to be named. They represent experts in peer review and open peer review from many major publishers  and two received no response. One confirmed participant was unable to attend due to illness, meaning that 15 participants took part in the workshop . All represented major publishers except one participant from Publons  and one active peer review researcher (from the PEERE network).The interactive workshop took place on 27 March 2018 at The meeting took the form of an interactive workshop which lasted 90 min. No audio or video recording of the workshop was made, but extensive field notes were kept by the researchers. No one else was present besides the participants and researchers. The session was moderated by Edit Gorogh with note-taking by Tony Ross-Hellauer and additional facilitation from Elisabeth Moylan, one of the attendees. Edit Gorogh first presented the motivation for the exercise. Participants were first asked to consider in pairs to consider the questions: ‘For those who have implemented/experimented with OPR, what one thing would you have done differently?’ or ‘For those without OPR experience, what’s the one thing you’d really like to know first?’ Each pair was then asked to report back to the group the main points of their discussion, enabling free-form discussion of main points of interest.The group was then split into two, and each group asked to discuss and record their main advice under each of the headings identified as main common themes following the interviews and questionnaire. Following group discussion of 15 min, each group reported back to the whole group, whereupon each group could give feedback on each other’s answers. In a final stage, the group was split into four sub-groups (self-selected) who then each provided advice specific to individual OPR traits . Again, each group reported back to the whole group for further discussion. The researchers kept notes during the discussion of major and minor themes.Following the workshop, the authors of this study used the answers from the workshop to further refine the structure and content of the guidelines. First, notes taken during the meeting were collated, and major and minor themes identified according to a grounded theory approach employing open and axial coding. Following this, a first draft of the guidelines was written and shared with all expert participants via Google Docs for collaborative feedback and further refinement. Following submission of this manuscript, and resulting useful critical comments from two very thorough reviewers, a revised draft was created in which the advice was further refined. This version was then shared with all participants in December 2018 for their further feedback before resubmission.Any journal editor or publisher wishing to implement some form of OPR would be well-advised to first do their homework. What do you want to achieve? How? For which reasons? Answering these questions first will enable you to orient your engagement with OPR. Examine which particular aspects of your peer review processes you would like to improve. For example, do you want to increase the transparency of your processes, give credit to peer reviewers, enable greater participation, or just speed up the peer review process? Being clear on these primary goals is vital.Open identities: authors and reviewers are aware of each other’s identity.Open reports: review reports are published alongside the relevant article.Open participation: the wider community are able to contribute to the review process.Open interaction: direct reciprocal discussion between author(s) and reviewers, and/or between reviewers, is allowed and encouraged.Open pre-review manuscripts: manuscripts are made immediately available  in advance of formal peer review procedures.Open final version commenting: review or commenting on final ‘version of record’ publications.Open platforms (‘decoupled review’): review is facilitated by a different organisational entity than the venue of publication.As discussed above, ‘open peer review’ can mean different things to different people. As a first step, familiarise yourself with the differences between each of these elements. For example, one of the current authors created a taxonomy of seven core traits:Read widely to familiarise yourself with the pros and cons of each of these elements .Consider directly surveying community opinions regarding open peer review models to gauge attitudes. This may work especially well for journals with close-knit communities—for example, society journals, which regularly seek feedback from authors or society members regarding journal policies. Alternatively, or as a complement to this strategy, consider targeted ‘qualitative interviews’ to gather insights from those with particularly strong opinions regarding open peer review.Engage journal communities—firstly by consulting your editorial board and reviewers to get them on board with the idea. It may be necessary to ‘sell’ the benefits of opening up peer review and provide reassurances. A committed and engaged Editor who can drive such discussions may help here. Find keen researchers to work with and gauge interest in the model among communities the journal serves. Let reviewers, authors and readers know in advance, and if you are unsure of how such developments might be received, consider announcing plans in a journal editorial and seeking community feedback. In any case, include requests for community feedback in any such announcements to ensure alignment with researcher attitudes.A deciding factor in your prioritising the elements of openness to include will be the technical possibilities of your system. Whether you are a small publisher using open source software or a large publisher which uses one of the major manuscript handling services, if your electronic editorial office and production/publication systems and workflows cannot currently be easily configured for OPR elements, they may be difficult and/or expensive to implement.It is important to recognise potential costs in advance. As things stand, there is a lack of infrastructure to facilitate automated workflows for many of the elements of OPR. Hence, development costs may be a major barrier—especially for smaller players. Ask yourself: Which options does your system already support, and do you have the technical staff or resources to fund system development? Consider also that costs will likely not only be in initial implementation , but also ongoing support costs . If your needs would require significant custom system adaptations from a third-party service provider, you might consider partnering with other publishers who use these services to spread costs in implementing these changes. Alternatively, some platforms are now offering specific OPR functionality to work together with more traditional publishing services. In any case, be aware that there will usually be different ways, with differing levels of elegance and cost, to implement OPR options. For instance, publication of peer review information could be as simple as manually compiling review components and publishing a single document as Additional file If you are just experimenting with OPR, it may be that rather than immediately extending your whole publication architecture, it might be better to start small with workarounds, although be aware that ad hoc workarounds may produce a less smooth user-experience which could affect uptake and user attitudes to the experiments. Consider, however, that the sub-optimal nature of workaround solutions may then become an inhibiting factor in the success of the experiment. One solution here would be for a third-party OPR platform to offer their service as a plug-in to existing workflows for conducting such experiments.Be flexible and choose your battles carefully. Change is difficult and you may run into problems if you try too many things at once. Your communities may be more receptive to some elements than others, and so, prioritising the areas you would like to change and being prepared to compromise from the ideal situation or at least take a phased approach may help you maintain traction and community buy-in. It will also make it easier to systematically assess the success or otherwise of any particular innovation.For elements you would like to introduce but think might prove controversial, you could make them optional. Thereby, it is possible to signal your support for this innovation while allowing reviewers or authors to opt-out. Note, however, that default policies may significantly affect outcomes—if the default policy is opt-in, this might lead to lower participation than if the default were to opt-out, for instance. If reactions among research communities may be uncertain, consider introducing OPR through a pilot study with an accompanying survey for participants which would show that any final decisions would be based upon real experiences, whilst allowing the journal to experiment with the confidence of the community.Once you have decided on the model you’d like to move to, you have your communities on board, and have prioritised which OPR elements to implement, you will still need to sell your communities on the concept. As a general strategy, you should engage with the research community to find academics who are enthusiastic about OPR to be ‘open champions’ in advocating to their peers—for example, by engaging people who responded positively to your initial community consultation in step B. Moreover, the arguments above in favour of the various aspects of OPR will help sell the concept, especially with regard to increasing transparency, enhancing credit for review activities and demonstrating and  potentially enhancing the quality of reviews.Misunderstandings could derail processes. As the stewards of the peer review process, publishers and editors have a duty of care to ensure reviewers and authors fully understand the systems of peer review in which they participate and its potential advantages and disadvantages. Use editorials, webinars, infographics and/or blog posts to articulate decisions and justify why these decisions have been made. Formulate clear policies which are easily findable on journal webpages for authors and reviewers.There is a need to track review quality and acceptance rates to monitor how OPR affects processes. As said above, it is good to decide a vision for the kind of peer review you want in the context of your end-to-end publication workflow and then prioritise goals in order to reach this vision. A key part of this planning should be deciding how you will define and evaluate success. Have a clear framework for assessing success . Systematically collect data and study the impact of the practice on journal performance. Key questions could be the following: is review quality improved? Is it more difficult to find reviewers? Are review times impacted? Are open reports being consulted and re-used? It is also advisable to consult with your journal community once the new process has been in place for some time, perhaps via survey, to gauge the development of their attitudes towards processes. Important here is to establish ex ante which quantifiable measures or performance indicators will be used for internal analysis. Outcomes should always be considered on an appropriate time scale, however. Change takes time.Bear in mind that cultural change takes time, and so, even where uptake is not as quick as wished, the broader ethical aims of transparency and accountability in scholarly publishing might make persistence desirable in spite of low uptake. However, if things really are not working, then it may be necessary to re-evaluate your goals in light of lessons learned. For example, revisit the advice in the ‘Giving updates on progress will enable community engagement, keeping authors, reviewers, editors and publishing staff updated on the progress of your initiative. These updates will also help others decide whether and how to implement similar approached. There is currently a lack of real scientific evidence on the efficacy of many traits of OPR. Once enough evidence has been gathered, consider writing up the results as a scientific study for peer-reviewed publication. Alternatively, consider partnering with peer review researchers from the start to ensure data is well-formed for such analyses and to enable rigorous external scientific analysis.Open identities peer review is a review where authors and reviewers are aware of each other’s identities. Reviews with open identities can seem to be more constructive in tone . There hMake sure to create a new standard reviewer invitation email which includes a clear description of the open identities review process and its potential advantages as well as disadvantages, as well as a standard follow-up text which goes deeper into these issues to convince those who are reticent. If you are keen to invite a specific person who is reticent, be ready to negotiate to persuade them by further explaining or, for example, offering more time to review if a reviewer believes they will need more time under such circumstances.A common concern regarding open identities is that junior researchers who give negative reviews to more senior colleagues may face retaliation in some form. While it is important to note that there is at present only anecdotal evidence of this, such concerns seem nonetheless very common and underlie much of the negative response to open identities. It is hence essential to deal with these concerns early on to set authors and reviewers at ease and limit the risk to which researchers who take part in review processes are exposed. Have in place clear processes for dealing with any reviewer concerns and encourage any reviewers experiencing negative consequences to contact the journal as a matter of academic ethics. Publishers should also use their experiences to contribute to the evidence base on this issue by monitoring whether open identities leads to more positive reviews overall, for example.Wherever reviewer names are disclosed along with publication, be sure to use identifiers  to link that activity to reviewer profiles and further enable credit and career evaluation. Assigning persistent identifiers like ORCID is a crucial element of best practice within current publishing practices. Moreover, if your journal is not yet participating with Publons, also consider enabling this partnership to further enable reviewer credit.eLife and MDPI). As said above, to signal the journal’s support for the concept but allow reviewers choice, you could make open identities the default but enable reviewers to opt out of the process. Another possibility would be to maintain a standard single- or double-blind review process but to publish reviewer names alongside the final article (the practice at the publisher Frontiers). Bear in mind, though, that changes in these conditions could introduce biases whereby reviewers who are inclined to be more lenient towards a manuscript may be more likely to accept the review.This point was made above for OPR elements in general, but is worth applying to open identities specifically since this is one of the most controversial elements of OPR. If you are interested but not ready to fully commit to open identities, as suggested above, you could start small with a pilot and scale up. Alternatively, you could allow reviewers to opt in or out , reviewer names are published alongside the reports. The main benefits of this measure lie in making currently invisible but potentially useful scholarly information available for re-use. There is increased transparency and accountability that comes with being able to examine normally behind-the-scenes discussions and processes of improvement and assessment and a potential to further incentivise peer reviewers by making their peer review work a more visible part of their scholarly activities   are published alongside the relevant article  Fig. .Fig. 1Sc‘Peer review materials are attached to the article as a single or numerous PDFs. Whether these materials are pulled together into one document or attached as separate documents, there should be some defined mechanism in the JATS XML tagging that would support the capture of any available metadata and identify these files in a machine-readable and interoperable way for publishers to tag this content appropriately.Peer review materials are appended to the article within the full text  as a sub-article component of the XML.Peer review materials are full-text XML ‘articles’ or ‘commentaries’ in their own right that link bidirectionally to the main article.’There is not yet an established industry standard for how to publish peer review reports, but this situation is now changing. The current best advice here is from Beck et al.’s 2018 article ‘Publishing peer review materials’, [The options presented are in order from most basic to most complex, but also from least to most desirable. This means best practice would be option 3, but most pragmatic would be option 1. In addition, machine-readable metadata should accompany the content.Beck et al.  advise tOpen participation peer review allows the wider community to contribute to the review process. This can be either during publication (by making a pre-review manuscript openly available online as a pre-print or discussion paper) or after publication  or whether to require some credentials before allowing comments. Various options are available depending on your own communities. At entities  Fig. 2)CopernicuAtmospheric Chemistry and Physics (ACP), which publishes pre-review discussion papers for community comments, only about one in five papers is commented upon [A further crucial issue is that open participation processes often experience low uptake. Thus, as already said, they are hence often used as a complement to a parallel process of solicited peer review. At the open access journal ted upon . Hence, In traditional peer review, reviewers and authors correspond only with editors. Reviewers have no contact with other reviewers, and authors usually have no opportunity to directly question or respond to reviewers. Open interaction peer review allows and encourages direct reciprocal discussion between reviewers and/or between author(s) and reviewers.EMBO JournaleLife.FrontiersAllowing interaction among reviewers or between authors and reviewers, or between reviewers themselves, is another way to ‘open up’ the review process, enabling editors and reviewers to work with authors to improve their manuscript. It is therefore important to decide what workflow you will follow. Examples of journals which enable pre-publication interaction between reviewers are the eLife consultation approach involves more work upfront , but time is saved later on if the editor decides on the revised version rather than sending back to the referees [While this extended dialogue might be expected to increase the editorial workload in some parts of the process, publishers practicing such methods actually report that they can also reduce workload in other parts. For example, the referees .Open peer review is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. As more journals move to implement and experiment with the myriad of innovations covered by this term, there is a clear need for best practice guidelines to guide implementation. This article has aimed to address this knowledge gap, reporting work based on literature research, expert interviews and an interactive stakeholder workshop to create best-practice guidelines for editors and journals who wish to transition to OPR. The guidelines offer practical and pragmatic advice to these purposes at both a general level and for specific OPR traits. Main points of guidance are (a) set open peer review goal(s), (b) listen to research communities, (c) plan technologies and costs, (d) be pragmatic in approach, (e) further communicate the concept and (f) evaluate performance.It is important to recognise some limitations of our approach to the creation of these guidelines, however. Firstly, our approach was to bring initial ideas forward individually and anonymously, but then to iteratively work on the development of the guidelines collaboratively with our whole group of experts. While we believe this open, collaborative phase enabled rich and in-depth discussion, it is possible that these group dynamics resulted in some ‘bandwagon effects’ whereby our experts influenced each other in their stated opinions. Secondly, these guidelines have not yet been pilot-tested, as would happen in the development of, for example, formal reporting guidelines. Nonetheless, given the range of experience and knowledge that has been incorporated here, we are confident the guidelines bespeak the best advice possible for the implementation of open peer review at this time. The area of open peer review is fast evolving. As journals and publishers look to experiment with new processes, it is our hope these guidelines prove useful in setting expectations and guiding best-practice.Additional file 1:Background resources for open peer review implementation. (DOCX 24 kb)


The Bulletin is fully open-access and charges no author fees. The mission of the Bulletin welcomes a variety of unsolicited manuscripts . These are initially screened in house for originality, relevance to an international public health audience and scientific rigour. If they pass the initial screening, they are sent to peer reviewers whose opinions are taken into account by the journal’s editorial advisers when they decide whether to accept a manuscript for publication. Accepted papers are subject to editorial revision, which may involve substantive changes, shortening or restructuring the text and deleting superfluous tables and figures. The word limits given for each type of contribution do not include the abstract (where applicable), tables, boxes, figures and references or appendices, if any. The principal types of manuscripts are outlined below.The 1.1.1.Bulletin. All manuscripts destined for the first four of these sections must include two paragraphs indicating what they add to the literature. The paragraphs should briefly explain:We welcome unsolicited submissions to the Research, Systematic reviews, Policy & practice, Lessons from the field and Perspectives sections of the what was already known about the topic concerned;what new knowledge the manuscript contributes.http://www.equator-network.org/about-us/uk-equator-centre/equator-publications/equator-network-publications-2010/. Operational and implementation research should be reported in compliance with the guidelines published in this issue ; peer reviewed. As clear reporting is needed for readers and reviewers when judging the quality of research, studies should comply with the relevant reporting guidelines, available on the EQUATOR Network website, at: http://www.equator-network.org/reporting-guidelines/prisma/. Exhaustive, critical assessments of published and unpublished studies on research questions concerning interventions, policies or practices in public health, with meta-analysis when feasible. Not more than 3000 words plus a structured abstract ; the number of references in accordance with the scope of the review; peer reviewed. How studies were included and excluded should be illustrated in a flow diagram. Authors should strictly follow the reporting guidelines for systematic reviews and meta-analyses (PRISMA) available at: Analytical assessments, debates or hypothesis-generating papers; not more than 3000 words and 50 references, plus a non-structured abstract ; peer reviewed.https://www.who.int/bulletin/volumes/84/1/3.pdf). Operational and implementation research reports should follow the specifications described above for the research section. Papers that capture experiences and practice gained in solving specific public health problems in developing countries. Convincing evidence of effect should be provided. Not more than 1500 words and 15 references, plus a structured abstract ; not more than one table and one figure; must include one box listing three lessons learnt; peer reviewed  surrounding an issue of public health interest; up to 1500 words, no more than 12 references; peer reviewed.1.1.2.The categories of articles shown below are normally commissioned by the editors. Authors wishing to submit an unsolicited manuscript for one of these categories should first contact the editorial office .Authoritative reviews, analyses or views of an important topic related to a theme or to one or more papers published in a given issue; not more than 800 words, maximum 12 references.A base paper on a controversial current topic in public health (not more than 2000 words and an abstract) is the core of a debate by several discussants invited to contribute not more than 500 words each.1.2https://www.wma.net/wp-content/uploads/2016/11/DoH-Oct2008.pdf) and with the additional requirements, if any, of the country in which the research was carried out. Any manuscript describing the results of such research must contain a clear statement to this effect and should specify that the free and informed consent of the subjects or their legal guardians was obtained and that the relevant institutional or national ethics review board approved the investigation. The Bulletin is a member of the Committee on Publication Ethics . Issues involving publication ethics may be referred to this committee by the editors. WHO Research Ethics Review Committee clearance is required for papers that report research supported by WHO or that are authored or co-authored by someone who was a WHO staff member while the research was conducted.The World Health Organization (WHO) publishes the results of research involving human subjects only if fully compliant with ethical principles, including the provisions of the World Medical Association Declaration of Helsinki  may be influenced by a secondary interest . We ask all authors to disclose at the time of submission any competing interests that they may have. Examples of competing interests may be found at: 1.4Authors should identify the sources that funded the work undertaken, affirm not having entered into an agreement with the funder that may have limited their ability to complete the research as planned, and indicate that they have had full control of all primary data.1.5http://www.who.int/bulletin/volumes/83/9/645.pdf.Authors of rejected papers can appeal against the decision by following the procedures outlined in 2.2.1Bulletin via our submissions website (http://submit.bwho.org), where full instructions are given. Queries about online submissions should be sent to: bulletin.submit.ask@who.int. Authors requiring assistance with online submission can contact the editorial office.Manuscripts should be submitted to the 2.2ICMJE recommendations for the conduct, reporting, editing and publication of scholarly work in medical journals. The complete document is available at: http://www.icmje.org/recommendations/. Manuscripts should be prepared in accordance with the 2.3Bulletin; the abstracts are translated into Arabic, Chinese, French, Russian and Spanish.Manuscripts should be submitted in English and will be published in that language in the 2.4ICMJE recommendations  must be rigorously observed. Each author should have participated sufficiently in the work being reported to take public responsibility for the paper’s content and should describe in detail on the online submission system (not within the manuscript itself) his or her particular contribution. The Bulletin encourages submissions from authors in low- and middle-income countries, and in line with this policy at least one author should have a professional affiliation in the country where the study was conducted.On the manuscript’s title page authors should give their full names and the name, city and country of their institutions. The corresponding author must also provide a full postal address, which will be published with the email address unless otherwise requested. Academic titles and the names of departments and subdepartments are unnecessary and are discouraged for reasons of space. If an author has several affiliations, only the most important one should be provided. The criteria for authorship described in the 2.5Bulletin is a fully open-access journal and charges no author fees. Authors are responsible for obtaining permission to reproduce in their articles any material enjoying copyright protection. They should send the letter granting such permission to the editorial office when they submit their papers.The On submission, the corresponding author must indicate agreement on behalf of all authors that their submission, if accepted, will be published under the intergovernmental organizations’ creative commons attribution licence (CCBY 3.0 IGO). 2.6These should be used only to enhance the understanding of the text, not to repeat what can be clearly communicated within the text. All figures, tables and boxes should be numbered consecutively .2.7Abstracts should highlight the text’s most important points and should be provided for the following types of papers: Research, Systematic reviews, Policy & practice, base papers for Round tables and Lessons from the field. The abstract should not exceed 250 words. It appears in English at the beginning of the paper and in Arabic, Chinese, French, Russian and Spanish between the end of the text and the reference list. Structured abstracts are required for Research papers and Systematic reviews  and for Lessons from the field papers . 2.8ICMJE recommendations (http://www.icmje.org/icmje-recommendations.pdf). The accuracy of all references is the authors’ responsibility and authors are also responsible for dating access to URLs, providing a record of when they were active.Reference citations should be numbered consecutively as they occur in the text and references should be listed in accordance with the 2.9https://www.un.org/Depts/Cartographic/english/htmain.htm following these standard operating procedures: https://gamapserver.who.int/gho/gis/training/DMF_GIS2010_2_SOPSforWHOMaps.pdf. A vectorial EPS (Encapsulated PostScript) file must be submitted.Papers should contain no maps unless an important finding cannot be conveyed without them or unless they are needed to make an essential point. Maps that show international borders, partially or in full, must be created from the following source, approved by the United Nations: