To gather guidance from expert reviewers, to aid authors striving to publish and reviewers aiming to perform their task effectively. This study surveyed the Journal of Graduate Medical Education (JGME) ‘Top Reviewers’ from 2017, 2018, and 2019. ‘Top Reviewers’ perform four or more reviews per year, with high average ratings. Top reviewers were sent an 11-item survey in February 2020. The survey included three demographic questions and eight open-ended, free-text questions about the concepts reviewers most often target in their reviews. We calculated descriptive statistics and performed a thematic analysis of open-ended responses. Of 62 eligible top reviewers, 44 (71%) responded to the survey. Only eight (18.2%) and seven (15.9%) respondents reported having ‘stock phrases’ or a reviewer template used for reviewer feedback to authors, respectively. The what , how , and why  were the resulting themes summarizing how reviewers categorized and responded to common problems. For ‘really good papers’ reviewers found the what acceptable and focused on how and why. For ‘really bad’ papers, reviewers focused on big picture feedback, such as the value of the study. Top reviewers from a single health professions education journal appear to have similar approaches to conducting reviews. While most do not use stock phrases or templates, they share similar strategies to differentiate ‘good’ vs. ‘bad’ papers through the what, why, and how of a manuscript.Experts have described ways to improve peer review quality. Perspectives  Peer review is integral to the dissemination of high-quality scholarship in health professions education (HPE). As informed members of a journal’s audience, peer reviewers share their methodological and content expertise to provide meaningful feedback to authors. Such feedback can help authors substantially, to clarify an article’s narrative, enhance the rigor of its methods, and broaden the scope of its conclusions . Furtherfor reviewers, with an aim to optimize the peer review process. Perspectives and resulting guidance from expert reviewers are largely absent in the HPE literature. The few reports that examine peer review from the reviewer’s perspective do so indirectly by analyzing reviewers’ comments and manuscript characteristics associated with rejection [Much has been written about ways to improve the quality of peer review, generally through highlighting the characteristics of high-quality reviews from the perspective of journal editors and experts in the field ,7–9. Theejection .Our objective was to obtain the perspectives of reviewers who consistently produce high-quality education journal reviews. We aim to gather useful guidance from expert reviewers, to aid authors striving to publish HPE scholarships and reviewers who are new to the peer review process.This study used a cross-sectional design to survey all ‘Top Reviewers’ from the Journal of Graduate Medical Education (JGME) during the 2017, 2018, and 2019 calendar years. We chose to study a purposeful sample of JGME reviewers given their deliberate approach to selecting top reviewers based on the quality of their reviews and our access to the study population . The topWe iteratively developed an original, 11-item survey instrument using Messick’s unified theory of validity . We collAcademic Medicine, and a PhD in measurement, assessment, and evaluation with a postdoctoral fellowship in medical education and qualitative research methods. We incorporated feedback from both individuals into the final version of the survey instrument, which we determined through consensus of all authors (see Appendix).Next, we collected validity evidence based on cognitive processes. To this end, we performed ‘think aloud’ interviews, which allowed us collect evidence of response process validity and to improve item clarity . We condNone of the authors were participants in the survey. We collected and managed the study data using REDCap, an electronic data capture tool hosted at the University of Washington. REDCap  is a secure, web-based software platform designed to support data capture for research studies.(M) and standard deviation (SD) of items as appropriate. We numbered all participants by the order in which they completed the survey. We conducted a thematic analysis of the open-ended responses using Microsoft Word 2019  and analytical memos [We conducted our data analysis using Microsoft Excel 2018 , to calculate descriptive statistics: percentage of respondents, mean al memos . As JGMEThe Human Subjects Division at the University of Washington deemed the study to be exempt from review.Of 62 eligible JGME top reviewers, 44 (71%) responded to the survey. Respondents reported that they had been a reviewer for HPE journals for, on average, 10.4 years (SD = 8.5 years). Thirty-nine (88.6%) respondents reported that they worked at an academic institution at the time of the survey distribution and the remaining five (11.4%) had done so previously. Because reviewers’ lived experiences may influence their responses, we provide additional demographic data in Only eight (18.2%) respondents reported having specific ‘stock phrases’ and/or paragraphs they used repeatedly in their reviewer feedback to authors, while 36 (81.8%) did not. Similarly, only seven (15.9%) respondents said they used a reviewer template when working through their peer review process, while 37 (84.1%) did not.what, the how, and the why of the problems identified  Occasionally, reviewers commented on specific details of the methods, in ways that they would not typically do if a paper was ‘really bad.’ For the how, reviewers often gave feedback on the writing style, the organization of the paper, and the overall story arc. For the why they tended to give the most feedback on how authors could better articulate a study’s generalizability or transferability and better explain its impact on the field.When reviewers perceived a paper as ‘really good,’ then, typically, they had already bought into the what (research question and methodology) or why (relevance and impact), reviewers recognized that the what and why are difficult to assess if there are major issues with how a study is presented. In other words, if a paper is disorganized or otherwise not clearly written, it limited reviewers’ interpretations of what was done and the study implications. In general, when reviewers found what they perceived to be fatal flaws, reviewers focused on being very direct and kept comments short. However, comments from our respondents indicated that most of the JGME manuscripts reviewed had fixable issues rather than fundamental flaws.When expert reviewers perceived that a paper was ‘really bad,’ they tended to focus on big picture feedback, such as the value and impact of the study, rather than the details. They often commented on the paper’s overall strengths and weaknesses first, and then followed with feedback on some of the minor details. They often took an empathetic and generous stance such as ‘I assume that my best friend is about to get this review’ (9) and placed positive feedback first. They also prioritized certain parts of the review when they had many concerns about a paper, by highlighting areas that were problematic. While this feedback often focused on the what (research question and methodology), how (organization and presentation) and why (relevance and impact), and the extent to which reviewers address each of these areas depends upon the overall perceived quality of a paper. Reviewers often limit the number of comments for papers considered ‘really bad,’ which may reflect an emphasis on efficiency, yet also describe taking an empathetic stance.Our small study reveals that many of JGME’s top reviewers have similar approaches to conducting reviews although few use stock phrases or a template. Focus areas of feedback can be grouped under not get published [Story Not Study: 30 Brief Lessons to Inspire Health Researchers as Writers. Their central thesis is that ‘Great research papers, like great stories, are compelling, memorable, and persuasive. They grab and hold readers’ attention, increasing the odds that the research findings will reach and influence their intended audience’ (p. 1) [There are many useful resources to guide authors in preparing a manuscript for publication. Aside from instructions for authors provided by individual journals, helpful resources include tips from editors, the ‘writers’ craft’ series by Dr. Lorelei Lingard, and Dr. Geoff Norman’s ‘12 tips’ paper on how to ensure your paper does ublished . There aublished . The pre’ (p. 1) .The top reviewers in our study reported that most submitted manuscripts had fixable issues and would be publishable. We therefore encourage authors to review their work carefully prior to submission, considering first and foremost the clarity of writing. Having a colleague outside the research team read the manuscript, with the how, what, and why framework in mind, can be useful to anticipate reviewer feedback and optimize chances for acceptance.Many journals have clear guidelines for reviewers, which indicate the areas for preferred attention ,24. Yet,what) are of sufficient quality. If not, the reviewer can decide whether to invest additional time in providing detailed feedback regarding the writing and organization (how) – the story – and the overall relevance and importance of the work (why). This strategy is likely to help the reviewer work more efficiently. The different categories in each of the themes (The results of our study provide insights for new reviewers as well as concrete suggestions for a structured approach to the organization and prioritization of reviewer comments. After reading a manuscript, a reviewer can focus first on whether the overall research question and methods (e themes  can servThe study findings are limited by our use of a short survey and a single journal. This makes generalizing or transferring our findings to other journals or settings difficult. In addition, we limited our survey to top reviewers, defined by one journal, who comprise a small fraction of all reviewers. This skews the results towards this small group, and while they may be consequential by the nature of their standing as top reviewers, they are not necessarily representative of expert reviewers for other HPE journals. Furthermore, while we intentionally refined our survey with specific steps to establish validity evidence and minimize undue bias, the initial survey was developed by authors’ consensus, which potentially limits the scope of the questions. We did, however, include several open-ended items, which allow respondents to offer new content. Because of our purposeful sampling strategy, this led to a rich data set with in-depth information about respondents’ approaches to the review process.. Finally, despite attention to rigor in our approach, the findings are limited by the quality of the qualitative data that can be gathered from open-ended survey questions, in which follow-up questions, to improve clarity and depth, are not possible .Future study of processes used by reviewers who consistently produce high-quality reviews, as defined by authors as well as editors, may provide additional insights for those who wish to improve the quality and efficiency of their reviews. Further work could also explore the impact of high-quality reviews on authors. While one hopes that high-quality reviews will help authors improve their work for resubmission as well as future manuscripts, to our knowledge there is no empirical evidence to support whether this is true and, if so, which review elements are most useful to authors.what, why, and how of a manuscript. For ‘really good’ papers, top reviewers focus on improving the paper in areas of how and why: writing, organization, relevance, and impact. In contrast, for ‘bad’ papers, which have problems with the research question or methods, reviewers typically provide brief comments and may use an empathetic tone.Top reviewers for a single, health professions education journal have similar approaches to conducting reviews. While they do not tend to use stock phrases or reviewer templates, they do share a similar focus on the


Nucleic Acids Research, the editorial team realizes that rapid growth of potentially relevant papers increases the challenge of preparing review articles that will appeal to our broad readership.Articles that summarize knowledge are important contributions to science. Such articles should provide a single authoritative source of information relative to a particular scientific field or question. At https://academic.oup.com/nar/pages/criteria_scope) to make our expectations clear. First and foremost, we expect that authors demonstrate a passion for explaining their topic. Authors should see an important unmet need, a topic where researchers lack up-to-date syntheses of the best information on a subject. Authors should also have deep expertise. The scientific literature is vast. Not all papers are equal in quality. Authors should be able to identify the best, most rigorous studies. They should provide guidance to readers on the hallmarks of rigorous (or less than rigorous) research. Manuscripts should provide some guidance about challenges for future research.Recognizing these challenges, we have recently revised our guidelines (We welcome expert opinion, and we hope that authors will enjoy sharing their hard-earned authority to judge with the community. We note, however, that we continue to welcome more traditional review articles that summarize key findings in a field. Implicit in such papers, however, is the fact that authors possess the expertise to identify the most reliable data and set it into context. We are looking for papers that weave a field together, not present it as a series of unconnected findings. Critical Review and Perspective articles can be commissioned by the editorial team, but we also welcome suggestions from authors. We are happy to provide feedback to authors if we feel that a potential submission is not ready but might have the potential to be improved.NAR review, because rather than focus on a particular scientific question, it focuses on the scientific process itself. In 1989, Dr Stanley Crooke left his position as head of Research and Development at Smith Kline Beckman to start Ionis Pharmaceuticals, a company dedicated to a daring new concept—turning synthetic oligonucleotides into drugs. There were many scientific challenges. How can a large negatively charged synthetic molecule enter cells? How can it be made economically on large scale? How can it be chemically modified for biological activity? In 1989, the basic science underlying the field was primitive—the first automated small scale synthesizers had begun to enter laboratories just a few years previously. Nothing was known about the pharmacological properties of oligonucleotides. With so little scientific background, identification of appropriate disease targets was speculative.In this issue, we present the first commissioned Critical Review and Perspective. This Perspective is not a typical Dr Crooke dedicated over thirty years to tackling these problems at Ionis. The teams he has helped supervise have created several approved drugs. One drug, Spinraza, has had a major impact on the treatment of spinal muscular atrophy, providing hope to patients and their families and demonstrating that nucleic acids drugs can have a major impact. While pursuing drug development, Dr Crooke has led his own research group and encouraged Ionis employees in general to publish their work and support basic science through collaborations.Nucleic Acids Research. The question of rigor is one that NAR editors and reviewers face when evaluating thousands of submissions each year. We hope that Dr Crooke's perspective will be a useful guide for those seeking to understand how to balance rigorous basic science with the practical demands of drug development. We expect that this Perspective will not only inspire scientists in industry, but also in academia.The question of how to address difficult scientific problems is one that is often faced by the authors submitting work to


As I take over the Editor‐in‐Chief position from Dr. Jack Kessler, I want to thank him for his vision for ACTN and 10 years of leadership in setting up the journal and growing it. I have been involved in ACTN from the beginning when Jack asked me to join as an Associate Editor. It has been a privilege to work together to start something new and see it grow. In the beginning, as an open‐access journal, success was not assured, but with Jack's leadership and support from the American Neurological Association and Annals of Neurology, ACTN is now recognized as a reputable and growing journal that publishes impactful research focusing on translational neurology.I am honored to take the baton from Jack and want to reiterate my commitment to publishing excellent clinical and translational research in neurological sciences. I want to see the ACTN be the home of clinical and translational neurological research that is impactful and timely. With rapid review and publication, ACTN can disseminate new knowledge in a timely manner and affect clinical practice.As the scientific publication landscape changes, we are seeing a big diversification of the manuscripts submitted to ACTN. The new editorial board and associate editors reflect this diversity. I would like to thank the previous associate editors and editorial board members and welcome the new associate editors, Drs. Jing Ji, Keith Josephs, Samia Khoury, Jun Li, Elisabeth Marsh, Ludy Shih, and Saurabh Sinha. Together, we will strive to have the manuscripts submitted to ACTN reviewed in a timely manner.InterACTN.org), an interactive case study based on interesting and challenging patients. These online cases and the interactive format proved to be popular with trainees. I would like to thank the current InterACTN editor, Dr. Danny Bega, and welcome the two new coeditors for the InterACTN, Drs. Leana Doherty and Rebecca Tran Pollard. Over the next year, we plan to expand the number of case studies within InterACTN and invite submissions from residents and fellows.During his tenure, Jack Kessler had started InterACTN (We will be creating a new section to solicit non‐research articles that highlight the humanistic side of being a neurologist. This section will publish short articles and poems about physicians' experiences as doctors. I believe neurology is rich in very rewarding physician–patient experiences and the addition of such a short section would help create a camaraderie among the neurologists. In this section, we also plan to include articles focusing on diversity and social justice and I invite such submissions.As I close this brief editorial, I again want to thank Jack Kessler and ANA for setting up the journal and invite my colleagues to submit their timely and impactful research and reviews to ACTN.


Communications Chemistry turns five, the Editors reflect on the journal’s values, achievements, and ambitions for the years to come.As  Communications Chemistry to provide a home for research of significant interest to focused communities within the chemical sciences, whilst delivering the highest standards of editorial practices and publication ethics to ensure quality in content as well as author experience. To put these goals to action, Communications Chemistry retains the traditional Nature model of in-house professional editors—PhD-level scientists with research experience who are comprehensively trained in all aspects of manuscript handling—and compliments it with an Editorial Board of active researchers recognized as experts in their field.When launching a new journal, it’s imperative to think about what values it will embody, and what role it will serve in the community. Five years ago, our vision was for Communications Chemistry has become. We have published over 800 open access articles, covering the entire breadth of chemistry, while also touching on adjacent areas such as materials science, nanotechnology and chemical engineering. Every publication has been carefully selected for inclusion within the journal, and we are delighted to see the impact that many of these are having on ongoing research in the chemical sciences. Indeed, it is exciting to see that our content is primarily being cited in highly visible and reputable outlets , and education and public outreach. Our our outstanding referees scheme, for going above and beyond what is expected of a reviewer. We also wish to extend an enormous thanks to our Editorial Board Members, and of course to our readers, for supporting the journal from day one.Our journal would be  nothing without our authors and reviewers. We extend a huge thank you to all those who have published with and reviewed for us over the years, in particular those authors who have published with us on numerous occasions, and those reviewers highlighted through five year anniversary Collection, featuring a selection of some of the most highly downloaded, cited and talked about content from the last five years. We hope you enjoy reading these papers as much as we enjoyed publishing them.Today, we are pleased to launch our Women in Chemistry Collection, or our Early Career Researcher travel grants, and would like to invite researchers to put themselves forward to be profiled on their research and lived experiences in a series of Q&A articles that we intend to publish in the journal.While we are proud of how far we have come, we still have plenty that we hope to achieve. We hope to build on our previous and existing DEI-oriented initiatives, such as our 2 reductionElectrocatalytic CO and Organomediated polymerization come to fruition, and we hope to do more in the coming years to support and surface important contributions that chemical research has to offer to achieve these vital goals.The United Nations’ Sustainable Development Goals are also front and center in our minds. We’re looking forward to seeing our recently launched Guest Edited Collections on custom computer code, NMR spectra, and electrocatalytic nitrogen fixation to ensure that we support reproducible research and open data in line with community gold standards.We also wish to further expand our data and reproducibility policies to strengthen our quality pledge and reflect the needs of the chemistry community. We have already implemented standards for As we continue to grow, we are committed to maintaining our competitive turnaround times, strong editorial guidance, and the excellent quality of our output. We greatly look forward to continuing to work with you all, and to hopefully welcoming many new faces into our (web)pages in the years to come!


Self-Help for Learned Journals. It reveals the postwardecades as the time when scientific research became something to be commodifiedand sold to libraries, rather than circulated as part of a scholarly mission. Itwill be essential reading for all those campaigning to transition academicpublishing – including learned society publishing – away from the sales-basedmodel once again.In the decades after the Second World War, learned society publishers struggledto cope with the expanding output of scientific research and the increasedinvolvement of commercial publishers in the business of publishing researchjournals. Could learned society journals survive economically in the postwarworld, against this competition? Or was the emergence of a sales-basedcommercial model of publishing – in contrast to the traditional model ofsubsidized journal publishing – an opportunity to transform the often-fragilefinances of learned societies? But there was also an existential threat: ifcommercial firms could successfully publish scientific journals, were learnedsociety publishers no longer needed? This paper investigates how British learnedsociety publishers adjusted to the new economic realities of the postwar world,through an investigation of the activities organized by the Royal Society ofLondon and the Nuffield Foundation, culminating in the 1963 report Yet, just over a century later, the investment bank Morgan Stanley coulddescribe scientific and medical journal publishing as an industry worth US$7billion, offering good returns to investors. During the twentieth century, in other words, the publishing of researchjournals had become a lucrative business.In 1895, the secretary to the Royal Society explained to the UK government that thepublication of scientific research journals simply could not be undertaken “on anordinary commercial basis.” This being so, “the burden” of publishing researchnecessarily fell on the learned scientific societies and their mission forsupporting scholarship. This paper, however, examines the shifts in thought and in practice thatoccurred at learned society publishers in the 1950s and 1960s.This transformation had two underlying components: a significantly increasedinvolvement of commercial firms in journal publishing after 1945, and changingattitudes toward the economics of journal publishing among the learned societies.The dominance of modern research journal publishing by international mediaconglomerates has motivated many of the current campaigns for the reform of academic publishing.circulating research toselling it, and from decades of publication deficits to a newworld of breaking even. Profits from publishing would be a later development , but this was when scientific research became something to becommodified and sold to libraries, rather than circulated as part of a scholarlymission. Understanding the reasons for the 1950s shift from a philanthropic model ofcirculation to a commercial model of distribution based on sales is essentialgrounding for all those campaigning to transition academic publishing – includinglearned society publishing – away from the sales-based model once again. It also offers us a different perspective on the links between science andcommerce during the late twentieth century.The postwar decades were a key period of transition for learned society publishers:this was when they shifted from  More recently, historians have investigated how authorship in journals cameto be such a dominant element in the prestige economy of scholarship, providing aroute to reputation and potential professional advancement. The learned societies and academies of the eighteenth and nineteenthcenturies are an important part of that story due to the tight connection theycreated between research publishing and communities of scholars. But we knowremarkably little about how academic research journals developed from tools forcirculating knowledge within communities of gentlemen-scholars into businessenterprises generating substantial commercial rewards for their owners and shareholders. The rise of such firms as Pergamon Press, North Holland Publishing, andElsevier  in the mid to late twentieth century isfamiliar to industry practitioners, scholarly communications campaigners, andscholars of scientometrics, but it has yet to receive adequate attention as ahistorical phenomenon depending on, and significantly affecting, scientific researchand the academic profession.Scientific journals have traditionally featured in the history of science astechnologies that enable the construction and circulation of knowledge claims, andas sources of information about those claims, their authors, and their reception. Governments, philanthropic foundations, and industrial enterprises becamemore involved in funding research, and their willingness to fund thepublication of research forms the backdrop to thecommercialization of scientific journal publishing. More generally, academicscientists were also affected by changes in the funding and organization of highereducation. For those based in the UK, that included adjusting from a Britishscholarly world, focused on the empire and Commonwealth, to one in which Europe and,especially, the United States were more prominent.The commercialization of scientific research publishing is intertwined with widertransformations of the scientific enterprise in the twentieth century: the increasedscale of scientific research; increased funding, particularly in the early decadesof the Cold War; international conferences and research collaborations; and theemergence of new and more specialized disciplines. This offered a sharp contrast to learned society publishing, which was rarelydescribed as “speedy” . Although British learned societies certainly did think about disseminationbeyond the UK, they typically focused on the needs of readers in the extendedBritish academic world, with some recognition of traditional centers of scholarshipin Western Europe. They had relatively little engagement with authors or readers in the UnitedStates, South America, Africa, East Asia, or the Soviet bloc.One contemporary who was well aware of the business potential of internationalscience was the publisher Robert Maxwell, the founder of Pergamon Press. Interviewedin 1964, he explained that “I believe that scientific research is an internationalthing not confined by national boundaries.” His approach to the publication ofscientific research was therefore to be “speedy production and world-wide dissemination.” Pergamon’s activities were partly responsible for the headline inNature in 1960 that asked plaintively, “How many more new journals?”In contrast, Pergamon Press drew upon the links Maxwell had developed with Germanpublishers during his military service in postwar Berlin, and was internationallyoriented from its foundation in 1951. Pergamon translated the research of Sovietscientists, published the proceedings of international conferences, and keptresearchers on both sides of the Atlantic up to speed with each other’s work. Italso created dozens of new journals to cater to emerging disciplines and specializations. Without mentioning names, concerns were voiced about the “less-scrupulousminority [of publishing firms] that act as entrepreneurs,” and whostart new journals “in order to ‘sell’ science as a commodity.” There is no doubt that many scientists appreciated Maxwell’s support forthemselves and their disciplines, but others believed he was “the greatest villain unhung.”The rise of entrepreneurial, international scientific research publishing was notuniversally welcomed. A Royal Society committee noted in 1963 that “the presenttendency for commercial publishers to initiate new scientific journals in greatnumbers is causing concern to many people.” The Royal Society’s concern was not, of course, disinterested: itslong-established and well-respected role as a publisher of research journals wasthreatened by the ease with which commercial publishers were establishinginfluential new journals.The Royal Society was prominent among those who feared that entrepreneurialpublishers posed a double risk to “the welfare of the scientific community.” In 1957its executive secretary, David Christie Martin, explained that it was essential thatscientific societies, rather than commercial firms, should “continue to predominatein scientific journal publication” because societies were committed, by theirmissions, to circulate research “as widely and cheaply as possible.” Furthermore,the expertise of their membership enabled them to ensure high intellectual standardsand to be “the guardians of the quality of scientific publication of original workin learned journals.” This was not a uniquely British problem: in the United States, for instance,journals were facing such “critical financial problems” that it was a focus for thenewly established National Science Foundation; its report led to changes thatenabled U.S. government agencies to support the publication of research through thepayment of page charges. In the UK, in contrast, there had been governmental support of learnedsociety journals since the 1890s, but in austerity Britain of the early 1950s, theextent (or availability) of such funding seemed uncertain. It was in this contextthat the Royal Society urged learned societies to become self-reliant.This existential threat arrived at a time when UK learned society publishers werealready struggling with their financial sustainability. The situation had become sodire in the mid 1950s that contemporaries feared that, without some kind of help,learned society journals “were on the way to extinction.” The project began with hopes of streamlining editorial and productionprocesses to save money where possible, but it ended by urging the societies tofocus on sales and marketing, especially internationally . Even though learned societies saw commercial publishers as a threatto the publication of research, the Self-Help project helped them learn how to thinkmore commercially themselves.This paper explores the perceived threats to the survival of learned societypublishers in the 1950s, and the strategies proposed to help them both adapt to thenew world of postwar scientific publishing and free their journals from the prewarreliance on government support or charitable donations. It does so by examining aproject intended to provide “Self-Help for Learned Journals” that ran from 1955 to1963. This project was funded and supported by the Nuffield Foundation, but led bysenior figures from the Royal Society, and it was intended to offer advice andfunding to struggling learned society publishers. Those “old terms” had included a focus on wide circulation  and uneconomic pricing. With so few copies being sold even at cost price, it is hardly surprisingthat learned society treasurers of the nineteenth century had been used to seeingjournal publications as a drain on resources rather than a potential incomestream.In the 1950s, learned society journals found themselves under such severe financialstrains that it seemed “impossible” to continue circulating scientific research “onthe old terms.” This government support, plus some bequests and donations, and somerationalizations to the free or subsidized circulation, propped up the philanthropicmodel of learned society publishing in Britain until the Second World War.Learned societies’ ability to fund the publication of ever-increasing amounts ofknowledge had, in fact, been under strain for decades. This was why the UKgovernment had agreed, in 1895, to provide an annual grant-in-aid of scientificpublications, administered by the Royal Society and disbursed as grants to learnedsociety publishers. Even once the worst had passed, learned society publishers faced twospecifically postwar problems. First, they could no longer rely on the “generousgood-will” they had been accustomed to receive from printers, who faced their own challenges. In 1953, for instance, Cambridge University Press renegotiated terms with alllearned societies for which it printed, including the Royal Society. Second, the cost of subsidizing the production and circulation of scientificresearch journals was constantly being pushed higher by the “sky-rocketing volume oforiginal scientific publication.”In the immediate aftermath of the war, all members of the British print tradesstruggled to deal with the problems caused by wartime destruction and postwarshortages of materials and manpower. The funding and clerical support would come from the Nuffield Foundation,while the Royal Society supplied expert advice. The Nuffield Foundation had beenestablished by car manufacturer William Morris in 1943, with £10 million in sharesand a remit to improve “social” or “industrial” medicine and scientific research inpostwar Britain. Its desire to support capacity-building in science led to severalcollaborations with the Royal Society in the mid 1950s, most notably theCommonwealth scholarships. The stimulus for the learned journals project appears to have come from theRoyal Society, but a parallel project was also established in collaboration with theBritish Academy.It was in spring 1955 that the Nuffield Foundation and the Royal Society agreed toundertake a five-year project “for the support of learned journals.” In 1955, its approach to the Nuffield Foundation continued this leadershiprole, but also reflected the society’s own recent experience of transforming itsjournals from deficit to break-even and its doubts about the willingness of the UKgovernment to continue supporting learned society journals.As the oldest scientific society in Britain, the Royal Society liked to positionitself as a leader among the newer, more specialized societies. It had led theappeal to government in 1895, and in 1948 it hosted international delegates at aScientific Information Conference.PhilosophicalTransactions and Proceedings had risen from 3,700pages to 5,460 pages a year, while production costs had risen even faster, fromaround £16,000 a year to over £35,000 a year. This forced the society to draw more heavily than usual upon its charitablereserves and upon the parliamentary grant-in-aid; otherwise, the society would haveshown a publications deficit of £5,000 for the year ending November 1953. As the society’s officers debated the possibilities for cutting costs, aletter arrived from Cambridge University Press announcing that it had “reluctantlydecided” that it could no longer continue “subsidising a number of learned Societies.” For the society’s officers, the possibility of having to pay a highercommission on the sales managed by the press put “a completely new complexion” onthe situation.The Royal Society had been forced to take a hard look at its own publishing practicesin 1953. Between 1949 and 1952, the printed output of its  This experience suggested to the Royal Society that learned societypublishers could act to transform their precarious financial positions.In less than a year, David Martin oversaw the creation of the Royal Society’sin-house publishing sales team, which took over sales, marketing, and subscriptionmanagement from Cambridge University Press (CUP) in October 1954.  Theresult was that sales, rather than philanthropy, came to be the main method ofcirculation for the society’s journals, and has remained so ever since. Severe cutsto the distribution of free copies meant that universities and research institutionsthat wanted to continue to receive the society’s journals henceforth had to purchasethem. Prices were raised, and the sales team began to target new markets,particularly in North America. In 1953, Cambridge University Press had generated£27,500 of income for the society; by 1955, the society’s team reported a salesincome of £58,500. But by the mid 1950s the political mood was changing, with chancellors fromboth dominant political parties implementing austerity measures. The grant-in-aidfor 1957 would be reduced to £20,000, and it was against this context that DavidMartin advised the Chemical Society that it would be unwise to rely on governmentsupport for scientific publications.  Martin further hinted that “indebtedness” togovernment might even “endanger the independence” of the society and prevent it frombeing seen as “a strong force in the scientific community.” Martin’s success in transforming the Royal Society’s publishing finances madehim unsympathetic to societies who continued to act as though “living in a fool’s paradise.” It was to help other societies become self-supporting that the Royal Societyhad sought the assistance of the Nuffield Foundation in 1955.The further concern was uncertainty about the future of the parliamentarygrant-in-aid for scientific publications. In the immediate postwar years, the UKgovernment had been willing to accept that learned societies needed more help thanever with their journals and had increased the annual grant-in-aid from £7,000 in1945 to £26,000 in 1954.The first meeting of the Nuffield Foundation’s “scientific advisory committee” washeld on April 15, 1955. Its remit was to advise on the disbursement of £20,000 overfive years, to assist learned journals in the sciences. The committee was dominatedby the Royal Society: it was chaired by Edward Salisbury, the director of KewGardens and one of the secretaries to the society, and its original members includedfour other fellows of the society, and David Martin. The other committee memberswere Leslie Farrer-Brown, a medical administrator who had directed the NuffieldFoundation since its origin, and Jack Morpurgo, director of the National BookLeague, an organization founded in the 1920s to promote reading and an appreciationof books. This emphasis on self-help was shared with the Royal Society committee thatadministered the parliamentary grant-in-aid, which, from 1957, required applicantsto demonstrate that they were attempting “to stand on their own financial feet.”At the first meeting of the scientific advisory committee, Farrer-Brown explainedthat the Nuffield Foundation intended its money to support what he termed “‘primary’journals in ‘primary’ subjects,” a phrasing that the committee subsequentlyinterpreted as journals publishing original research, with some bias toward themathematical and physical sciences. The Nuffield Foundation appears to have sharedDavid Martin’s conviction that societies should learn to help themselves, and laiddown a guiding principle that support would be contingent on a journal showing itwas “doing all within its power to make its economic position stable, or else isprepared to accept and act upon the advice offered by the panel of advisers.”Farrer-Brown explained that this condition was made to “encourage” societies “to putinto practice possible economies in the production and distribution of theirjournals that may eventually lead to ensuring their solvency.”Unlike the parliamentary grant committee, which had to distribute all its funds tosocieties in need, the Nuffield committee had no such restriction. As  The Nuffield committee consulted experienced publishers, including Allen Lane(of Penguin), Robert Lusty (of Michael Joseph Ltd), and Basil Blackwell (ofBlackwell Scientific Publishing). Robert Lusty was then asked to pursue some informal enquiries, guided by alist brought to the first meeting by David Martin that classified twenty-fivelearned society publishers “from a financial point of view” . In mid 1956, Hutt began discussions on behalf of the Nuffield committee withthe Royal Astronomical Society, the Royal Anthropological Institute, the RoyalMeteorological Society  had recently retired from a long transatlantic publishing career thathad included a directorship at Faber & Faber (with T. S. Eliot) in the late1920s, and a senior role at Harcourt Brace in New York during the war years. He alsohappened to be a former Rhodes scholar with a D.Phil. in mathematics from Oxford.Nature in September 1957 carefully introducedFrank Morley to the wider scientific community as someone who would be sympatheticto learned societies and scholars. As well as his own credentials, Morley’s familyconnections proved useful: he had two brothers in literary and academic roles in theUnited States, and a father who was a mathematics professor and journal editor.According to Nature, this meant that Morley had been “familiar withand interested in the problems attendant on the production and distribution” of hisfather’s journal “from an early age.” He was presented to academics and editors oflearned societies as someone whose “wide and varied experience” might “prove ofpractical value,” but, equally importantly, as someone who understood their world.A short article in  Morley began with the societies on Martin’s 1955 list, but later reached outto some of the larger provincial societies and to smaller societies in morespecialized fields. As the committee’s work became better known, it also began toreceive unsolicited applications. Few of these were deemed eligible for funding, butthe committee was willing to offer Morley’s advice to anyone who sought it,including the editor of Business History.Overall, Morley engaged with at least twenty-eight societies, fifteen of whichreceived some form of financial assistance.  Morley had to reiterate that he and the committee would only “considerfinancial matters, such as (1) price of journal; (2) need for new subscribers.” Even so, Salisbury would later note that there had been resistance from“those who had, under other economic conditions in the past, given ungrudging andsuccessful service as editors and members of editorial boards.” The implication is that many scientists did not (yet) fully appreciate thechanged economic realities of the postwar publishing landscape and were hoping thatjournal publishing could, somehow, keep going as before.The interest from smaller societies contrasted with Morley’s early difficultiespersuading the larger societies to participate. There was some reticence aboutsharing financial details with outsiders, as well as a fear of forcible“interference with editorial policy or any lowering of standards.”Quarterly Journal of Experimental Physiology, whose editorapplied for assistance in 1957. Although the journal was “published by Livingstonesin Edinburgh, who act as distributors,” its ownership was vested in three (academic) trustees. The committee encouraged the editor to talk to Morley, but when hesubsequently applied for a grant, he was informed that “there was really no goodcase.” The reasons were twofold: the journal’s publisher had agreed to contributetoward the cost of the proposed publicity campaign, but most crucially , the journal was now making a small profit. The Nuffield committee was apparently willing to take a generous definitionof a “learned journal,” but its financial support was limited to those (still) “indifficulties.”As a general rule the Nuffield committee helped learned societies, but one exceptionwas the  Later, it investigated the “exact contractual arrangements” between theLondon Mathematical Society and its printer. The variety of arrangements it discovered would be demonstrated by the threepages of examples included in Self-Help. For societies that paid for publishing (as well as printing) services,sales-based commissions made it tricky for the Nuffield committee to help thesociety with sales and marketing without also helping its publisher; in such cases,the committee expected the publisher to contribute to the costs of any publicitycampaign from which it would also benefit. The ambivalence about working with profit-seeking partners can be seen mostclearly in the Royal Meteorological Society’s proposed reprinting project: itsprinter was willing to underwrite the project, but the society preferred to take aloan from the Nuffield committee so it could retain its ownership rights and “remain independent.”As the project wore on, Morley and the committee became increasingly conscious of thevarious relationships between learned societies and the firms who provided servicesto them. For instance, in 1957 a grant to the Geological Society was made oncondition that the publicity material clarified that the printer “was acting solelyas the printer, on behalf of the Geological Society, and would derive no financialgain from the resulting sales.”Journal of Animal Ecology was informed that index-making couldonly be supported if “there was any indication that the Journal would improve itsfinances” as a result. As the next section will show, the committee’s support was largely directedtoward helping learned societies run their journals on more commercial lines.The Nuffield committee entirely declined to support ideas that had clear scholarlyvalue but little apparent chance of commercial success. For instance, theSelf-Help with a discussion of cost-savingopportunities, but it was a notably brief section. As Lusty had earlier observed, itwas difficult to find simple changes that would dramatically improve a society’spublishing finances. Morley pointed out that a change of page size might reducepostal charges, or make it easier to accept adverts, but he was wary aboutsuggesting changes that might be seen as lowering a society’s traditional productionvalues. His most concrete recommendation was for those societies whose journalsmixed original research articles with such content as the annual accounts, reportsof meetings, and members’ news. Removing such “parochial” material would save paperand postage costs, and make the journal more attractive to overseas (and nonmember)subscribers. Thus, the British Glaciological Society created a members-only bulletinseparate from its research journal.Morley began Apart from that, Morley reminded academic editors and their assistants that theycould avoid many delays and expenses if they would take the time to learn how aprinting office really worked. Editors who understood that the printer has “tosatisfy other customers who are on time” would realize why punctuality in returningproofs mattered, and understanding how corrections were made to typeset text wouldreveal why authors “should be utterly forbidden” the “ancient luxury” of rewritingpapers at proof stage.Self-Help, however, and most of the discussionin the committee minutes, focused on ideas for revenue-generation. They reveal thesteps British learned societies took as they transitioned to a more commercialoutlook for their publishing activities.Most of the pages of The pricing strategies of learned societies had developed in the age ofindependent gentleman scholars, with the underlying assumption that researcherswould access journals by acquiring personal copies. Those who were members of aparticular society typically got the society’s journal free or discounted, and,even for nonmembers, the societies set prices low to enable wider circulation.For instance, in the early twentieth century the Royal Society had offereddiscounted rates on its journals to members of other scientific societies andkept its official nonmember price low for the benefit of younger scientists whowere not (yet) members. Lusty had been swift to label such pricing practices as“hopelessly uneconomic,” and Morley was scathing of society members who expected“to be supplied at prices fixed before the First World War.” In an age ofinflation, the simplest way to prevent societies “slipping into insolvency” wasto raise prices. This was why Pergamon Press adopted a subscription system thatdifferentiated between individual and institutional purchasers. Morley did not suggest societies should replace their traditionalmember/nonmember distinction, but he did argue that both rates must beeconomic.By the 1950s, very few nonmembers were buying personal copies. Most scientificresearchers now had access to institutional libraries affiliated to theiruniversity, industry, or military laboratories. Nonmember prices had been setlow to support early-career scientists, yet it was actually libraries that were“almost the only other purchasers” (other than society members).Self-Help, Morley used circulation figures from the(anonymized) Chemical Society to demonstrate the effects of a series of pricerises during the 1950s. In 1949, a subscription to its journal had cost £1.10s for members, and£5 for nonmembers. Ten years later, the rates were £8 and £20 respectively. Thedramatic increase in price for society members hit hard: over 75% of theUK-based members stopped taking personal copies of the journal. On the otherhand, a successful marketing campaign aimed at libraries increased the number ofnonmember subscribers by almost 50%. Overall, the total circulation decreased by900 copies, but the journal became financially self-supporting.In The story encapsulates the transition from the philanthropic to the commercialmode of scientific journal publishing: the journal was now funded by librarysubscribers rather than subsidized by grants; its readers relied on librarycopies rather than their own personal copies; and the market becameinternational rather than British. Morley argued that each society “must strikeits own balance” between how far “it can afford to cosset its own members” andhow far “to seek commercial return from non-members,” but he insisted that “theeconomic pressure is ruthless.” He saw this as “part of the whole social change”in postwar science, and the postwar world more generally.The Chemical Society’s new financial stability illustrated the importance of asuccessful marketing campaign, as well as higher prices. Almost all the grantsmade by the Nuffield committee were for the printing and mailing costs ofpromotional campaigns, and it was their design and implementation that occupiedmost of Frank Morley’s time in late 1957 and 1958. The biggest cost in anypublicity campaign at this time was the postal charges, so it was essential thatany material sent out was effective and that it was sent to the rightpeople.Morley worked with societies to design their promotional material, usuallyrecommending a two- or four-page leaflet that could be used to “present theright ‘image’” of a journal. For instance, he advised the Geological Society on“a more effective” layout and advised the Mineralogical Society on “a rathermore elaborate leaflet. . ., including perhaps a plate.” Morley insisted that itwas worth “spending a little to produce a better leaflet.” It was possible to hire an agency to send promotional material “to ‘agood list’ of 5,000 addresses or more” – but for specialized scientificjournals, this was unlikely to be money well spent. The Royal Astronomical Society had initially sent a leaflet promoting itsMonthly Notices “to a list supplied by the printer,” andalthough it told the Nuffield committee that the results had been “lessdisappointing than was feared at one stage,” they subsequently planned to sendthe leaflet to “a hand-picked list of 500 scientific and national libraries and institutions.” The Chemical Society, too, created a “hand-picked” list, though its focuswas “1,500 industrial addresses.” Morley explained that he encouraged society officers to use “their ownknowledge” to identify “individual heads of departments at American universitiesand colleges”; this was far better than using standard lists of Americanlibraries because “librarians are shy birds to shoot at.”When it came to circulating this publicity material, Morley repeatedly arguedthat “head-work” was more important than “unguided promotional efforts,” andthat learned societies had – in their own membership – excellent resources forselecting and scrutinizing a specialist mailing list. Most societies applying for Nuffield help were interested in the“considerable untapped market in America.” Shared language made the United States an obvious target for Britishpublishers, but before the war, both the book trades and the academic worlds ofthe United States and the UK had been only loosely connected. The creation ofthe Fulbright scholarships in 1946 and the opening of a U.S. branch office byCambridge University Press in 1949 were among the signs of change. The Royal Meteorological Society, the Palaeontological Society, and theSociety for Applied Bacteriology were among those who received fundingspecifically for publicity campaigns in the United States.This focus on overseas, and especially North American, markets was a change ofapproach for British learned societies, which had traditionally focused on theirBritish-based members and on scientific institutions in the British sphere ofinfluence. It reflected a widespread postwar awareness of the importance ofinternational scientific communication, within (and perhaps beyond) the NATO bloc. Thus, in 1960 Morley was able to improve the mailing list drafted by theYorkshire Geological Society by drawing on lists compiled by the GeologicalSociety and the Royal Society.The Nuffield committee also encouraged the sharing of resources. For instance,from September 1958 it began collecting specimen mailing lists from societiesseeking grants, and making them available for consultation by other (approved) societies. The Nuffield committee members were enthusiastic about this plan, butprogress was slow as the organizations involved bickered over who should takeresponsibility for compiling and – even more critically – publicly endorsing theselected, annotated lists of journals. It took more than two years before thefirst such list – for geological journals – was in print.Morley also coordinated a collaboration with the British Council. Its officesoverseas were to supply lists of British scientific journals to “scientists,science libraries and bookshops in many countries,” hopefully benefiting manysocieties by “extending sales and the use of these periodicals overseas.” Morley therefore tried to persuade the major geological journals to worktogether so that they could, as a group, enter an advertising contract withShell-Mex and BP. This “group advertising” would make the logistics easier forthe advertiser as well as offering access to a larger combined audience. AsMorley later put it, such “a really wide coverage of alert minds” ought to beattractive to the sort of scientific and technical firms that sought to recruit(or sell to) “the best and best-trained brains in the country or overseas.” He hoped that Shell-Mex and BP would be the first of many largeindustrial advertisers for British scientific journals.Many editors were interested in the potential of income from advertising, but itwould prove elusive. Early in the project, Frank Morley contacted the generaldirector of Shell-Mex and BP , and discovered that he was certainly interested inbringing his company to the attention of researchers in the geological andgeophysical sciences, if it could be done without having to deal “separatelywith individual journals.” There were some issues from the Shell-Mex end, but the real problem laywith the geological societies. A practical sticking point was that, for such ascheme to work effectively, journals would need to coordinate certain elementsof their production processes, such as page size and periodicity, but these weresignificant changes to make for an as-yet-uncertain (but certainly modest)financial benefit. Morley reported in 1959 that only the “smaller societies” hadshown much interest. He later described the challenges of getting societies to work together –and particularly, persuading the larger societies to work with smallerorganizations – as a difficult “psychological problem,” akin to persuading lions“to lie down with lambs.” The abortive group-advertising scheme demonstrates one of the bigchallenges for the Nuffield scientific advisory committee: individual societiescertainly wanted help, but were not keen to work together.By February 1958 the scheme had been “tentatively organised,” but it neverprogressed further. Journal back-runs were a case in point, as the successful publicitycampaigns attracted the attention of libraries that might be persuaded topurchase the recent back-run as well as current and future issues. In 1957, forinstance, the Company of Biologists “made a big profit” on the sale of backnumbers of its journal. This plan, however, depended on having complete sets to sell, and mostsocieties had gaps in their warehouse holdings. This was why the Royal Societyembarked on a reprinting program to generate sets of its ProceedingsA  from 1939–56. By 1959 it was reporting “veryencouraging” sales, but the “considerable” upfront costs would be a barrier tomost societies.A more fruitful mechanism for income-generation turned out to be the exploitationof “hidden assets” or “buried treasure”: these were assets owned by the societythat “may be profitably reproduced.” A year later, its sales were said to be “going very well.” Extra revenue from the exploitation of “hidden assets” did not change thelong-term sustainability of a society’s day-to-day journal publishingoperations, but it could be used to plug any remaining gaps in the publicationfinances in the short term, or as capital to support innovation.This was where the flexibility available to the Nuffield scientific advisorycommittee proved its worth: from spring 1958, it began to offer interest-freeloans to societies see . The sca It became even more common in the 1960s and appears to be linked to thewillingness of private funders and U.S. government agencies to pay such chargeson behalf of their authors. The absence of page charges from the UK debates presumably reflects thedifferent history of government funding for scientific research, and a differentpre-existing mechanism for supporting its publication. It is unclear how much British and American learned societies knew abouteach other’s practices.For modern academic authors, the absence of any discussion of page charges mayseem striking. In the United States at this time, some learned societies wereindeed trying to improve the financial situation of their journals by askingauthors to contribute to the cost of publishing their articles. The AmericanInstitute of Physics had been doing it since the 1930s, and in 1954 the NationalScience Foundation reported that at least sixteen society publishers were usingpage charges. There was a general sense that the committee’s efforts had been successful,and its work would be taken forward in two ways. First, Morley continued to work forthe Foundation, overseeing the completion of the “select lists” of journals fordistribution by the British Council overseas, offering advice to more societies, andtransforming his private report into Self-Help for Learned Journals(1963). Second, some of the other committee members continued to engage with theproblems facing learned journals through the Royal Society’s “scientific informationcommittee.” For instance, in 1963 the Royal Society hosted a meeting ofrepresentatives from fifty-five UK learned societies at which a draft ofSelf-Help was circulated to stimulate discussion about theproblems of scientific publishing; this led to an annual Conference of Editorsthroughout the 1960s.By 1960, the Nuffield Foundation’s scientific advisory committee had reached the endof its original terms of reference. Frank Morley submitted his report to theeighteenth and final meeting in October that year, and was thanked by the chairman,Edward Salisbury, for his “valuable work.”Self-Help praised the Nuffield committee’sintervention as a “major service to scientific publication in this country” that hadresulted in “vistas of growing deficits” being turned into “prospects of financial stability.” Morley was more down-to-earth, writing that it was “a success story – if onlyto the extent that none of the patients died.” Few traces survive of the financial details of the societies that receivedassistance, though copies may survive in the societies’ own archives. For instance,in January 1958 the honorary secretary of the Royal Anthropological Institute reported that the sales income from itsjournals had been greater than in any previous year. She was unwilling to attributethis improvement entirely to the Nuffield grant for a promotion drive, but she didallow that “some portion of this is due to their assistance and is surely in theright direction!” Eighteen months later, the Nuffield committee was pleased to learn that theRoyal Anthropological Institute’s annual publication deficit of over £2,000 in1956/7 had been reduced to just £17. That a deficit of £17 was seen as a success reminds us that the aim wassustainability, not – in contrast to later decades – profit.Salisbury’s preface to The Nuffield scientific advisory committee’s success in persuading learned societiesto analyze and adjust their journal publishing practices was due partly to itsability to draw upon the Royal Society’s institutional and personal networks, andpartly to its appointment of Frank Morley. Morley’s personal experience andbackground was key, but the very decision to hire a consultant, and offertransformative advice as well as money, distinguished the scientific advisorycommittee’s approach from that of the parliamentary grant-in-aid committee, variousprivate charities, and even (apparently) the Nuffield Foundation’s British Academylearned journals committee. Their survival has meant that, although academic journal publishing in theearly twenty-first century has come to be dominated by just four international mediaconglomerates, those four firms do not dominate the natural sciences as much as theydo (for instance) the social sciences. Scientific learned societies in both the UK and the United States remainimportant journal publishers, alongside the big commercial firms. It is true thatthe most significant UK society publishers – the Institute of Physics and the RoyalSociety of Chemistry – were among those that were already designated “successful” in1955, but the Nuffield project enabled many smaller society publishers tosurvive.Learned society journals did not, therefore, go extinct in the “lean years” between1955 and 1962.Robert Maxwell created Pergamon Press to thrive in the conditions of postwarscientific research, with its expanding number of specialist fields, theinternational scope of its collaborations and communities, and its well-fundedlaboratories and libraries. Older journal publishers, in contrast, had to learn howto adapt their existing practices to the changed conditions. This was true of botholder firms, such as Taylor & Francis and Wiley, and the learned societypublishers. But the transition was more dramatic for the learned societies, as itinvolved reinterpreting their mission to circulate the results of scientificresearch as widely as possible. Rather than being seen in largely philanthropicterms, journal publishing came to be seen as a commercial enterprise – even atlearned societies whose officers warned of the potential dangers of undue commercialinfluence. An unwillingness to abandon such work underpinned the initial warinessdisplayed by many learned society publishers toward the open access movement of theearly twenty-first century.It is a change that has had long-term consequences for learned society publishers.Becoming self-supporting turned out to be not only a solution to the immediateexistential threat, but the first step toward making profits. By the later twentiethcentury, publishing income enabled learned societies to support a variety ofactivities – from policy work, to research grants, prizes, and the promotion ofscience in schools – that would have been utterly impractical for theirnineteenth-century predecessors.This paper demonstrates that a sales-driven, profit-seeking commercial approach isnot a natural or essential element of scientific journal publishing. It has come todominate journal publishing only since the 1950s and 1960s, and it did so in veryparticular politico-economic and technological circumstances, not all of which stillpertain in the early twenty-first century. This paper also reminds us that evenorganizations with proud histories and engrained practices can adapt successfully tochanged circumstances. whether an existential threat, similar to that facing learnedsociety publishers in the 1950s, is a necessary stimulus.


We provided journals with peer reviewers specifically assigned to check for undisclosed discrepancies between registrations and manuscripts submitted to journals. We performed discrepancy review on 18 manuscripts submitted to Nicotine and Tobacco Research and three manuscripts submitted to the European Journal of Personality. We iteratively refined the discrepancy review process based on feedback from discrepancy reviewers, editors and authors. Authors addressed the majority of discrepancy reviewer comments, and there was no opposition to running a trial from authors, editors or discrepancy reviewers. Outcome measures for a trial of discrepancy review could include the presence of primary or secondary outcome discrepancies, whether publications that are not the primary report from a clinical trial registration are clearly described as such, whether registrations are permanent, and an overarching subjective assessment of the impact of discrepancies in published articles. We found that discrepancy review could feasibly be introduced as a regular practice at some journals interested in this process. A full trial of discrepancy review would be needed to evaluate its impact on reducing undisclosed discrepancies.Undisclosed discrepancies often exist between study registrations and their associated publications. Discrepancies can increase risk of bias, and when undisclosed, they disguise this increased risk of bias from readers. To remedy this issue, we developed an intervention called  Other rbefore publication would be more desirable than relying on individual readers' scrutiny and correction efforts. One trial attempted to reduce discrepancies at medical journals by sending peer reviewers information about the registration associated with the manuscript they are reviewing  of registered studies have at least one primary outcome discrepancy and 50–75% (95% CI) have at least one secondary outcome discrepancy   and in h% CI of rdiscrepancy review—to improve transparent reporting between registrations and their final manuscript. Our intervention provides journals with peer reviewers specifically assigned to check for both outcome and non-outcome discrepancies and asks them to prepare an itemized list of constructive recommendations to manuscript authors for how to reduce or disclose discrepancies between their registration and submitted manuscript. Importantly, the goal of discrepancy review is to encourage transparent reporting, not to inhibit publication of manuscripts with discrepancies. In many cases, discrepancies are entirely justifiable and can be necessary to improve a study design or analysis.Here, we propose a novel intervention—. 1.1We pre-registered three overarching objectives, but no hypotheses. Our objectives were to (1a) evaluate the feasibility of incorporating discrepancy review as a regular practice at scientific journals, (1b) evaluate the feasibility of conducting a trial on discrepancy review; (2) explore the benefits and effort required to incorporate discrepancy review as a regular practice at scientific journals and (3) refine the discrepancy review process.. 1.2registration throughout this manuscript to refer to time-stamped documents stored in a permanent and publicly accessible repository  that contain information about a study . Registrations can be prospective (posted before participant enrolment) or retrospective (after participant enrolment begins). Another term for prospective registration is pre-registration. While these terms are sometimes used interchangeably, clinical trials often use the term prospective registration, and various other disciplines often use the term pre-registration. Clinical trial registrations differ from registration on platforms such as the Open Science Framework (OSF) in their history, the functions they were designed to serve, and implementation details . Because of these differences, we try not to conflate clinical trial registration with pre-registration more broadly and mostly use the term registration, which encompasses both and makes no assumption regarding the timing of registration.We use the term discrepancy to refer to any incongruity between the content of a manuscript and its associated registration. We use the term discrepancy review to encompass the entire process we developed to check for and report on discrepancies, and discrepancy report for the written report that comes from discrepancy review and is shared with the action editor.We use the term . 2There are several minor discrepancies between this manuscript and the pre-registered protocol, which are outlined in the electronic supplementary material A.. 2.1Via email, we invited the editors-in-chief of 18 journals in medicine and psychology to participate. We selected 11 journals from the list that offer pre-registered badges (cos.io/initiatives/badges) as well as seven additional journals our team thought might be interested in participating. We selected journals based on previous contact with editors-in-chief, a presumption of their interest in open science, and coverage of disciplines that the project lead (RTT) is familiar with.. 2.2n discrepancy reviews performed = 3), assistant professor , postdoctoral researcher  and PhD student .Discrepancy reviewers included the project lead, a frequent collaborator and three team members recruited from a call on Twitter and selected for their experience with registration and range of career stages. While all the reviewers have experience with meta-research and registration, as well as general knowledge in neuroscience or psychology, they did not have domain expertise in the topics of personality psychology and nicotine/tobacco research which they reviewed. At the time of reviewing, their career stages included associate professor , which we used on 16 manuscripts, we developed and switched to a simplified process ?Is the registration  3.retrospective?Does the registration date suggest that the timing of registration may be  4.hypotheses?Are there discrepancies in the number, content or prioritization of  5.independent variables, exposure variables or study/experimental grouping match?*Do the study arms,  6.outcome measures?Are there discrepancies in the number, content or prioritization of  7.analyses match?*Do the  8.additional discrepancies or questionable research practices you may raise them. These could include sample size, control variables, covariates or moderators, eligibility criteria, analytic decisions (e.g. outlier definition), randomization, blinding and data preprocessing, among others.*While answering the previous questions, if you identified notable *For these items, reviewers were instructed to not spend time looking for minor discrepancies. Electronic supplementary material E contains the exact instructions for the updated discrepancy review process.. 2.5Two team members independently prepared a written discrepancy report for each manuscript. Only the first report was sent to the action editors. We used the second report to examine consistencies and differences among discrepancy reviewer reports. For eight manuscripts, both the first and second discrepancy reports were written after performing the original discrepancy review process; for eight manuscripts, the first report was based on the original discrepancy review process and the second report was based on the updated discrepancy review process; for five manuscripts, both reports were based on the updated discrepancy review process.. 2.6We sent an optional questionnaire to action editors and manuscript authors to help us refine the discrepancy review process and understand it from their perspective. The questionnaire had 16 questions. These included whether editors and authors would use discrepancy review again, how much time it took them to implement discrepancy review, whether the added benefit of discrepancy review was worth the effort, and open-ended questions about likes, dislikes and other thoughts regarding discrepancy review. The questionnaires are provided in the electronic supplementary material H and I.. 3. 3.1Of the 18 journals we contacted, five agreed to participate, two were interested in participating but did not begin sending us manuscripts by the time we concluded data collection, six declined to participate and five did not respond. Of the five journals that agreed to participate, two did not receive any manuscripts reporting a registration during the study period, and one had difficulty adding discrepancy review to their manuscript handling procedures and therefore did not send us any manuscripts to review.Of the journals that declined to participate, three expressed interest but were currently too busy, one stated they receive too few manuscripts reporting a registration and believed the process would be complicated to set-up, one stated they were uncomfortable using their journal for research purposes and one stated their editors already check for discrepancies rendering their journal ill-suited for the present study. All invitations were sent within the first year of the COVID-19 pandemic, which may have impacted the willingness of journals to participate.Nicotine and Tobacco Research over a period of three months and three manuscripts submitted to the European Journal of Personality over a period of five months. When assigning manuscripts to peer review, the action editors at Nicotine and Tobacco Research additionally invited our team to perform discrepancy review. These reviews were submitted through an online submission portal (ScholarOne) within three weeks of invitation, as done for regular peer reviews. One of our team members (MRM) is the editor-in-chief of Nicotine and Tobacco Research and served as action editor on two manuscripts included in this study, but did not perform any of the discrepancy reviews. The editor-in-chief of the European Journal of Personality invited our team to review manuscripts on which he himself served as the action editor. For manuscripts that passed an initial round of review at the European Journal of Personality, the editor-in-chief shared the manuscript with our team and informed authors that their manuscript would soon undergo discrepancy review. We conducted discrepancy review within a week and the editor-in-chief sent our review to the manuscript authors in an email separate from the standard peer reviewer comments and editorial decision.We reviewed 18 manuscripts submitted to in button on the journal's manuscript submission page; (ii) an opt-out button on the journal's manuscript submission page; (iii) through an automated email following manuscript submission or (iv) in the email containing the initial editorial decision. One journal chose Option (i) and had an opt-in rate of 79%2 over four months. The opt-in text was relatively vague and stated that the journal had partnered with researchers to improve peer review, but did not mention anything about registration in particular.3 Due to difficulties setting up the opt-in button, editors forgetting to flag manuscripts that both opted-in and reported a registration, and a limited number of manuscripts that met both these criteria, we did not receive invitations to review manuscripts from this journal. One participating journal selected Option (iv), informing authors in the same email as the initial editorial decision and standard peer review comments. The other participating journal selected a modified version of Option (iii), where they manually sent an email to authors after an initial editorial decision to revise and resubmit, but before discrepancy review. These emails informed authors that they could withdraw the data collected based on their manuscript. No author requested their data be withdrawn.We suggested that journals inform manuscript authors about the study in one of four ways: (i) an opt-. 3.2We planned to use discrepancy review for two purposes: to provide the basis for writing discrepancy reports (the intervention) and to evaluate the presence of discrepancies in published manuscripts (for outcome assessment). We found our original implementation of discrepancy review fit for neither purpose; mainly because registrations were less comprehensive and less precise than we anticipated. Electronic supplementary material, table B1 lists additional shortcomings in the original discrepancy review process and how we addressed them. Nonetheless, the data collected from the original discrepancy review process, which we used as the basis for writing discrepancy reports, provided several insights : many di. 3.2.1reports are provided in §3.4 of this manuscript). In particular, inter-rater agreement was low for judgements of discrepancies as negligible, minor or major . The original discrepancy review process allowed for subjectivity. For example, the survey asked reviewers to state whether each of the 18 dimensions was reported ‘sufficiently’ for the reviewer to replicate the study if they had the expertise.4 We also asked reviewers to judge whether a discrepancy or lack of transparency for each dimension presented a negligible, minor or major issue.5 Whereas some reviewers made use of ‘negligible’ for issues of transparency (where an item was reported in neither the registration nor the manuscript), other reviewers often considered these minor issues. Some reviewers also used the option n.a. (not applicable) more frequently than others, in turn lowering inter-rater agreement. Taken together, we concluded that our original implementation of discrepancy review could not reliably assess whether discrepancies were present, or the severity of their impact.Inter-rater agreement was low across many of the 18 dimensions coded in the original discrepancy review process (inter-rater comparisons of the written discrepancy . 3.3n reviews = 8) versus 105  min (n = 16); OSF registrations: 50  min (n = 7) versus 210  min (n = 7); PROSPERO registrations: 43  min (n = 3) versus 90 min (n = 1)  min ( (n = 1) . We feel. 3.46 manuscripts that were not the main paper related to a clinical trial registration; substantial discrepancies in the outcome measures of studies that used clinical trial registrations and substantial discrepancies in the hypotheses of studies using OSF registrations. Three pairs of reports differed considerably. One reviewer reported rushing two reviews and agreed with the presence of major issues when the other reviewer presented them. The third pair of reports was in regard to the first manuscript we received. It was the first manuscript reviewed, but the last manuscript to receive a second review—by which point we had substantially refined how we performed discrepancy review.We qualitatively compared each pair of written discrepancy reports and found that their content was relatively similar in regard to substantial discrepancies, but less similar for minor concerns. These were comparisons of the written text in the discrepancy reports and are distinct from the inter-rater agreement we calculated across the 18 dimensions in the original discrepancy review process and the inter-rater agreement of potential outcome measures presented in §3.6. Regardless of whether reviewers used the original or updated discrepancy review process, their reports were relatively consistent in identifying non-permanent registrations ;Reports often differed in the minor issues they presented, which may have arisen due to the latitude reviewers had in what to report. Reports based on the updated discrepancy review process mentioned fewer minor issues. The style and wording of reports differed somewhat between reviewers. In summary, although inter-rater agreement was low across the specific 18 dimensions coded in the original discrepancy review process, the written reports had comparable overall conclusions.. 3.5Five of the 13 action editors involved in our study responded to an optional questionnaire in regard to one of the manuscripts they handled. All five editors responded that they would use discrepancy review again and that it would be possible to implement discrepancy review for all manuscripts submitted to their journal if they were provided with discrepancy reviewers. To limit the impact our study had on the publication process beyond transparent reporting, we specifically asked editors to not allow discrepancy review to influence their editorial decision. We found that four of the five editors reported that discrepancy review did not influence their editorial decision and the fifth reported it encouraged acceptance. Three editors rejected the manuscripts they received and two asked for revisions. Our questionnaire prompted these two editors who asked for revisions with additional questions, such as the time it took them to implement discrepancy review. They reported taking 6 min and 30 min and that it was worth their time.n = 2), during (n = 3) or after (n = 2) standard peer review (one author responded ‘unsure’ and another author did not respond). The questionnaire included questions about the value of discrepancy review as well as an optional open-ended question asking what they disliked about discrepancy review. Two authors but no editors answered the question about disliking discrepancy review. One author disliked that discrepancy review increased the overall word count and another disliked that the discrepancy review provided a recommendation to include information regarding missing data, but that this information was already reported in the manuscript. Across the questionnaires, no response indicated against conducting a trial of discrepancy review .Authors from 4 of the 21 manuscripts responded to an optional questionnaire. They all agreed that they would like their next manuscript to undergo discrepancy review and were positive about discrepancy review in their open-ended responses. They reported taking 30, 90 and 180 min to address the discrepancy reviewer comments (one author did not answer this question). Responses from both authors and editors were mixed regarding whether discrepancy review should be implemented before (. 3.6Our pre-registered outcome measure of discrepancies across each of the 18 dimensions was intended to answer two overarching questions: (i) whether authors address comments from discrepancy reviewers and (ii) whether a trial of discrepancy review could use this outcome measure. Based on information gained while performing this feasibility study, we revised our outcome measures to better answer these same questions.. 3.6.17 We coded several comments as partially addressed because the text in the published manuscript was imprecise. Inviting the discrepancy reviewers to review revised manuscripts may ensure that comments are more precisely addressed. In short, these data show that revised manuscripts often take into account the comments from discrepancy reviews. We do not plan to use this outcome measure for a trial. Rather we would recommend comparing endpoints between the control and experimental groups. Nonetheless, it provides evidence that discrepancy review has the potential to impact reporting quality.Our first revised outcome measure is whether published manuscripts addressed the specific recommendations in the discrepancy reports. For the first few discrepancy reports we prepared, each recommendation was directly linked to one of the 18 dimensions. However, we found this way of writing discrepancy reports cumbersome and switched to writing itemized comments that were not necessarily directly linked to one of the 18 dimensions. Thus, we assessed whether authors addressed the itemized recommendation of the discrepancy reports, but did not subdivide between the 18 dimensions. The discrepancy reports contained between 1 and 10 comments . Of the 59 comments in the 12 manuscripts that were published, 31 were fully addressed, 10 were partially addressed, 17 were not addressed and 1 presented issues that the discrepancy reviewer later noticed did not need to be addressed. We categorized these 12 manuscripts into four bins: six addressed all or nearly all comments, three largely ignored discrepancy review and addressed very few comments (accounting for 13 of the 17 comments that were not addressed), one addressed some comments and two were secondary publications associated with a clinical trial registration but did not explicitly state that some aspects of their study were not registered. One manuscript did not address any comments, probably due to an editorial misunderstanding.. 3.6.2We selected several other revised outcome measures that could feasibly be used in a trial. These include whether published manuscripts (i) are properly registered, (ii) are transparently identified as secondary publications associated with a clinical trial registration, when relevant, (iii) contain at least one primary outcome discrepancy, (iv) contain at least one secondary outcome discrepancy, and (v) are assessed to have discrepancies that are important to address .κ = 1.00) on whether there were issues in the registrations for submitted manuscripts (where 4/21 had issues) and had one disagreement (Cohen's κ = 0.75) for published manuscripts (where 2/12 had issues). Coders agreed when identifying whether manuscripts were a secondary publication associated with a clinical trial registration for 18/21 submitted manuscripts (Cohen's κ = 0.67) and 11/12 published manuscripts (Cohen's κ = 0.8). Of the six published manuscripts linked with clinical trial registrations that were published, only two of them were the main report of the clinical trial. Coders agreed that neither had a primary outcome discrepancy and that both had at least one secondary outcome discrepancy. Coders' subjective assessments of whether discrepancies were ‘quite important’, ‘somewhat important’ or ‘not important’ to address8 matched in five cases were off by one category in seven cases, and off by two categories in one case for submitted manuscripts (Cohen's κ = 0.05). For published manuscripts, they matched in four cases and were off by one category in one case, and off by two categories in two cases (Cohen's κ = −0.21). Inter-rater agreement was low for these subjective assessments, but differences in coding were easily resolved through discussion because one coder would often share information that the other coder had missed. In general, the coder who stated discrepancies were more important to address shifted the other coder's rating. More precise instructions for these subjective assessments may improve inter-rater agreement to the point where it could be reliably used as an outcome measure in a trial.Inter-rater agreement varied greatly across these potential trial outcomes. Coders had a perfect agreement (Cohen's . 4We found that (i) when provided with discrepancy reviewers, some journals can incorporate discrepancy review into their manuscript handling procedures; (ii) registrations were less precise and less comprehensive than our original discrepancy review process was designed for; (iii) a semi-structured discrepancy review process was more feasible to implement than a rigidly structured and comprehensive implementation; (iv) authors addressed the majority of discrepancy reviewer comments, (v) clinical trial registrations and OSF registrations should be treated separately in a trial, as the former are generally more precise and the latter are generally more comprehensive, and (vi) a trial of discrepancy review appears feasible in terms of the procedure and outcomes measures, particularly so for clinical trial registrations where primary and secondary outcome are consistently and clearly demarcated.The outcome measures that could be used in a trial should depend on the types of registrations being reviewed. In contrast with OSF registrations, clinical trial registrations in our sample were always properly registered, but many manuscripts were secondary publications associated with a clinical trial registration. OSF registrations were generally less precise than clinical trials registrations, making it difficult to confidently claim an absence of primary or secondary outcome discrepancies. The lack of standardization in the content and precision of OSF registrations may also makes the process of awarding pre-registered badges challenging. Box 2.Eligibility criteria. Manuscripts submitted to participating journals that include a clinical trial registration.Design: Two-arm parallel-group randomized controlled trial. Randomized at the level of submitted manuscripts within each journal.Intervention. Updated discrepancy review process.Primary outcome. Absence of undisclosed primary outcome discrepancies. If a manuscript is a secondary publication associated with a clinical trial registration and is clearly labelled as such, it will be considered to have no undisclosed primary outcome discrepancies. This binary outcome will be coded from manuscripts accepted for publication.Other outcomes. The time to prepare discrepancy reports. To be used to inform future cost-effectiveness analyses.Sample size and analysis. This trial could be approached with a power analysis or precision analysis. For a power analysis, we would need to select a smallest effect size of interest. Determining this effect is difficult without knowing the cost that journals are willing to incur for discrepancy review and the cost of undisclosed discrepancies . A precision analysis, in contrast, remains agnostic regarding whether the effect is meaningful and could be used in follow-up cost-effectiveness analyses.α = 0.05, power = 0.80). Alternatively, to estimate the difference between the two groups within a range of 20% , would require 120 manuscripts per group .9Both power and precision analyses require assumptions about the proportion of manuscripts with discrepancies in both the control and experimental group. For the control group, we assume a base-rate of 33% of manuscripts with at least one primary outcome discrepancy, which is the point estimate from a relevant meta-analysis [Several questions remain unanswered. First, we have not established the degree to which general research knowledge and domain expertise—which the discrepancy reviewers did not have in relation to the manuscripts they reviewed—facilitate or improve discrepancy review. Second, our study was not designed to identify elements of discrepancy review that could be automated (e.g. with a standard email asking authors to ensure accurate reporting). Third, our study did not evaluate the feasibility of randomizing manuscripts to the discrepancy review intervention, which could pose a challenge when conducting a trial. Fourth, it remains unknown whether editors who received discrepancy reviews will become more aware of these issues and check for discrepancies in future manuscripts. Fifth, we did not assess how easy it would be to recruit discrepancy reviewers outside the context of a trial and whether journals are willing to incur those potential costs. Additionally, the respondents to our questionnaires were probably biased towards those who were more interested in discrepancy review.Taken together, a trial of discrepancy review appears feasible if enough journals agree to participate. Such a trial may provide evidence for one potential intervention to improve reporting quality. At the same time, this intervention cannot directly improve study design or the quality of registrations and may add to the workload of researchers and reviewers. Parallel efforts that aim to improve quality early on in the research pipeline (e.g. pre-study peer review) could prove complementary to interventions, such as discrepancy review, that occur after study completion.