Peer review is the "gold standard" for evaluating journal and conference papers, research proposals, on-going projects and university departments. However, it is widely believed that current systems are expensive, conservative and prone to various forms of bias. One form of bias identified in the literature is “social bias” linked to the personal attributes of authors and reviewers. To quantify the importance of this form of bias in modern peer review, we analyze three datasets providing information on the attributes of authors and reviewers and review outcomes: one from Frontiers - an open access publishing house with a novel interactive review process, and two from Spanish and international computer science conferences, which use traditional peer review. We use a random intercept model in which review outcome is the dependent variable, author and reviewer attributes are the independent variables and bias is defined by the interaction between author and reviewer attributes. We find no evidence of bias in terms of gender, or the language or prestige of author and reviewer institutions in any of the three datasets, but some weak evidence of regional bias in all three. Reviewer gender and the language and prestige of reviewer institutions appear to have little effect on review outcomes, but author gender, and the characteristics of author institutions have moderate to large effects. The methodology used cannot determine whether these are due to objective differences in scientific merit or entrenched biases shared by all reviewers. As reported by Armstrong, “journal peer review is commonly believed to reduce the number of errors in published work, to serve readers as a signal of quality and to provide a fair way to allocate journal space”3. Surveys of authors and expert reviewers6 support this view. However, many members of the scientific community also believe that peer review is expensive, conservative and prone to bias18. Critics point to the major delays it introduces into the publication process19, to biases against particular categories of papers , to the unreliability of the review process25, to its inability to detect errors and fraud26, and to unethical practices by editors and reviewers28.Peer review is the “gold standard” for the evaluation of journal and conference papers, research proposals, on-going projects and university departments and there is a strong consensus in the scientific community that it improves the quality of scientific publicationsAnother common criticism of peer review is that review results are not determined exclusively by scientific merit, but also by the social and demographic characteristics of authors and reviewers. In some cases, these effects may constitute a form of social bias. For instance, reviewers may give different scores to papers of equal merit by authors with different personal characteristics . Differences may also be determined by the interaction between author and reviewer characteristics, i.e. by biases of reviewers with specific attributes for or against particular categories of author. Finally, reviewers with different personal characteristics could score the same paper differently. Although this would not constitute bias, it would mean that the make-up of the review panel for a paper would affect its score, independently of its scientific merit – an undesirable result.Behavioral Ecology led to an increase in the number of accepted papers with female first authors, compared to five similar journals where reviewers were not blinded to author gender. In the same spirit, a widely cited study of grant awards in Sweden30 suggests that proposals from male candidates receive systematically higher evaluations than those from female candidates with similar academic records, a result confirmed by a recent follow-up study15. A meta-analysis of 21 studies of peer review to assess research applications or applications for post-graduate fellowships also found robust gender effects on peer review results31. All these findings suggest that gender bias is real. However, other studies reached opposite conclusions. For instance, Budden’s study ofBehavioral Ecology was later contested byNature andScience is unrelated to gender33.Studies of the effect of potential bias against authors with specific characteristics have focused on gender. For example,34. A study by Tregenza reports that review results vary with the country of origin of the author35. Marsh and colleagues36, show that grant applications from authors in high-ranking Australian universities are accepted more frequently than applications from authors in lower-ranked institutions. However, the authors of these studies agree that these differences do not in themselves constitute proof of bias and could simply reflect differences in scientific merit, as discussed inStudies of potential bias with respect to other author characteristics  have been less frequent but have also produced contrasting results, comprehensively reviewed by Lee and colleagues37 report a quasi-experiment demonstrating that papers by authors from high-prestige institutions have a significantly higher chance of acceptance than similar papers by authors with less prestigious affiliations, and participants in surveys of authors are reported to believe in such an effect38. This finding is supported by a study of papers submitted to scientific sessions at the American Heart Association’s annual research meeting39, which shows, not only that mean review scores for papers by authors from institutions in non-English-speaking countries and from institutions of low academic prestige, are lower than those for papers by authors from English speaking countries, and from higher prestige institutions but also that these differences are lower in reviews where the reviewers have been blinded to authors’ identities and affiliations. This is evidence that review scores may indeed be affected by social bias.More convincingly, for the purposes of this paper, Peters and Ceciinteraction between author and reviewer characteristics. For instance, an experimental study by Lloyd40, reports that manuscripts with female author names have a far higher acceptance rate when they are reviewed by female rather than male reviewers (62% vs. 21%) and that female reviewers accept papers with male author names less frequently than male authors (62% vs. 10%). Link has shown that while US and non-US authors both give higher scores to US than to non-US authors, the difference is significantly higher when the reviewer comes from the USA41. Similarly, Jayasinghe and colleagues report that papers by Australian authors are more likely to be accepted by Australian reviewers than by reviewers from other countries42. Again however, not all the evidence points in the same direction. For instance, Marsh and colleagues’ previously cited study of Australian review practices36 finds no interaction between researcher and reviewer gender.Additional evidence can be obtained by studying the43. However, the sample size was small and the effect was observed only for “statistics reviewers” and not for “content reviewers”. Marsh has shown that US reviewers asked to review Australian grant applications give higher review scores than reviewers from other countries36. A study by Wing and colleagues reports that female reviewers were less likely than men to accept or accept with minor revisions, producing a higher proportion of reviews that external assessors judged to be very good or exceptionally good44. By contrast, a study by Caelleigh and colleagues shows no significant effect of reviewer gender45.Compared to evidence for social bias with respect to author characteristics and interaction effects, support for systematic differences in scoring behaviour by reviewers with different characteristics is relatively weak. An experimental study by Gilbert and colleagues finds that the distribution of review scores given by male reviewers were broader (i.e. more extreme) than the distribution of scores from female reviewersAttempts to remedy potential biases present in traditional peer review have led to a diversification of peer review practices, for instance through the use of author-blind and non-selective review, the removal of traditional reviewer anonymity, and the introduction of various forms of community review. To date, however, there have been few attempts to measure their effectiveness. Furthermore, many past studies of bias in peer review are relatively old and it is not clear whether or how far biases detected in the past have been affected by changes in social attitudes. In response to these concerns, we analyze data for the peer review systems used in Frontiers (an open access publishing house which uses a novel interactive review process), three computer science conferences  held in Spain between 2003 and 2007 and four international computer science conferences , held between 2002 and 2011.Frontiers is a large open access scientific publisher, which published its first journal in 2007. In January 2015, Frontiers had a portfolio of 49 open-access journals with over 50,000 researchers serving on its editorial boards and more than 380 academic specialty sections. Papers are published within these sections. Each paper is assigned to a scientist acting as the editor who coordinates the review process, and is responsible for publication/rejection decisions. Reviewers are selected automatically, based on the match between their individual specialties, and key words in articles submitted for review . During the review process, reviewers remain anonymous. However, accepted papers carry the names of their reviewers. This gives reviewers a strong incentive not to accept papers before they have reached a good level of quality.select papers as toimprove the quality of papers in a collaborative, interactive dialog between authors (whose identities and affiliations are available to reviewers) and reviewers (who at this stage of the process are anonymous). In other words, Frontiers adopts a single blind review process. At the beginning of the review process, in which the reviewers review submitted papers independently of each other, they are asked to answer a series of open questions concerning different aspects of the paper , a computerized system for managing the submission and review of conference papers. WebConf was developed by a team from Malaga University led by one of the authors (RC). WebConf implements a classical review process similar to the processes used by Springer, Elsevier and other large commercial publishers. Conference contributions are usually reviewed by three independent reviewers, occasionally by two or four. Reviewers are chosen by the conference program chair, who draws on a database of potential experts called the program committee. In general, the program committee is made up of authors who have previously submitted papers in a particular area of research and have expressed their willingness to act as reviewers. The WebConf system suggests a list of potential reviewers based on the degree of matching between paper topics and reviewers field of expertise. The final selection is based on the judgment of the program chair.The conferences in these two datasets all used WebConf  and textual comments. Scores on individual scales are usually expressed in terms of categories . The final publication decision depends on the program chair. If one of the reviewers expresses a strongly negative view of a paper, it will typically be rejected. In cases where there is a very significant difference in reviewers’ opinions, the program chair can ask for an additional review. Acceptance rates vary between a minimum of 29.3% and a maximum of 87%. The combined acceptance rate for the seven conferences in the WebConf database was 57.9%.Frontiers. The Frontiers database includes details of all authors and reviewers for all scientific papers submitted to Frontiers  between June 25, 2007 and March 19, 2012, the name of the journal to which the paper was submitted, the article type , the name and institutional affiliations of the authors and reviewers of specific papers, individual reviewers scores for the summary scales shown inSpanish computer science conferences (IEEE-Spain). This dataset includes details of 1,131 reviews referring to 411 papers submitted to three IEEE conferences . The majority of authors and reviewers for these papers come from institutions in Spain and Portugal. The data provided include the name of the conference to which the contribution was submitted, the type of contribution  the name, gender and institutional affiliations of the authors and reviewers of specific contributions, individual reviewers scores and the final decision (accepted/rejected). All the papers in the database are in the area of computer science.International computer science conferences . This dataset provides data for 2,194 reviews, referring to 793 computer science papers submitted to four IEEE conferences , managed using WebConf and involving authors and reviewers from all over the world. This dataset provided the same data collected for IEEE-Spain.Names of authors and reviewers were canonized: accent and symbols were removed; double spaces replaced by single spaces, and upper-case characters replaced with lower-case characters. Names were rewritten in the normalized form <first name, last name>. Intermediate names were omitted. The only role of names in our analysis was to allow identification of author and reviewer gender in the case of the Frontiers dataset. Any errors would thus have only a limited effect on the analysis of gender bias for this dataset. Quality controls on automated gender assignment are described below.Names of institutions  were canonized as above. After normalization, the name of the institution was recoded using the first three words in the full name. The only use of names of institutions in the analysis was to infer their Shanghai ranking. Quality controls on the attribution of Shanghai ranking are described below.http://pecl.php.net/package/gender). We then used volunteers of different nationalities  to assign genders to first names not contained in the dictionary. Additional names were assigned by manually searching for specific authors and reviewers on Google and Facebook. At the end of this process, we were able to assign genders to 96.4% of authors and 87.1% of reviewers. The majority of unassigned names were Asian (mainly Chinese). Genders for WebConf authors and reviewers were inferred manually, with no missing values in the dataset.Neither the Frontiers nor the WebConf databases included data for author and reviewer gender. In the Frontiers case, gender was inferred semi-automatically in a multistep process. First we matched the first names contained in our database to an open source dictionary providing genders for more than 40,000 first names used in different countries . The difference was probably due to the broader range of countries  in the Frontiers dataset. For reasons we will consider in the discussion, errors in gender assignment are unlikely to have influenced the conclusions of our study.To analyze potential country, regional and language biases, we assigned each author and reviewer to the country of the institution to which they were affiliated, as listed in the original datasets. Authors and reviewers with multiple affiliations were assigned to the country of the first affiliation listed. In the Frontiers dataset, we deduced country information from the affiliation given by authors and reviewers. Since 9.0% of reviews lacked information on first author affiliation and 5.3% lacked the information for reviewers, it was not possible to deduce the geographical location or the language of their affiliated institutions. In the IEEE (Spain) and the IEEE  datasets, authors and reviewers provided country data information directly and the number of missing values was <1.0% in both cases. The geographical region and language of authors’ and reviewers affiliated institutions were determined using the country information. USA, UK, Scotland, N. Ireland, Ireland, Australia, Canada, and New Zealand were classified as English-speaking countries. All other countries were classed as non-English speaking.http://www.shanghairanking.com/FieldLIFE2012.html) (Frontiers) and for computer sciences (http://www.shanghairanking.com/SubjectCS2012.html) (WEBCONF). The institution names in the Shanghai ranking were normalized using the same procedure used to normalize institution names in the three datasets (see above). To check the quality of our automatic assignment of university ranking, we extracted a random sample of 125 universities from each dataset and checked the university ranking manually, finding error rates of 11.2% for Frontiers, and 9.6% for IEEE . The IEEE (Spain) dataset was excluded from the analysis for other reasons (see below).Authors’ and reviewers’ affiliated institutions were classified in terms of their position in the 2012 Shanghai academic ranking of world universities for the life sciences (Frontiers. The Frontiers review process produces a very low rejection rate. This means that the database used for our study contained relatively few rejected papers (N=478). To create a more informative indicator of reviewers’ evaluations, we computed for each paper the average of the scores expressed by each individual reviewer for each of the summary scales shown inWebConf. The WebConf system asks each reviewer to assign an overall score to the paper he/she has just reviewed. Scores are expressed on a scale of 0 to 10.ijδ in the random intercept model:For the purposes of the study, we define bias as the interaction termsij + βijAi + γijRj + δijAiRj + εy = b + µ          (1)b denotes the random intercept,i indexes properties of authors,j indexes properties of reviewers, andε is the error term. This method is similar but not identical to the method proposed inwhere y denotes the score given in the review,F, such asregion, the variablesiA andjR are indicator (dummy) variables indicating that the first author and reviewer belong to categories i and j of factor F, respectively, that is:Given a factori of F,i=1,A ELSEi=0A        (2)IF one author belongs to categoryANDj of F,j=1,R ELSEj=0R        (3)IF one reviewer belongs to categoryijβ is the fixed effect of author category,ijγ is the fixed effect of reviewer category andijδ is the fixed effect of the interaction between author and reviewer category. Since the expected value of the random interceptb is 0, the fixed effects allow us to estimate the following mean scores:Thusij = µij + βij + γij + δijG    author in categoryi, reviewer in categoryjjˇiG =ij + βijµ             author in categoryi, reviewernot in categoryjiˇjG =iˇjµ +ijγ              author not in categoryi, reviewer in categoryjiˇjˇ = µijG             authornot in categoryi, reviewernot in categoryj.ijB of reviewers from categoryj of factor F towards authors from category i by the expression:We define biasij =B (ijG –jˇiG) – (iˇjG –iˇGjˇ)ij= δ       (4)ijδ and does not depend on the main effectsijβ andijγ. In other words, it is independent of any general tendency of authors in category i to write better papers than other authors, or of any tendency of reviewers in category to give generally higher scores. In this setting, reviewers from categoryj arebiased in favor of authors from categoryi, if Bij > 0 and are biasedagainst authors from categoryi if Bij < 0. Bias is significant at a levelα, if we can reject the null hypothesis:Since the intercept and main effects cancel out, bias is the interaction termH0 :ijB = 0Otherwise, we assume absence of bias.The majority of papers in our databases had multiple authors. In preliminary studies, we explored statistical models that used this data in different ways: (i) the model used only the properties of the first author, (ii) the model used only the properties of the last author, (iii) the model considered the properties of all the authors. The three approaches yielded similar results (data not shown). In what follows, we apply the first method, unless otherwise stated.i andj denote female (F), andiˇ andjˇ male (M). Then the termsij = GFF, Gjˇi = GFM, Giˇj = GMF,G andiˇjˇ = GMMG, have the following meanings:To illustrate the concept of bias defined in , consideFFG   : mean score when first author and reviewer are femaleFMG  : mean score when first author is female and reviewer is maleMFG  : mean score when first author is male and reviewer is femaleMMG : mean score when first author and reviewer are maleIf we assumed that papers by female authors have the same quality as papers by male authors,FF – GFMG > 0       (5)would imply that female reviewers are biased in favor of female authors andMF – GMMG > 0      (6)would imply that female reviewers are biased in favor of male authors.FF ≠ 0), only if FF ≠ 0), FF – GFM) – (GMF – GMM) > 0, the following statements are equivalent:Female reviewers are biased in favor of female authorsFemale reviewers are biased against male authorsMale reviewers are biased against female authorsMale reviewers are biased in favor of male authorsIt should also be noted that, in our example, if B =  and 2,194 from IEEE . The relative proportions of male first authors and reviewers  were similar in all three datasets. In the Frontiers and IEEE  datasets, means scores for male first authors were significantly higher than those for female first authors . The IEEE (Spain) dataset showed the same pattern but the difference was not significant . Reviewer gender had no significant effect on review scores in any of the datasets, nor did the interaction between author and reviewer gender have a significant effect in any of the datasets. In brief, none of the datasets showed evidence of gender bias. The significance of these results will be examined in the discussion. Complete data for the analysis can be found inThe analysis examined the role of the region of first author and reviewer institutions in determining review scores, and tested for possible regional bias. Authors and reviewers were grouped into 11 geographical regions  according to the location of their respective institutions. To avoid problems with the convergence of the mixed model algorithm and to guarantee the statistical power of the analysis, pairs of first author/reviewer regions with less than 30 reviews were discarded. Distributions of author and reviewer regions differed significantly among the three datasets. In the Frontiers and International IEEE datasets, the majority of authors and reviewers came from institutions in North America and Western Europe, while the majority of authors and reviewers in the Spanish IEEE dataset came from institutions in Southern Europe.In all three datasets, differences between the scores of first authors from different regions were statistically significant, even after application of the Bonferroni correction for multiple hypothesis testing see. In the Differences in scores linked to the geographical location of reviewer institutions were rare see. In the To test for bias, we applied the random intercept model to all author/review region pairs with more than 30 reviews see. After aWe hypothesized that reviewers could be biased against papers written by authors who were not native English speakers. We, therefore, analyzed potential reviewer bias for and against papers whose first authors came or did not come from institutions in English-speaking countries. As a further test, we analyzed potential bias for and against papers, which had, or did not have, at least one author belonging to an institution in an English-speaking country.The Frontiers and the IEEE  datasets both included large numbers of authors and reviewers from institutions in native English-speaking, and from non-English-speaking countries. In contrast, approximately 97% of the authors and reviewers in the IEEE (Spain) dataset came from Spain. Since none of the papers with an English-language first author, and only one paper with at least one English author, were reviewed by an English-language reviewer, it was not possible to measure bias using the random intercept model. This dataset was therefore discarded from the subsequent analysis.In the remaining datasets, papers with a first author from an institution in a non-English-speaking country scored significantly lower that papers with first authors from institutions in an English speaking country, regardless of whether they were reviewed by native English-speaking or non-native English speaking reviewers . Reviewer language had no significant effect on score ; IEEE International: difference=0.01; p=0.80, d=0.0). In neither case did we find a significant interaction between author and reviewer language (not shown). Results for papers with at least one author from an institution in an English-speaking country were similar. Details of the analysis are shown inReviewers from institutions with high academic prestige could be biased in favor of authors from other high prestige institutions and against authors from lower ranking institutions. To test this possibility, we classified all authors and reviewers in the three datasets by the position of their institutions in the Shanghai classifications, as described earlier. The Frontiers and the IEEE International datasets both contained significant numbers of authors and reviewers from universities in all three categories. However, a large majority of authors and reviewers in the IEEE Spain dataset came from universities in category 3. Given the lack of data for authors and reviewers from higher-ranking institutions, this dataset was excluded from the subsequent analysis.In the Frontiers dataset, authors from universities in category 1 scored significantly higher than authors from category 2  and from category 3 , regardless of the origin of the reviewer. No significant difference was observed between the scores of authors in category 2 and 3. In the IEEE International dataset, authors from universities in categories 1 and 2 received similar scores and both scored significantly higher than authors in category 3 .In the Frontiers dataset, there was no significant difference between the scores given by reviewers in different categories. The IEEE  data showed no significant differences between scores from reviewers affiliated to institutions in category 1 and category 2 or 3 but a significant difference between scores from reviewers in category 2 and 3 . Neither dataset showed a significant interaction between the prestige of author and reviewer institutions. Full details of the analysis are given inThe results of the study see show thaThe finding that author characteristics have a significant effect on review scores is compatible with two distinct explanatory hypotheses. The first is that papers submitted by authors with a particular characteristic (e.g. authors from institutions in a particular region) are, on average, of higher scientific merit than papers by authors with different characteristics (e.g. authors from institutions in other regions). The second is that reviewers share a general bias against authors with particular characteristics, regardless of their own personal attributes (e.g. reviewers from institutions in English and non-English speaking countries share a bias against authors from non-English speaking countries). The methodology of the study cannot distinguish between these hypotheses.There are several ways of detecting generalized bias in peer review scores. These include experimental studies, comparisons between scores when reviewers are blinded to author characteristics and scores when they are not e.g.,40, ana42 but contradicts findings from other authors  and the IEEE  datasets use a traditional approach. Authors and reviewers in the Frontiers and the IEEE International datasets come from all over world. The IEEE (Spain) dataset is dominated by authors and reviewers from Southern Europe. Despite these differences, analysis of the three datasets gave similar results. This suggests that the findings of this study could be valid for a broad range of peer review systems. The large size of the datasets used in the analysis  provides additional evidence of robustness. The main differences between the datasets were in their patterns of regional bias, which are different in each dataset. Unfortunately the many differences between the Frontiers and the IEEE systems make it impossible to untangle the roles of different contributory factors.Our study does not evaluate the full set of potential biases described in the peer review literature. For instance, we do not consider confirmation bias or alleged reviewer biases in favor of positive results, sophisticated experimental and statistical methodology, or against interdisciplinary studies, replication studies, etc. These are important limitations. A second limitation is that the study makes no attempt to control for the quality of papers, as testified, for example by citations. A third is that the study methodology has so far been tested with just three peer review systems, all applying to scientific papers, and all with relatively high acceptance rates. It is possible that other forms of peer review, such as peer review of grant applications, are subject to different forms of bias.In conclusion, our study shows that authors' personal characteristics play an important role in determining the scores received in peer review, but finds no evidence that review results are influenced by the personal characteristics of reviewers, and only weak evidence for social bias due to interactions between author and reviewer characteristics. These findings do not rule out generalized bias against authors with specific characteristics or forms of bias not considered in the study.To protect the identities of authors and reviewers, the source data used for this study has not been made public. My comments have all been addressed. The revised version presents the authors’ findings in a clearer and more precise way. This serves to highlight the contribution this study makes to the existing literature on bias in peer review. Their efforts to revise to this extent were well worth it.I don’t have any further comments.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. All reviewers' points have been appropriately addressed.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. In this paper the authors analyses the relationship between the attributes of authors and reviewers and reviewer outcomes in three datasets, one an innovative peer-review model and two which use traditional peer review. They find no evidence of gender or institutional bias and limited interaction between author and reviewer region.Although social bias in peer review has been investigated in the past, I think this study is timely and the question of whether this previously found bias still exists with changing social norms and values is interesting. However, there is a need for some clarification of the methods before the significance of the authors’ findings can be determined.Major revisionsand reviewer attributes, or bias as a function of reviewer characteristics (ref 29). They make the point that their methodology cannot determine biases that are shared by all reviewers regardless of reviewer characteristics. In the Introduction, the authors switch between describing previous research on reviewer characteristics and previous research on author characteristics. For example, paragraph 2 is predominantly about studies of reviewer characteristics, but ends on author characteristics. Paragraph 3 appears to begins on author characteristics, but then cites studies on reviewer characteristics. It would be much easier for the reader to understand what contribution this study makes to the literature if the authors made a clearer distinction between current  evidence onreviewer characteristics (including those studies which looked at both reviewer and author characteristics) and other research on bias, which would include research focused only on author characteristics.The authors are very clear that their study investigates ‘bias’ as defined by the interaction between authorAs part of the peer review process for Frontiers reviewers can, if they wish, complete the ratings questionnaire shown in Fig 1 and it was these ratings that were used in the study analysis.  Do Frontiers reviewers complete the rating independently of each other before the collaborative process? If this is the case, it is not clear to me why individual scores for each paper were averaged for the Frontiers dataset, but apparently not for the other two datasets. If the aim of this study is to investigate the interaction between reviewer attributes and those of authors, wouldn’t averaging the reviewer scores in this way confound this aim?Alternatively, if the ratings form is completed by reviewers after the collaborative process, how have the authors accounted for the potential confounding effect of the collaboration?Also, if reviewers complete the rating independently of each other, the characteristics of peer review for the Frontiers dataset used in this study are the same as that for the other datasets, i.e. it is single blind peer review. In all three datasets the reviewers are made aware of the authors names, but the authors do not know the reviewers’ – is that correct?In the discussion the authors state that the findings of this study could be valid for a broad range of peer review systems. However, this study did not include the interactive component of Frontiers peer review process, or if it did, it is not clear how. All three datasets appear to have used the single blind system of peer review. This statement in the discussion should be rephrased.MinorI think it would be useful for the authors to clarify the following:Reference 31 is for a commentary. Can authors should provide the reference for the original Swedish study?Can the authors provide data on the error rate for their gender assignment process?I think the authors could provide a more informative title, for example, 'Bias in peer review: the interaction between reviewer and author characteristics.'Please note, I do not have the expertise to comment on the model and statistical analysis used in this study.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Comment: The authors are very clear that their study investigates ‘bias’ as defined by the interaction between author and reviewer attributes, or bias as a function of reviewer characteristics (ref 29). They make the point that their methodology cannot determine biases that are shared by all reviewers regardless of reviewer characteristics. In the Introduction, the authors switch between describing previous research on reviewer characteristics and previous research on author characteristics. For example, paragraph 2 is predominantly about studies of reviewer characteristics, but ends on author characteristics. Paragraph 3 appears to begins on author characteristics, but then cites studies on reviewer characteristics. It would be much easier for the reader to understand what contribution this study makes to the literature if the authors made a clearer distinction between current evidence on reviewer characteristics (including those studies which looked at both reviewer and author characteristics) and other research on bias, which would include research focused only on author characteristics.This was an extremely useful suggestion. We have now reorganized our introduction to talk first about effects regarding author characteristics, then to interactions between author and reviewer characteristics and finally to the characteristics of reviewers. We believe the paper gains significantly in clarity from this reorganization.As part of the peer review process for Frontiers reviewers can, if they wish, complete the ratings questionnaire shown in Fig 1 and it was these ratings that were used in the study analysis. Do Frontiers reviewers complete the rating independently of each other before the collaborative process? If this is the case, it is not clear to me why individual scores for each paper were averaged for the Frontiers dataset, but apparently not for the other two datasets. If the aim of this study is to investigate the interaction between reviewer attributes and those of authors, wouldn’t averaging the reviewer scores in this way confound this aim?The ratings questionnaire is filled in in the initial non-interactive part of the review process. We have clarified this in the text.Also, if reviewers complete the rating independently of each other, the characteristics of peer review for the Frontiers dataset used in this study are the same as that for the other datasets, i.e. it is single blind peer review. In all three datasets the reviewers are made aware of the authors names, but the authors do not know the reviewers’ – is that correct?Yes this is correct. We have clarified our description of the Frontiers process to show that it is single-blind.In the discussion the authors state that the findings of this study could be valid for a broad range of peer review systems. However, this study did not include the interactive component of Frontiers peer review process, or if it did, it is not clear how. All three datasets appear to have used the single blind system of peer review. This statement in the discussion should be rephrasedThe differences between the datasets concern not just the way the review is organized (interactive vs. non-interactive) but also the disciplines covered, and the geographical distribution of authors and reviewers. In the text, we clarify that all three review systems in the study are single blind. However, we hypothesize that reviewers preparing for an interactive process may behave differently from reviewers in a traditional review process.Reference 31 is for a commentary. Can authors should provide the reference for the original Swedish study?As you correctly note, the reference (now reference 30) was published in the form of a commentary. De facto, however, the article represents the first publication of results from an original study, which, to our knowledge was not published elsewhere, prior to the date on which the commentary appeared.Can the authors provide data on the error rate for their gender assignment process?On the basis of random sampling, we estimate errors rates of 7.5%, 0.0% and 5.2% for the Frontiers, IEEE (Spain) and IEEE  datasets respectively. We give these figures and discuss their significance in the text.I think the authors could provide a more informative title, for example, 'Bias in peer review: the interaction between reviewer and author characteristics.'We agree with this suggestion and have revised our title accordingly. It now reads, "Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study".We have now submitted a new version of our paper, which we have revised in the light of your comments, which we found extremely useful and constructive, though they caused us significant extra work. We are particularly grateful for your suggestions concerning the organization of the introduction to the paper, which we have attempted to take on board, and for your points on quality control  deals with the fairness of the peer review process: They investigated “social biases” in the processes ofFrontiers - an open access publishing house with a novel interactive peer review process, and two peer review processes from Spanish and international computer science conferences.Peer review is the most important instrument for assessing scientific research. However, the instrument is not without drawbacks. As the most important weaknesses, a missing reliability, fairness and predictive validity have been seenet al. describe the process of normalizing authors’ and reviewers’ names. Here, Walker,et al. should ensure that the names are completely cleaned: the same author and the same institutional unit should receive the same name. A general problem of this kind of data from peer review processes is that they are not cleaned and contain several name variants.On page 5, Walker,et al.  present their results with the reporting of statistical significance information. I recommend that not only the statistical, but also the practical significance of the results (effect sizes) should be reported.2,3,4Walker,5 and Bornmann and Daniel (2010)6 investigated the peer review processes of theAngewandte Chemie – International Edition andAtmospheric Chemistry and Physics. They considered citations for the single papers as a proxy for quality. Although citations measure only one part of quality (namely impact), it is more favorable to consider them than doing not. When examining the association of bias variables and peer review outcomes without controlling quality it is impossible to establish unambiguously whether a particular group of papers receives more favorable recommendations or decisions due to these variables, or if the more favorable recommendations and decisions are simply a consequence of the papers’ scientific quality.In the Discussion section the following major limitation of the study should be mentioned: The quality of the papers was not controlled. For example, Bornmann and Daniel (2009)The study is very interesting. I recommend that the authors revise the manuscript according to the following points:I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. In our study, authors' and reviewers' names are relevant for gender assignment. Institution names are critical for assignment of language, region, and institutional prestige to author and reviewer institutions. Informal checks on our cleaning process suggest that it does not introduce substantial errors into our analysis. In the next version of our paper we will introduce more formal checks.We agree with the reviewer on this point. The next version of our paper will include estimates of the practical significance of our results. We completely agree with the reviewer that some differences in scores may be due to differences in scientific quality and tried to make this point in our text. In the next version, we will attempt to clarify this issue.I think these comments are very relevant and we will take them into account in the next version of our paper. As concerns the specific points you raise: On page 5, Walker, et al. describe the process of normalizing authors’ and reviewers’ names. Here, Walker, et al. should ensure that the names are completely cleaned: the same author and the same institutional unit should receive the same name. A general problem of this kind of data from peer review processes is that they are not cleaned and contain several name variants.Thank you this comment, which, together with comments in a similar vein from Jigisha Patel led us to conduct a thorough review of our data. The review found that errors in the normalization of author, reviewer and institution names and missing values led to down-stream errors in automated gender assignment and in the assignment of university rankings. In the case of the university rankings we were able to correct a number of errors. In the case of the gender assignment, this would have been too onerous to be possible. Instead, as suggested by Jigisha Patel, we used sampling to check the error rates in our data, which we report in our text, together with a discussion of their significance.2,3,4Walker, et al.  present their results with the reporting of statistical significance information. I recommend that not only the statistical, but also the practical significance of the results (effect sizes) should be reported.Thank you for this suggestion, which we have implemented in the revised version of our manuscript. We began by calculating Cohen's d for all our data. In the results and the discussion sections, we analyse the practical significance of the effects observed both in terms of the number of reviews concerned  and the effects on publication decisions. In the cases we study, this was not large. However, in more selective review systems it could be larger.5 and Bornmann and Daniel (2010)6 investigated the peer review processes of the Angewandte Chemie – International Edition and Atmospheric Chemistry and Physics. They considered citations for the single papers as a proxy for quality. Although citations measure only one part of quality (namely impact), it is more favourable to consider them than doing not. When examining the association of bias variables and peer review outcomes without controlling quality it is impossible to establish unambiguously whether a particular group of papers receives more favourable recommendations or decisions due to these variables, or if the more favourable recommendations and decisions are simply a consequence of the papers’ scientific qualityIn the Discussion section the following major limitation of the study should be mentioned: The quality of the papers was not controlled. For example, Bornmann and Daniel (2009)We now make explicit reference to this limitation in our text. We go on to explain that most of the papers in our datasets have very few citations. This means, that in our case, we are unable to use citations as a proxy for quality.Thank you for your comments, which we found extremely useful and constructive. Particularly useful was your suggestion that we should include effect sizes in our analysis. This suggestion has been implemented in this revised version of our paper. We would also like to thank you for your suggestions regarding quality control, complementing similar suggestions from Jigisha Patel. Thanks to these suggestions, we found a number of problems with the data, which had previously passed unnoticed. All these changes have been incorporated in the new version of the paper which we have just submitted.


We anticipate that this new alliance between The Korean Society of Animal Science and Technology and BioMed Central will elevate our journal one step higher, and facilitate easier access for those overseas as well as domestic researchers, who in turn will support us to contribute updated research outcomes in this field. We cordially invite all of you to participate and submit the results of your research so that everyone can learn and strive towards the goals that we all share. It is both a pleasure and honor to be writing this editorial on behalf of The Korean Society of Animal Science and Technology (KSAST), and to announce our partnership with the global publisher, BioMed Central, for the future publication of our journal.Documentations of hunting animals as a source of meat pre-date the evolution of man to the era of the chimpanzees . As humaAmong the numerous benefits provided by domesticated animals, the nutritional values obtained from sources with high quality protein  required to maintain our health is one of the most significant. Consumption of animal-derived products generally tends to increase with the developmental status of the country’s economy. Currently, 30 percent of the total human demand for food and agriculture comes from livestock in forms such as meat, milk, eggs, and fertilizer for crops, while also serving as an imperative cash reserve in many farming systems. This large contribution to food security comes from approximately 5,000 types of breeds comprised from 40 domestic animal species. Increasing the breeding of livestock will be a key factor in overcoming food shortages in countries where consumption exceeds amount of available food. Thus, we can expect to see changes in livestock production in the decades to come.Though there are many cases for the beneficial aspects of livestock, we should not refrain from mentioning the risks and importance of containing these risks. As animal trade has become a rewarding industry the magnitude of global movement of animals is overwhelming. Animals are legally imported for exhibitions at zoos, scientific education, conservation programs, food, tourism, etc. However, this globalization has increased the spread of infectious diseases and the risk of non-communicable diseases. Research in animal science reduces the possible spread of disease by investigating the physical health of the animals being imported or exported. Scientists suggest solutions that employ familiar tools such as post-arrival screening of animals using reliable laboratory tests, experiential treatment for known diseases, or isolation of the animals for an appropriate length of time. Despite these efforts little success has been made in dealing with diseases such as mad cow disease, the avian flu, foot and mouth disease, and others, which has had deathly consequences for animals and humans alike. Only through extensive research - utilizing the latest resources and technologies along with the collaboration of international scientists - will we contain these potential epidemics .Journal of Animal Science and Technology to initiate this transfer of information by reshaping its journal publication system into one that provides convenient availability of resources that can be shared.It is important to understand that domestic animals are no longer bound to specific regions or countries and are continuing to globally impact everyone. In this respect, nationwide collaborations and efforts to share the information obtained by researchers will play a leading role in accenting the beneficial aspects and minimizing harmful consequences. It is the goal of the Recent advents of biotechnology and its application to livestock research are rapidly shifting the classical pdigm of research in this field. For instance, genome research in farm animals has sped the understanding of functions in various organisms at the genomic level , 4. ThisJournal of Animal Science and Technology (JAST) initially launched in the year of 1956 and was renamed in 2000 to its current title. JAST is a peer-reviewed, open access, online journal that publishes original research, review articles, and notes in all fields of animal science. Topics covered by the journal range from classical livestock-oriented research to the most recent issues in animal biotechnology, animal behavior, environment, and welfare. In addition, research results on aquatic and wildlife species and laboratory animal species that address fundamental questions related to livestock and companion animal biology will also be considered for publication. JAST is thus dedicated to addressing those issues by publishing research-oriented views and results that aid in the betterment of animals as a whole.To enlighten other researchers of new information in this field, the JAST will have greater visibility that will attract new authors from around the world to publish in our journal and gather new readers. The journal will be indexed in PubMed and Scopus, and published articles are archived in PubMed Central. In an attempt to continually improve the impact and competitiveness of this journal, manuscripts will receive immediate attention upon arrival. Submitted manuscripts will be handled by a Section Editor and evaluated by at least two independent reviewers who are either members of the Editorial Board or ad hoc referees. The reviewers, experts in the area to which the manuscript pertains, will critically analyse the manuscript in a timely manner. Acceptance for publication is based on the scientific content, originality, and presentation of the material. Accepted manuscripts will be published electronically within a few weeks of their final approval date.We are certain that by associating with this prestigious publishing house, JAST, published by BioMed Central as open access manuscripts, are on cattle breeding and genetics [The first articles in volume 56 of genetics , immunolgenetics , and mongenetics . We welc


First of all some data: We published 35 articles in 2014. A good half of them (16) are research articles, almost a third are project reports (11), the remaining ones are reviews (1), position papers (3), and comments (4). These numbers illustrate that our journal lives up to its self-declared mission i.e. to promote the scientific discourse within the entire domain of education and professional development in medicine, dentistry and veterinary medicine as well as in the health professions overall. However, there is still room for improvement. From an editorial perspective we would wish to have more review articles since we consider the integration of insights from individual studies of utmost importance as these become increasingly difficult to keep track of in many domains.The first issue of a new volume, which is the 32http://www.egms.de/en/index.htm] and on PubMed where our articles are also available as full text we enjoy growing popularity (see Figure 1 That we are also on the right track for our readers is documented by the growing access of our sites: Both on our “home” platform GMS [http://www.egms.de/en/journals/zma/about.htm#editorial] that is now more international than before. We are really happy that we could win over some distinguished colleagues from different parts of the world to support us in developing our journal. Once again we would like to give an official and very warm welcome to our new editors!The most important change relates to our editorial board [http://www.egms.de/en/journals/zma/authors.htm#guide]. With regard to Boyer’s definition, scholarship embraces the discovery of new insights, the integration and synthesis of what is already known and also the application of scientific knowledge in practice . Ac. Achttp:The third innovation at the beginning of this year will hopefully turn out as a further incentive for authors to publish in our journal . For theTo reduce the time till then we recommend the article of the current issue of our journals as a very rewarding reading! And don’t forget to submit your best manuscripts to Z Med Ausbild!The authors declare that they have no competing interests.


With strong competition and many manuscripts submitted, it is difficult to convince editors and reviewers to opt for acceptance. A pragmatic approach to publishing may increase one's odds of success. Here, we – a group of postdocs in the field of plant science – present specific recommendations for early career scientists on advanced levels. We cannot provide a recipe-like set of instructions with success guaranteed, but we come from a broad background in plant science, with experience publishing in a number of journals of varying topics and impact factors. We provide tips, tricks, and tools for collaboration, journal selection, and achieving acceptance.It is trite to say “publish or perish,” yet many early career researchers are often at a loss on  Publishing is essential for scientists to disseminate their research and communicate insights to as broad of an audience as possible. For early career scientists, in particular, publication record is the fundamental metric for judging their career advancement . These suggestions are based on the experiences of the ten workshop participants, the instructor, and the literature.Here, we present our pragmatic tips for advanced publication strategies. The recommendations were developed on September 28, 2015 at a workshop for PLANT FELLOWS, an international postdoc fellowship program in the field of plant sciences . Look into your local grant opportunities that promote collaborative efforts.Put yourself out there. Do publicity for your previous publications to make you and your research visible to the scientific community npg, . PresentCollaborations between industry (biotechnology companies for example) and academia can come with numerous benefits for both parties including networking, recruitment possibilities, financial support, and access to “novel techniques and concepts”  cell biology, development, genetics, genomics, and physiology. Others have a more targeted audience . We have listed the top 30 journals in plant science, ranked by their SCImago Journal Rank for 2014, and indicated which sub-disciplines are covered within their scope . Refer to www.scirev.sc).Take information provided by journals about their decision and publication speed with a grain of salt. Many papers are “soft” rejected and require resubmission, making the decision time appear to be short. Consult your peers to get a more realistic estimate of start to finish , the review process can be unusually long. If the decision is taking longer than promised at the time of submission, do not hesitate to send a friendly inquiry email to the managing editor.Open access (OA) journals have been on the rise in the past decade  and the Frontiers in series cover a broad range of topics and publish high quality research, including plant science. Frontiers in Plant Science or BMC Plant Biology are examples of plant-specific journals from these publishers. In addition to being OA, these journals focus on the scientific soundness of the article, rather than the potential impact. Thus, their acceptance rate is generally higher than that of traditional journals and it may be easier to get your paper accepted . Thus, it's possible to take advantage of OA benefits while at the same time publishing in a highly-regarded journal.With the rise of OA journals in recent years, the line between OA and traditional journals has become blurred. Many traditional subscription-based journals offer OA options for individual articles .Be aware of OA publishing outlets that have no adequate peer review system for quality control. There are a few warning signs which can help recognize a predatory publisher. For example, unsolicited spam emails to request submissions, an extremely fast decision to publish, and not receiving the final peer reviewer comments are big red flags , bioRxiv (http://biorxiv.org), and PeerJ Preprints (https://peerj.com)  for a searchable database of preprint policies by journal.Preprints are scientific articles that are not yet peer-reviewed, but published on an online database such as ArXiv  check it before sending it off for expert review. Between two to four experts review the paper and send back comments which are compiled by the editor(s) and used to make a decision on whether to publish, send back for revision, or reject. Be prepared for discrepancies between reviewers opinions and for diverging suggestions for manuscript improvement.“You are right, here's how we addressed the criticism…”“This is a misunderstanding because things were unclearly formulated. We have clarified…”“This is a legitimate/pertinent concern in general, but in this specific case…”Generally, an effective strategy is to go point by point through the reviewer comments and either make the suggested changes or politely explain and clarify the misunderstandings. If there is a legitimate disagreement between you and the reviewer, back up your point of view with other research from the literature. Typical ways to address the reviewer concerns include:It is important to realize that the decision lies ultimately with the editor. If you feel like the reviewers (or the editor) were unjust or misunderstood your work, make your case to the editor.When you have spent years creating something it is extremely difficult not to take peer criticism personally (Annesley, Suggestions and questions by reviewers represent those that may be asked by your general audience and, once addressed, should increase the value of your work. Systematically address each of the reviewer concerns in a polite manner.Do not forget the random component. Due to unforeseen/subjective factors, you still might need a bit of luck to get accepted.The foundation for getting your work published is by first conducting high-quality research. But even after this, getting published is a complex issue. The publishing landscape is constantly changing, so be willing to adapt your strategy. Even though many factors are out of the control of authors, various approaches are possible and it makes sense to make conscious and well-informed moves. Collaborating with researchers within as well as outside your field, sending your research article to an appropriate journal, and adequately highlighting the novelty and impact of your research can help pave the way to scientific recognition. The demanding task of implementing these recommendations in your own publishing strategy is left to you. Good luck.NG, IA, GG, LG, RG, KK, PL, KR, KS, WY, and PM all contributed to the brainstorming, writing, and critical review of this manuscript. NG and PM edited the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Nature Communications now have the option to choose double-blind peer reviewAuthors at  Nature Communications is an open journal and our articles can be read online by all. Our aim is to publish high-quality research across all of the natural sciences, reporting discoveries that are important to specialists within their respective research fields. We welcome scientific submissions from anyone, and we aim to select the papers that we publish based on the significance of the science presented without any bias towards an author's ethnicity, nationality, gender, number of prior publications or any other factor.For those manuscripts passing our editorial screening process, peer reviewers are involved in assessing papers against editorial and scientific criteria. Although we believe our reviewers judge all manuscripts individually based on their merits, we recognize that as humans there is a potential for subconscious bias despite striving to reach an objective decision. Our editorial processes, including involving experts, can help to reduce such problems, yet no system can ever be perfect.A widely discussed initiative to limit further bias in the manuscript selection process is double-blind peer review. Our existing peer review system is ‘single-blind', in that authors are unaware of the identity of the reviewers assessing their manuscript. In double-blind peer review, the reviewers are also unable to see the identity of authors, thus removing potential or perceived bias from reviewers against individual authors.Nature-branded sister journals, Nature Communications is very pleased to offer our authors a double-blind peer review option. We believe that double-blind peer review represents an exciting option for our authors, and welcome submissions from anyone interested in selecting this option.Following on from our http://www.nature.com/authors/double-blind-checklist.pdf.During double-blind peer review the identity of authors is known to the editors, and some might call for a ‘triple-blind' system to eliminate the potential for editorial bias. This would not only be difficult to implement, it would also come at the cost of preventing direct discussions between editors and the authors they serve. However, most of our submissions are assessed by more than one editor, and decisions after double-blind peer review are, of course, made on the basis of the reports of reviewers who are not aware of the authors' identities. A checklist that might be useful to ensure that author identity is not readily apparent to reviewers can be found at Double-blind peer review is not a complete solution to potential bias. For example, experts in a field serving as peer reviewers might be able to guess the identity of authors based on the science contained in the paper, or from what they know about the work from conference talks or posters. We already ask our reviewers to disclose any conflicts of interest when we approach them to review a paper, or if this becomes apparent during the review process. This requirement will remain for manuscripts submitted for double-blind peer review.Nature-branded journals, we very much welcome your input on this initiative.Double-blind peer review is only one measure to enhance the objectivity of the publication process. And it will not be the last. We will continue to assess further ways to improve our publication process and to provide the best possible service to our authors as well as to our readers. As with the other


It has been too long since we have provided an editorial overview – we hope you have not missed us. However, let us catch you up on the journal and our continued focus on providing an outlet for research in ecology and evolution.First, we can celebrate the growth and vigor of our journal. Our partnership with society‐based  and Wiley journals remains strong, and we continue to receive excellent manuscripts transferred with reviews from our partners. Space limitations and the weight of submissions mean that all of our partners receive many more manuscripts than they can publish, and most of these will eventually find an outlet. The rigorous review process orchestrated by our partners improves all of the papers submitted to their journals, and if a paper is rejected, the review may have been for naught. By offering to transfer a paper with these reviews, our partners can provide yet another service to the authors that submit their work to them, the opportunity to be considered for publication by us  without the potential double jeopardy of a second round of reviews. Moreover, it helps the reviewer by ensuring that her/his work has an influence. Thus, transferring papers with reviews benefits authors and reviewers and helps reduce the growing burden on the reviewing community.The option to transfer a paper is just that – an option. The decision to cascade their work resides solely with the authors. Authors often choose to pursue publication elsewhere, but many take the option to transfer. Transferred papers have been carefully reviewed. Indeed, often the papers come to us having already been reviewed twice (once after revision) at the original journal. Our goal was to treat these papers with respect for the authors and referees, both of whom have invested time, and as rapidly as possible. After all, none that are transferred were considered flawed . The most common reason for rejection we see in papers that are transferred is “lacking in novelty,” which is an opinion (rarely shared by the authors) and readily fixed. Our goal was to provide a high profile, open, outlet that puts your research into the hands of the community as quickly as possible.What, then, do we do with papers that are transferred? We are guided by our philosophy of being “author friendly”. In its simplest form, this philosophy means that we approach every manuscript asking the question of “let us find a way to publish these data and ideas” rather than “what is wrong with this paper?” This does not mean we accept whatever you write, and we (the editors) try to work with the authors to ensure that they are presenting their work with a good story and a reasonable tone and approach. Nevertheless, “overinterpreted” can be in the eyes of the reader and when in doubt we are happy to let the community decide on the fate of a paper.This has led to an interesting phenomenon: The direct submissions to Ecology and Evolution are rapidly increasing. While we are delighted by the positive response of the community to the journal, and by the diversity of papers we receive that strengthens Ecology and Evolution as a research outlet for everyone, this presents us with a dilemma. Where we strive for a one‐week turnaround with transferred manuscripts, direct submissions  end up allocated to the workflow any “normal” manuscript goes through at any journal. We retain our author‐centered philosophy here and certainly try to provide a rapid decision, but the paper has to be assigned to an associate editor, reviewers found, reviewers agree, and reviews returned before a decision can be made. Thus, for direct submissions Ecology and Evolution is acting as a traditional open‐access journal.The full range of decisions occurs for direct submissions, from “reject” to “accept with minor revisions” (has anyone in history ever received “this is perfect” on a first submission? No? We have not either). We suspect we are no faster than any other journal with direct submissions, given many of the steps are out of our control. Moreover, the rapid growth creates teething pains for figuring out who should handle manuscripts, and we apologize if yours has been caught clogged in the system for too long. We are investigating how we might become more efficient because we too worry that the process just should not take so long. The good news, once a paper is accepted it will appear online very rapidly . But whatever we decide, we will continue to promote our “author‐friendly” philosophy. After all, we too are active researchers and authors, and we also check daily on the status of our manuscripts submitted to other journals.This brings us to the second topic and the reason for the title. We are an open‐access journal, an online‐only publication, and this brings authors and publishers a lot of advantages. Not only will we (through Wiley) continue to investigate ways that online publishing can enhance your work and facilitate reaching your audience, there are many areas you can use today. Color is encouraged in your graphs – the use of black and white really reflects the approach to publishing from the last century. Include photographs , color graphics, maps, and contours with color – whatever enhances your ability to communicate your message. Be creative.There is another advantage to online and open publication. We really cannot fathom why supplementary material continues in its present form. We already promote data sharing through open archives and are actively discouraging supplementary material. Regardless of the approach of other journals, unless you have a video, or perhaps a huge dataset , there should not be supplementary material in an online article. Supp Mat, as it is not fondly known , was invented by glossies that had too few pages to actually provide accurate information. Okay, that may be harsh and inaccurate and is just an opinion. But why in the world would you bury any information that is helpful or even necessary to understand your work? We think such information is better placed in an appendix that forms part of the paper. There is no overhead here, as there are no page limits with an online‐only journal. This would make sure that ALL of the information needed to understand your work is provided in a single download. If you have additional helpful figures, put them in an appendix. If you have detailed methods, or code, or tables with the original sources for a meta‐analysis, and appendix is fine. Use the flexibility of online publishing to be accurate and complete.Supplementary material is currently where data and methods go to die, never to be viewed again. We are author friendly. We also strive to be reader friendly. Our readers are busy, and while it may seem trivial to download two files instead of one, how do you ensure that the two remain linked in whatever folder in which you downloaded files? If you are like us, two months later you are staring at an unhelpfully labeled “smith_et_al_supp_mat” file wondering just what might be in it and which Smith this might be and why we thought her paper so memorable. So, whatever the rationale or rules at other journals, let us be open and share our work. Make use of appendices instead of sup mat. Create a single file. Use visuals where possible. Upload your data to GenBank or Dryad or wherever you make it freely available. Be reader friendly. We will continue to work to produce a journal that is author friendly as well.


Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle. The Ame[T]he act of publishing is a quid pro quo in which authors receive credit and acknowledgment in exchange for disclosure of their scientific findings. An author's obligation is not only to release data and materials to enable others to verify or replicate published findings  but also to provide them in a form on which other scientists can build with further research. All members of the scientific community—whether working in academia, government, or a commercial enterprise—have equal responsibility for upholding community standards as participants in the publication system, and all should be equally able to derive benefits from it.The Committee on Responsibilities of Authorship in the Biological Sciences, National Research Council , p. 4 plFor many decades, openness was frustrated by technological limitations. Physical, paper journals simply could not accommodate a more complete record of the methods and data presented in scientific articles. With the advent of digital information technologies and the Internet, these limitations have disappeared. Instead of being stored in dusty boxes in a laboratory, records of one's materials, data, and analyses can be made publicly available for anyone in the world to access indefinitely. For most research—particularly research in the behavioral and social sciences—it is possible to provide a nearly complete public record of the research from consent form templates, surveys, stimuli and experimental instructions and software, to the data and analysis scripts. Over the past few decades, valuable tools have been developed to facilitate open research: data journals (e.g. ) and datThe possibilities allowed by new technological developments are exciting. Scientific progress is accelerated as more data are available for verification, theory-building and meta-analysis, and experimental materials are available for easier replications and extension studies. The availability of stimuli, questionnaires, programmed paradigms and data analysis scripts will make scientific research more efficient, as research can be more easily understood and replicated, mistakes can be found more readily, and needless duplication of effort across laboratories can be reduced. With open research, scientific articles become part of a collaborative effort to learn what is likely to be true, rather than static records of the results of specific analyses. The meaning of a scientific article is not limited to the interpretation that the original authors give it; with open data, the interpretation of experimental data can evolve as theories develop, or as researchers with varying perspectives analyse them. The open scientific future we glimpse on the horizon holds great promise.And yet, despite the fact that openness in research is a core scientific value—and despite the fact that nearly complete openness in research has never been easier to achieve—open research practices are rare in many parts of the scientific literature. For decades, authors have found low rates of data sharing, across many scientific disciplines –15. We kAlthough openness is an ethical obligation of scientists that carries many demonstrable benefits, and despite it being easier than ever before due to the availability software and Internet resources, many areas of research are still largely not open. The current approach to advocating openness—a mixture of extolling the benefits of openness to researchers and pushing for policy change at journals and granting agencies—has proved to be insufficient to motivate the adoption of open research practices.Perhaps the lack of success of this approach is no surprise. The way that the incentives in science are structured, the ability to publish is rewarded above all else. Learning open practices is not difficult, but implementing them does take time and may delay the publication process, even if only for a few days. Opening one's own data and methods could also be perceived as benefiting one's competitors with no guarantee of reciprocal openness, possibly placing one at a disadvantage within one's scientific field. This creates a social dilemma: individuals might value open research practices as being in the best interest of the scientific community, yet simultaneously avoid them because they do not believe that implementing open practices is in their individual best interest. Given this social dilemma, the current slow pace of change is perhaps not surprising.We believe many researchers are excited by the promise of open scientific practice, and surveys have shown their agreement with open principles ,1. Resea2.http://opennessinitiative.org) is a statement that researchers can sign that indicates that after a set, future date—1 January 2017—they will begin to apply certain minimal open research standards to the manuscripts they review. Central to the Initiative is the idea that reviewers can engage authors on issues of scientific openness during the review process.Although some progress has been made towards open practices, actual behaviour in many scientific fields is far from the ideal of openness by default. We suggest a new strategy: a concrete statement of open-research requirements for publication, along with a plan for putting these into action. The Peer Reviewers' Openness Initiative . EngageThe authors can respond in one of two ways to such a request: first, they can provide a link to the data and materials and agree to share a link in the published version of the manuscript. However, as we discuss later, there are valid reasons for incomplete openness. In the case that the authors will not open aspects of their research, the authors can respond with a justification for this lack of openness, and include their justification in the manuscript's author note so that reviewers and future readers can understand the Authors' reasons for not releasing some part of the data or materials. If a link to the data and materials or a justification for lack of openness is provided, and if these will be public, then the review process can continue as normal. If the authors will neither release their data and materials nor publicly justify their choice not to, then the reviewer should offer a short review that focuses only on the lack of openness and failure to justify it. We believe there is simply no argument for authors not justifying their data and material sharing practices; knowing why authors choose not to be open would be tremendously useful to the scientific community at large.2.2Data should be made publicly available. All data needed for evaluation and reproduction of the published research should be made publicly available, online, hosted by a reliable third party.— Stimuli and materials should be made publicly available. Stimulus materials, experimental instructions and programmes, survey questions and other similar materials should be made publicly available, hosted by a reliable third party.— In case some data or materials are not open, clear reasons  should be given. These reasons should be outlined in the manuscript.— Documents containing details for interpreting any files or code, and how to compile and run any software programs should be made available with the above items. In addition, licensing or other restrictions on their use should be made clear.— The location of all of these files should be advertised in the manuscript, and all files should be hosted by a reliable third party. The choice of online file hosting should be made to maximize the probability that the files will be accessible for many years, and to minimize the probability that they will be lost for trivial reasons .— Openness and transparency are core values of science. As a manifestation of those values, a minimum requirement for publication of any scientific results must be the public submission of materials used in generating those results. As reviewers, it is our responsibility to ensure that publications meet certain minimum quality standards. We therefore agree that as reviewers, starting 1 January 2017, we will not offer comprehensive review for, nor recommend the publication of, any manuscript that does not meet the following minimum requirements. Once such a manuscript has been certified by the authors to meet these minimum requirements, we will proceed with a more comprehensive review of the manuscript.Authors of submitted articles wishing to signal to reviewers and readers their adherence should add a statement to the author note specifying the data and materials that are open and where they can be found, or their justification for not sharing some of their data and materials.3.et al. )One thing that senior researchers can do to help students is to ensure that their data and research materials are well curated. In particular, having a laboratory-wide plan based on good research management practices—and making sure students understand it—has the immediate benefit that others in the same laboratory easily make use of the materials after the student has left, and the side benefit that openness will be a simple matter of making the already well-curated materials publicly available.Authors may be unable to share (some of their) data or materials. It is our goal to create the strong expectation that research practices be open. In some cases, it may not be possible for data or materials to be shared for a variety of reasons, such as legal or ethical considerations. These cases can be dealt with on an individual basis, and the reasons for exceptions to open research practices should be clearly explained. Certainly, if anyone understands the constraints on openness that researchers face in a particular field, it will be the peer reviewers; this is a major point in favour of peer reviewers taking the lead on this issue.2 Other problematic cases include issues of copyright: if scientists are using materials that are copyrighted by another organization, they may not have permission to release them publicly. The common thread in all examples of limited openness is that they should be thoughtful, justifiable in principle, and actually justified to readers and reviewers. For further discussion of the limits of openness, see [The Royal Society Science Policy Centre  notes thess, see , ch. 3.Finally, some authors simply may not wish to share their materials publicly before acceptance of their paper. In such cases, some data repositories (e.g. Data Dryad and figshare) allow the embargoing of shared materials until publication, while still privately sharing the materials with reviewers. Such solutions remove from the authors the burden of remembering to share the materials after publication.As a reviewer, I do not have time to check Authors' analyses. An important aspect of the Initiative is that it is not designed, by itself, as a fraud or mistake detection mechanism. It is designed, rather, to increase open practices. By joining the Initiative, you are not obligating yourself to check the analyses using the data provided. Reviewing remains as it always has been, except that you would have the option to check the analyses, if you chose to do so. We expect that once providing open data and materials becomes more common, norms will be established regarding how reviewers are expected to use these new resources, but until then reviewers should use their own judgment.I am concerned that the Initiative may be used aggressively. The purpose of the Initiative is twofold: first, to alert reviewers to the fact that open practices are a legitimate target for peer review, and second, to give authors the incentive they need to begin implementing open research practices. The Initiative is thus intended entirely constructively. In some rare cases—for instance, in contentious areas of research—over-zealous application of the Initiative's standards might be used as a weapon by one's theoretical rivals. This would be undesirable, possibly causing researchers to resent requests for more openness instead of seeing them as opportunity to improve their practices. If any dispute arises over whether the points of the Initiative have been met, reviewers are encouraged to cede to the opinion of the authors and action editors.Reviewers should assume good faith on the part of authors who have certified that they meet the points of the Initiative, and avoid aggressive, legalistic interpretations of the Initiative and making unreasonable demands. Signers of the Initiative should remember that the minimal requirement of the Initiative is mere public justification of lack of openness; even if one believes a particular justification is insufficient, if the justification is public it will become part of the broader conversation about openness among scientists. This broader conversation is substantially more valuable than openness in any single paper.How do I apply the terms as an action editor? The application of the terms of the Initiative may complicate action editing if the Initiative is seen as too radical by journal senior editors. For this reason, the Initiative is targeted at reviewers and not action editors. Signatories of the Initiative are, of course, free to make reference to the Initiative as action editors, but signing does not obligate them to do so. As a signatory of the Initiative, your role as an action editor and your role as a reviewer are seen as separate. There is one important role that action editors have as a part of their typical responsibilities: judging whether authors have made a good-faith effort to meet a reviewer's request for open practices. As noted above, we believe that reviewers should defer to Authors' and action editors' opinions about whether an article has met the requirements of the Initiative. One of the main responsibilities in action editing is weighing the concerns of reviewers and authors. We expect that action editors will not find juggling demands for open research to be any more problematic than juggling other reviewer demands.5.The Peer Reviewers' Openness Initiative provides the best possible chance for open research practices to become commonplace in the scientific literature. Now is the time for all of the sciences to be open; we hope that researchers who value open research practices—even ones who have not yet put them into action—will join us in signing the Initiative and help promote open research. We must let current and future researchers know that openness is not just a value we talk about, but one we make manifest.