Maximising the impact of academic research
How to cite our blogs
Our blog family
The Impact of the Social Sciences
The Accelerated Academy
#AcWri2016
Data and Society series
HEFCEmetrics
Politics of Data
The Handbook on Maximising Impact
Open Access eCollection
Engaged Social Science: Impacts and Use of Research in the UK (January 2014)
Open Access Futures in the Humanities and Social Sciences (October 2013)
Social Science in the Public Sphere (July 2013)
Future of Academic Impacts (December 2012)
Academic Podcasts
Audible Impact
Simple Guide to Academic Podcasting
Twitter Guide
Essential 'how-to' Guides
Gender bias in peer review – opening up the black box
Gender bias in peer review has been much discussed in the wider research community. However, there have been few attempts to analyse the issue within the social sciences. To celebrate International Women's Day (IWD), this post by Alex Holmes and Sally Hardy highlights research undertaken by the Regional Studies Association to investigate the effect of gender on peer review outcomes. Taking a close look at the data, they reflect on how other factors may contribute to producing particular patterns in peer review decisions and suggest how undertaking this research has been critical to developing an equality agenda within the Regional Studies Association.
International Women's Day, happening this year on March 8th, is a day to celebrate the social, cultural, economic and political achievements of women across the world. Instigated in 1911, it is an international opportunity for people to unite behind efforts towards gender equality. This blog is our contribution towards the collective efforts of IWD's call to action for gender parity and is part one of two blogs focusing on the Regional Studies Association project to examine gender in relation to peer review practices. Gender imbalance in academic publishing is a documented phenomenon especially in the sciences, (although there is less evidence for the social sciences); with much speculation and no consensus for the unevenness in gender representation. We undertook this research to identify whether there was any systematic bias in our journal processes.
The Regional Studies Association has been committed to diversity and inclusivity since its formation over 50 years ago. As an international and multi-disciplinary learned society, with members coming from academia, policy making and practice who are interested in regional issues, and especially regional inequalities, it seemed appropriate for the Association to undertake a reflective project examining diversity. Diversity issues are increasingly recognised within the academic community as needing to be both better understood and addressed, but rather than undertake a broad-based research project skimming the surface of the many different issues regarding diversity simultaneously, we narrowed our field to gender, and specifically to an examination of gender patterns in research submitted for publication to the largest of our five peer reviewed journals.
The research focussed on the journal Regional Studies which publishes monthly, using article submissions and peer review data submitted through ScholarOne between 2011 and 2016. Using genderize.io to assign gender, 83% of authors and 81% of reviewers in the deduplicated (removal of individuals with multiple entries) dataset were assigned gender.
So, what did we find out? From all the articles submitted where gender could be assigned, more males submitted manuscripts than females: 67% were submitted by males compared to 33% by females and this ratio remained unchanged for both 'all author' and 'corresponding author' definitions.
Where articles had multiple authors, 'all female' papers were significantly lower in number than 'all male' submissions, 16% as compared to 47%, although 37% had both male and female authors. Male corresponding authors submitted 68% of papers, while 32% were submitted by female authors. These proportions change slightly when we consider the papers which go on to be accepted or rejected. 27% of all accepted papers, and 35% of all rejected papers, were by female corresponding authors. Therefore, compared with the submitted papers, a slightly lower proportion of papers with female corresponding authors were accepted, and a slightly higher proportion were rejected. Out of the 100% of articles submitted by males, 31% were accepted; for women 24% of their total submissions were accepted. Not exactly equal, but not as demoralising as inferred from our first glance of the data.
How does this all help our understanding of academic publishing? We are acutely aware that our analysis of author submissions to a single, albeit multidisciplinary and international journal, provides only a snapshot of historical publishing patterns which cannot, and should not, be extrapolated into cross-disciplinary generalisations. Our provisional inferences from the results are little more than hunches based on our knowledge about the regional studies community, specifically the Association membership, global conference attendances and the authorship of articles submitted to and published in, Regional Studies. We might speculate that some influencing factors could be that there is a higher proportion of female contributors in their early career and who therefore may not yet have attained the research and writing maturity to succeed in the peer review process. Our knowledge of the field also suggests that it is possible that more of these females may be from emerging economies and therefore less likely to have been trained in a Euro-American research tradition, which could also disadvantage them in the review process. Connected to this, having English as an additional language may also influence peer review outcomes.
A rough sampling of the Association's membership base reveals a gender skew towards males, with roughly 35% females across the various fields of geography, economics, planning, political science, management and business studies. Numbers of female Association members have increased in recent years and is currently heavily weighted towards the PhD student and early career researcher membership categories (female PhD and EC researchers form 21% of the total membership representing more than half of all female members). This may suggest that more males are submitting to Regional Studies because there are simply more males in the field at large.
What next? We are aware that the first sweep of the data based on our original questions has left us with many unanswered questions, some of which intersect with other issues regarding diversity: about the seniority of authors; geographic location of PhD institution; first language and whether the patterns observed change over time. We are looking forward to the next stage of the project where we hope to tease more out of our data, limited though it is. Why? We hope that our ongoing analysis will reveal the patterns of peer review in Regional Studies and contribute to our policies to ensure gender parity in our journal processes as well as more widely across amongst the RSA community.
In our next blog we will consider what our data allows us to infer about patterns of corresponding authors, a contested topic with commentators finding different practices including gender bias. We will also consider the thorny questions of both editorial conscious and unconscious bias.
We would like to thank to the research team at our publishers, Taylor & Francis, and particularly Anna Gilbert, Ross Barker, Sarah Robbie, Jon Manley, Sarah Stacey and Ryan Carson, for undertaking the mammoth data extraction and preliminary analysis, as well as contributing into the ongoing interpretations. We also gratefully acknowledge the Regional Studies editorial teams and RSA Board members over the past six years who readily agreed to make this data available to us.
Image credit: Markus Winkler via Unsplash (licensed under a CC0 1.0 license).
Note: This article represents the views of the author, and not the position of the LSE Impact Blog, nor of the London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
Alex Holmes is the Communications and Membership Manager at the Regional Studies Association. Prior to this she was a physics teacher for many years, working to engage more girls in STEM-related careers. Her PhD examined educational factors, including issues of gender, contributing to perceived declines in STEM graduates.
Sally Hardy is the Chief Executive of the Regional Studies Association. She has a longstanding interest in scholarly communications and in particular in how researchers write for and publish in journals. She has lectured globally on many related topics. Sally is leading the Association's work on diversity and inclusivity and this piece of research forms a part of that activity.
March 8th, 2019|Academic publishing, Gender bias, Peer review, Research ethics, Women in Academia|5 Comments
Albert March 8, 2019 at 11:13 am - Reply
You do know that when you use such highly abstract and complex activity such as academic publishing, cutting it down into male/female dichotomy kinda misses the point. Just as with the gender paygap, there are a slew of different factors that could explain your results other than bias. Furthermore, what's your rationale for assuming that there SHOULD be a 50/50 gender split?
Gavin Moodie March 8, 2019 at 2:14 pm - Reply
Presumably the reviews were blind and therefore did not take gender into account directly, but that presumably was not true for editors and therefore editors' decisions on whether to send submissions for review, to encourage authors who were asked to revise and resubmit, and to decide difficult cases.
Katie March 8, 2019 at 8:33 pm - Reply
In Economics, at least, multiple indicators make it easy to identify authors, even if referees are technically 'blind,' including prior knowledge of who is working on what topics, where, and using which methodologies. Working papers are presented often and available on a number of platforms, especially as preprints become more popular. I would wager that most referees can tell who authored a paper they are asked to review, and blinding is not especially effective in that regard.
Klavs Hansen March 9, 2019 at 3:52 pm - Reply
It seems that bias is one explanation. It is easy to come up with ten other candidates.
Btw, International Women's Day happens every year on March 8, by definition.
Gender bias in peer review – Opening up the black box II | Impact of Social Sciences March 26, 2019 at 4:06 pm - Reply
[…] their previous post, Alex Holmes and Sally Hardy examined the results of research undertaken by the Regional Studies […]
This work is licensed under a Creative Commons Attribution 3.0 Unported License unless otherwise stated.
Visit our sister blog LSE Review of Books
Visit our sister blog USA Politics and Policy
Visit our sister blog British Politics and Policy
Visit our sister blog European Politics and Policy
Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a Creative Commons Attribution Unported 3.0 License.
This work by LSE Impact of Social Sciences blog is licensed under a Creative Commons Attribution 3.0 Unported.