Update on the Systematics in the ALMA Proposal Review Process after
  Cycle 8
  We present an updated analysis of systematics in the Atacama Large
Millimeter/submillimeter Array (ALMA) proposal ranks from Carpenter (2020) to
include the last two ALMA cycles, when significant changes were introduced in
the proposal review process. In Cycle 7, the investigator list on the proposal
cover sheet was randomized such that the reviewers were aware of the overall
proposal team but did not know the identity of the principal investigator (PI).
In Cycle 8, ALMA adopted distributed peer review for most proposals and
implemented dual-anonymous review for all proposals, in which the identity of
the proposal team was not revealed to the reviewers. The most significant
change in the systematics in Cycles 7 and 8 compared to previous cycles is
related to the experience of PIs in submitting ALMA proposals. PIs that submit
a proposal every cycle tend to have ranks that are consistent with average in
Cycles 7 and 8 whereas previously they had the best overall ranks. Also, PIs
who submitted a proposal for the second time show improved ranks over previous
cycles. These results suggest some biases related to the relative prominence of
the PI have been present in the ALMA review process. Systematics related to
regional affiliation remain largely unchanged in that PIs from Chile, East
Asia, and non-ALMA regions tend to have poorer overall ranks than PIs from
Europe and North America. The systematics of how one region ranks proposals
from another region are also investigated. No significant differences in the
overall ranks based on gender of the PI are observed.
