Considerations in the Interpretation of Cosmological Anomalies
  Anomalies drive scientific discovery -- they are associated with the cutting
edge of the research frontier, and thus typically exploit data in the low
signal-to-noise regime. In astronomy, the prevalence of systematics --- both
"known unknowns" and "unknown unknowns" --- combined with increasingly large
datasets, the widespread use of ad hoc estimators for anomaly detection, and
the "look-elsewhere" effect, can lead to spurious false detections. In this
informal note, I argue that anomaly detection leading to discoveries of new
physics requires a combination of physical understanding, careful experimental
design to avoid confirmation bias, and self-consistent statistical methods.
These points are illustrated with several concrete examples from cosmology.
