The power of disagreement
Scientists looking at the same data can disagree profoundly in their interpretation of those data. The advance of scientific knowledge stands to benefit from such disagreement, and we at Nature Methods see our role as publishers not only of work that presents new advances and reconciles differences, but also of the opinions of dissenting peers. There is no simple rule for how to best handle disagreement, but a good place to start is to agree on a few definitions. For example, epistemic peers are equals in training and resources with access to the same evidence regarding a particular question. Scientific peer disagreement can be defined as having the same epistemic goal specifically, to gain knowledge with regard to a given question but disagreement on how to achieve it. Such disagreement can arise for a variety of reasons: because scientists come from different backgrounds and have different sets of prior assumptions; because different methods are used for data interpretation; or because existing evidence is often incomplete, making it impossible to draw definitive conclusions without gathering more data. A fascinating example, though admittedly a bit removed from the purview of Nature Methods, is the discussion of whether the small hominid found in a cave in the Indonesian island of Flores in 2003, dubbed Hobbit in the popular press, should be classified as Homo floresiensis, a new hominin species, or represents samples of Homo sapiens suffering from congenital disorders leading to microcephaly and dwarfism. Although the original debate was based solely on fossil evidence, the dispute has triggered research into the scaling of brain size during severe body reduction and has led to insights suggesting that the relationship is more complex than previously assumed. More than ten years after the original discovery, research into this classification question is ongoing. Much closer to home are two Perspectives on genomic footprinting in this issue. The first Perspective, on page 213, comes from Jeff Vierstra and John Stamatoyannopoulos at the University of Washington. By leveraging massively parallel sequencing technology, this group has generated large data sets of DNase I fragments that constitute a fraction of chromatin accessible to binding by regulatory factors. Their research team was the first to systematically derive genome wide information on transcription factor (TF) binding from such data, in a process termed digital genomic footprinting. Genome wide data on TF occupancy are critical for deriving the architecture of individual regulatory regions as well as for understanding the networks that regulate transcription. The authors describe the key components of the technique, discuss pitfalls and mention the need for improvement, but overall they stress the potential of the method for providing insight into complex TF networks. The second Perspective is written by Myong Hee Sung, Songjoon Baek and Gordon Hager, researchers at the US National Cancer Institute who study nuclear receptors to understand how chromatin reorganization regulates gene expression. They point to limitations of genomic footprinting and caution that, because of the short residency time of many TFs on DNA, some footprints are indistinguishable from DNase cutting bias; they argue that those data should thus not be used to derive transcriptional networks. Although these pieces come to contradicting conclusions on the reliability of genomic footprinting, an open discussion of this disagreement is valuable in several respects. It underscores the importance of careful attention to best practices to ensure that interpretive disagreement is not due to inferior data sets, that necessary controls are included and that computational programs best suited for a particular analysis are used. Disagreements also spur the generation of new evidence and the development of alternative approaches that prove or disprove current data interpretation and go beyond present limitations. For example, different techniques to isolate chromatin accessible to binding by regulatory factors, paired with alternative sequencing techniques, may shed more light on the binding preferences of certain TFs. Alternatively, refined computational tools may be better able to detect footprints. In a Nature Methods Analysis currently available online, Costa and colleagues compare ten methods for footprint analysis and provide guidance as to the strengths and weaknesses of each. The authors also point to the need for further improvements to discern footprints from TFs with short residency times. Dissenting opinions can bring to light confirmation bias and prompt researchers to take a second look at evidence that is not in agreement with their hypothesis, rather than dismiss it as artifacts. As researchers working in cognitive science have observed, we tend to provide better arguments when we make a case against an opponent than when we present an unopposed finding (Behav. Brain Sci. 34, 57 111, 2011) In dialogue with those who hold opposing views, scientists have the opportunity to reexamine their hypotheses and the evidence, find new methods, and put forward a convincing case that moves us one step closer to answering puzzling biological questions. The power of disagreement
Nature Methods