A common pitfall in materials discovery involves the use of cv. If the goal of an approach is to predict fundamentally new materials (i.e. materials extrapolation rather than interpolation), a special "grouping" cv scheme (termed gcv in this work) may be used to ensure the model predictions are not overly optimistic. [1]} first introduced the idea of lococv or gcv and [2]} discussed the difficulty of making predictions when many mechanisms interact to cause outstanding properties. [2]} described how ml can be used for structure-composition-property-processing relationships and review successful examples of materials discovery for structural materials (fatigue, failure), high-entropy alloys, and bulk metallic glasses. For example, in the case of [4]}, all training data for the Co-V-Zr ternary were removed before making predictions in that group (hence gcv). [5]} performed cv on chemical formula groups rather than on all of the training data as a whole to make sure that cross-validated predictions were not simply interpolations between temperatures within a chemical formula group. To illustrate, the "trails" seen in the ml parity plots of fig:kauwe2018-svm-lr-rf-cac exhibiting systemic deviation from parity are likely present because of the gcv scheme. By taking a non-group cv approach, the model would likely favor temperature interpolation and mild temperature extrapolation, causing the trails to disappear at the expense of heavily overoptimistic predictive performance. We believe the question, "are my model predictions overly optimistic?", is wise to ask when pursuing true materials discovery.
<FIGURE>