1. A data analyzer for use with a pattern classifier to compress a set of indexed data having common characteristics and noise, comprising: a. means for determining a common characteristic threshold for the indexed data set; b. means for removing indices having an ensemble statistic higher than the common characteristic threshold value in order to provide a retained dataset, wherein the ensemble statistic is a statistic taken from across a set of spectra; c. means for calculating the ensemble statistic of each retained index in the retained dataset; d. means for determining a noise threshold; e. means for removing indices from the retained dataset wherein the ensemble statistic is lower than a noise threshold value; and f. means for normalizing the indexed data.