Unequal Covariance Awareness for Fisher Discriminant Analysis and Its Variants in Classification
Fisher Discriminant Analysis (FDA) is one of the essential tools for feature
extraction and classification. In addition, it motivates the development of
many improved techniques based on the FDA to adapt to different problems or
data types. However, none of these approaches make use of the fact that the
assumption of equal covariance matrices in FDA is usually not satisfied in
practical situations. Therefore, we propose a novel classification rule for the
FDA that accounts for this fact, mitigating the effect of unequal covariance
matrices in the FDA. Furthermore, since we only modify the classification rule,
the same can be applied to many FDA variants, improving these algorithms
further. Theoretical analysis reveals that the new classification rule allows
the implicit use of the class covariance matrices while increasing the number
of parameters to be estimated by a small amount compared to going from FDA to
Quadratic Discriminant Analysis. We illustrate our idea via experiments, which
show the superior performance of the modified algorithms based on our new
classification rule compared to the original ones.