Testing for differences between impact of attributes in penalty lift analysis
Abstract Penalty lift analysis is a useful tool to identify so called drivers of product liking or other hedonic measures from check all that apply (CATA) or similar datasets that include some hedonic measures along with them. A recurrent question in recent projects is about how many attributes to consider as "top drivers" and how to define a reasonable cut off. It is straightforward to test whether any attribute's impact is significantly different from zero, but this is likely not of major interest, as at least the top attributes will almost always show a statistically significant impact. However, how can we compare the penalty lift of two different attributes on the hedonic response? To this end, two different strategies were considered. For the first one, the dataset is reduced to remove all information that does not really impact the difference in liking. An F test from a two way ANOVA is then employed to test whether there is a difference between the attributes' penalty lift. As the dependency structure in a CATA study might invalidate the test, a randomization approach was used to validate the results, showing a reasonable fit of the parametric approximation. The second strategy employs a simple re coding of the two attributes under consideration into one single variable with 4 factor levels. Using a relevant contrast from a two way ANOVA, the impact of different attributes is compared using a t test, again validated through a randomization approach. The approaches were successfully applied to amend the penalty lift analysis for a CATA study on strawberries, in which the t test proved more powerful than the F test.