Job satisfaction and opinion surveys ask respondents to provide ratings on a set of items. One response scale might be a five point rating scale measuring satisfaction (e.g., 5-point strongly dissatisfied to strongly satisfied) with the item content (e.g., Opportunities for promotion). Some tests also gather for each item importance ratings.What are the psychometric properties of importance versus satisfaction scales?To what extent is the incremental information provided by importance ratings justified?
Whether to ask satisfaction and importance questions for each item in a survey?
social psychology;io psychology;measurement;attitudes
Computing overall satisfactionThere is discussion in the literature about whether measures of overall satisfaction should be calculated as some form of weighted composite of facet satisfaction where weights are derived from importance ratings.These debates have been applied to various types of satisfaction including life, job, and customer. In this context, it does not seem like weighting by importance makes much difference, and it may even be detrimental.For examples, Trauer et al (2001) state that:Many Quality of Life (QoL) instruments ask respondents to rate a  number of life domains in terms of satisfaction and personal  importance, and derive weighted satisfaction scores by multiplying the  two rat- ings. This paper demonstrates that this practice is both  undesirable and unnecessary. QoL domains are selected on the basis of  their inherent importance, rendering separate importance rating  partially redun- dant. Weighted scores present difficulties in  interpretation. Further, we show that multiplicative composites have  undesirable psychometric properties. There is evidence that  multiplicative composites have little or no advantage over unweighted  ratings in correlational or predictive studies. Apart from the face  validity and the intuitive appeal of multiplying satisfaction ratings  by importance ratings, there appear to be no sound reasons for doing  so, and several good reasons not to do so.A similar critique of importance ratings in the job satisfaction domain can be seen in Blood (1971).Nonetheless, this all assumes that importance measurement is a means for  forming an overall composite. In some instances, measurement of importance is of substantive interest. Trauer et al (2001) acknowledge this when they write: It is not suggested that the assessment of the importance of domains  of satisfaction is inconsequential. It has been shown that domains  that are closer to the personal core (e.g. health, marriage, family)  are typically judged as more im- portant than more peripheral domains  (e.g. membership of organizations)...Study of mean importance ratingsImportance ratings can be used to assess average perceptions of importance which might inform causal hypotheses regarding what is influencing satisfaction. Take for example, Rice et al (1991) among many other things, they report importance ratings of various job facets. Among other things the authors asked 97 employed college students to ratefacet importance on a 9-point scale with two verbal  anchors: not at all important and extremely important (e.g., How  important to you is the amount of opportunity for promotion?).If we sort the results reported in Table 1 by importance we see the following:Job Facet                     Mean ImportanceHourly pay (dollars)              7.4Hours per week                    7.3Control over schedule             7.3Opportunity to learn skills       6.7Decision making                   6.4Promotion opportunity             6.3Mental effort required            6.1Conversation with co-workers      6.0Customer or client contact        5.7Physical effort required          5.4Supervisor contact                5.3Commuting time (minutes)          4.3This is presumably an interesting empirical finding in itself.Relationship between importance and satisfactionSome researchers have explicitly studied the relationship between importance and satisfaction to see whether there is a relationship. E.g., Borg (1991) provides an example of job satisfaction exploring how intra-individual variability in facet job satisfaction relates to facet importance. They find support both for the idea that more important facets are deemed more satisfying and for a u-shaped relationship where it is the most and least satisfied facets that are the most important. This variation depends on the domain being studied.ReferencesBlood, M.R. (1971). The validity of importance.. Journal of Applied Psychology, 55, 487.Borg, I. (1991). On the relationship between importance and satisfaction ratings on job aspects. Applied Psychology, 40, 81-92.Rice, R.W., Gentile, D.A. & McFarlin, D.B. (1991). Facet importance and job satisfaction.. Journal of Applied Psychology, 76, 31.Trauer, T. & Mackinnon, A. (2001). Why are we weighting? The role of importance ratings in quality of life measurement. Quality of life research, 10, 579-585.