Online surveys have the potential to support adaptive questions, where later questions depend on earlier responses. Past work has taken a rule-based approach, uniformly across all respondents. We envision a richer interpretation of adaptive questions, which we call dynamic question ordering (DQO), where question order is personalized. Such an approach could increase engagement, and therefore response rate, as well as imputation quality. We present a DQO framework to improve survey completion and imputation. In the general survey-taking setting, we want to maximize survey completion, and so we focus on ordering questions to engage the respondent and collect hopefully all information, or at least the information that most characterizes the respondent, for accurate imputations. In another scenario, our goal is to provide a personalized prediction. Since it is possible to give reasonable predictions with only a subset of questions, we are not concerned with motivating users to answer all questions. Instead, we want to order questions to get information that reduces prediction uncertainty, while not being too burdensome. We illustrate this framework with an example of providing energy estimates to prospective tenants. We also discuss DQO for national surveys and consider connections between our statistics-based question-ordering approach and cognitive survey methodology.
Survey researchers avoid using large multi-item scales to measure latent traits due to both the financial costs and the risk of driving up non-response rates. Typically, investigators select a subset of available scale items rather than asking the full battery. Reduced batteries, however, can sharply reduce measurement precision and introduce bias. In this article, we present computerized adaptive testing (CAT) as a method for minimizing the number of questions each respondent must answer while preserving measurement accuracy and precision. CAT algorithms respond to individuals’ previous answers to select subsequent questions that most efficiently reveal respondents’ position on a latent dimension. We introduce the basic stages of a CAT algorithm and present the details for one approach to item-selection appropriate for public opinion research. We then demonstrate the advantages of CAT via simulation and empirically comparing dynamic and static measures of political knowledge. In this review we explore recent developments in computerized adaptive diagnostic screening and computerized adaptive testing for the presence and severity of mental health disorders such as depression, anxiety, and mania. The statistical methodology is unique in that it is based on multidimensional item response theory (severity) and random forests (diagnosis) instead of traditional mental health measurement based on classical test theory (a simple total score) or unidimensional item response theory. We show that the information contained in large item banks consisting of hundreds of symptom items can be efficiently calibrated using multidimensional item response theory, and the information contained in these large item banks can be precisely extracted using adaptive administration of a small set of items for each individual. In terms of diagnosis, computerized adaptive diagnostic screening can accurately track an hour-long face-to-face clinician diagnostic interview for major depressive disorder (as an example) in less than a minute using an average of four questions with unprecedented high sensitivity and specificity. Directions for future research and applications are discussed.