Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> Summary and Conclusions Use of the World Wide Web to conduct surveys provides enormous opportunities as well aschallenges. The cost advantages of collecting large amounts of data at very little cost means thatits use will escalate rapidly. At the same time, maintaining quality, whereby the results ofsurveys can be generalized to a large population is far from automatic.Respondent-friendly questionnaire design, found important to improving response to self-administered mail questionnaires, is also important for the development of web questionnaires.However, in a web context the term takes on a broader meaning. A respondent-friendly webquestionnaire is one that interfaces effectively with the wide variety of computers and browserspossessed by respondents. It also makes other aspects of the response task easy and interestingfor the respondent to complete. Respondent-friendly designed is aimed explicitly at reducingthree of the four types of error that typically prevent accurate surveys from being done, i.e.,nonresponse, measurement, and coverage of the survey population.Three important criteria for achieving respondent-friendly design have been stated. Theyinclude, first, compliancy with technology available to the respondent. If a browser cannotrecognize the programmed information being sent to it, the respondent cannot receive it.Questionnaires that use advanced programming that takes longer transmission times and requiresmore advanced browsers may result in unintended coverage and nonresponse error. A secondcriterion is the necessity of bridging between the logic by which respondents expect a question-naire to operate and the logic associated with operating a computer. It’s this connection thatprovides a questionnaire design challenge not previously faced by survey methodologists. Inaddition to these criteria, web designers must also consider the high likelihood that coverageproblems will require that most quality web surveys, for the foreseeable future, will require themixing of survey modes, whereby some respondents are surveyed by mail or telephone whileothers are surveyed via the web. <s> BIB001 </s> Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> As we enter the twenty-first century, the Internet is having a profound effect on the survey research industry, as it is on almost every area of human enterprise. The rapid development of surveys on the World Wide Web (WWW) is leading some to argue that soon Internet (and, in particular, Web) surveys will replace traditional methods of survey data collection. Others are urging caution or even voicing skepticism about the future role Web surveys will play. Clearly, we stand at the threshold of a new era for survey research, but how this will play out is not yet clear. Whatever one's views about the likely future for survey research, the current impact of the Web on survey data collection is worthy of serious research attention. Given the rapidly growing interest in Web surveys,' it is important to distinguish among different types of Web surveys. The rubric "Web survey" encompasses a wide variety of methods, with different purposes, populations, target audiences, etc. I present an overview of some of the key types of Web surveys currently being implemented and do so using the language of survey errors. In order to judge the quality of a particular survey (be it Web or any other type), one needs to do so within the context of its stated aims and the claims it makes. By offering a typology of Web survey designs, the intent of this article is to facilitate the task of evaluating and improving Web surveys. Web surveys represent a double-edged sword for the survey industry. On the one hand, the power of Web surveys is that they make survey data collection (as opposed to survey participation) available to the masses. Not only can researchers get access to undreamed of numbers of respondents at dramatically lower costs than traditional methods, but members of the general <s> BIB002 </s> Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> E-mail and World Wide Web surveys have been the subject of much hyperbole about their capabilities as well as some criticism of their limitations. In this report, the authors examine what is known and not known about the use of the Internet for surveying. Specifically, they consider evidence in the literature regarding response rates, timeliness, data quality, and cost. Using this evidence, the authors evaluate popular claims that Internet-based surveys can be conducted more quickly, effectively, cheaply, and/or easily than surveys conducted via conventional modes. They find that the realities of cost and speed often do not live up to the hype. Nonetheless, it is possible to implement Internet-based surveys in ways that are effective and cost-efficient. The authors conclude that the Internet will continue to grow in importance for conducting certain types of research surveys <s> BIB003 </s> Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> Aims: To examine the use of the Internet in a survey of drinking among students, and the effectiveness of incentives to encourage participation. Methods: In a survey of drinking in university students, a random sample of 160 students were randomly assigned to one of four token incentive conditions. All received posted invitations, and reminders by e-mail and telephone. Results: Overall response was 85% and did not differ significantly by incentive condition. Conclusion: Internet surveys are effective in obtaining alcohol use information from students. Minimal incentives may suffice if coupled with intensive follow-up. <s> BIB004 </s> Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> We carried out an experiment that compared telephone and Web versions of a questionnaire that assessed attitudes toward sci- ence and knowledge of basic scientific facts. Members of a random digit dial (RDD) sample were initially contacted by telephone and answered a few screening questions, including one that asked whether they had Internet access. Those with Intern et access were randomly assigned to complete either a Web version of the questionnaire or a computer- assisted telephone interview. Ther e were four main findings. First, although we offered cases assigned to the Web survey a larger incen- tive, fewer of them completed the online questionnaire; almost all those who were assigned to the telephone condition completed the interview. The two samples of Web users nonetheless had similar demographic characteristics. Second, the Web surv ey produced less item nonresponse than the telephone survey. The Web questionnaire prompted respon- dents when they left an item blank, whereas the telephone interviewers accepted "no opinion" answers without probing them. Third, Web respondents gave less differentiated an swers to batteries of attitude items than their telephone counterparts . The Web questionnaire presented these items in a grid that may have made their similarity more salient. <s> BIB005 </s> Sampling Methods for Web and E-mail Surveys <s> Improving response rates for Internet-based surveys <s> Two meta-analyses review the effectiveness of incentives in Web studies. The first analysis summarizes 32 experiments on the impact of material incentives on response. It is based on a sample of 212,810 people. A significant effect (odds ratio = 1.19; 95% confidence interval: 1.13—1.25) is revealed, indicating that incentives motivate people to start a Web survey. The second analysis comprises 26 experiments on the impact of incentives on retention and is based on a sample of 7,073 people. It yields a significant effect (odds ratio = 1.27; 95% confidence interval: 1.12—1.44), indicating that once people have accessed a survey for whatever reasons, they are more likely to finish if an incentive is offered. The established incentive effects are stable across various study characteristics. Conclusions did not appear to be due to publication bias. <s> BIB006
Response rates have a direct effect on sampling: the higher the response rate, the fewer people need to be sampled to achieve a desired number of survey completions. In addition, higher response rates are associated with lower nonresponse bias. Unfortunately, in a summary of the academic survey-related literature up through 2001, BIB003 concluded that 'Web-only research surveys have currently only achieved fairly modest response rates, at least as documented in the literature.' S. BIB005 similarly summarized the state of affairs as 'Web surveys generally report fairly low response rates. ' A good illustration of this is the  study in which employees of five US federal government statistical agencies were randomly given a mail or e-mail survey. Comparable procedures were used for both modes, yet higher response rates were obtained for mail (68-76 percent) than for e-mail (37-63 percent) across all of the agencies. Incentives are a common and effective means for increasing response rates in traditional surveys. Goritz (2006) is an excellent review of the use of incentives in survey research in which he distinguishes their use in traditional surveys from Internet-based surveys and provides a nice discussion of the issues associated with using incentives in Internet-based surveys. Open issues include: • how best to deliver an incentive electronically; • whether it is better to provide the incentive prior to a respondent taking the survey or after; • whether incentives have different effects for individuals taking a survey one time versus prerecruited panel members who take a series of surveys. Individual studies of Internet-based surveys have generally found incentives to have little or no effect. For example, Coomly (2000) found that incentives had little effect on response rates for pop-up surveys, and BIB004 found no effect in a web-based survey. However, BIB006 conducted a meta-analysis of 32 experiments evaluating the impact of incentives on survey 'response' (the fraction of those solicited to take the survey that actually called up the first page of the survey), and 26 experiments evaluating the effect of incentives on survey 'retention' (the fraction of those who viewed the first page that actually completed the survey). From the meta-analysis, Görtiz concluded that 'material incentives promote response and retention in Web surveys' where 'material incentives increase the odds of a person responding by 19% over the odds without incentives' and 'an incentive increased retention by 4.2% on average'. In addition to incentives, Dillman (2007) and BIB001 have put forward a number of survey procedural recommendations to increase survey response rates, based on equivalent methods for traditional surveys, which we will not re-cover here since they are mainly related to survey design and fielding procedures. While we do note that the recommendations seem sensible, BIB002 