FIELD Crawford METHODS / WEB SURVEY DATA COLLECTION SYSTEMS REVIEWS Evaluation of Web Survey Data Collection Systems <s>  <s> As we enter the twenty-first century, the Internet is having a profound effect on the survey research industry, as it is on almost every area of human enterprise. The rapid development of surveys on the World Wide Web (WWW) is leading some to argue that soon Internet (and, in particular, Web) surveys will replace traditional methods of survey data collection. Others are urging caution or even voicing skepticism about the future role Web surveys will play. Clearly, we stand at the threshold of a new era for survey research, but how this will play out is not yet clear. Whatever one's views about the likely future for survey research, the current impact of the Web on survey data collection is worthy of serious research attention. Given the rapidly growing interest in Web surveys,' it is important to distinguish among different types of Web surveys. The rubric "Web survey" encompasses a wide variety of methods, with different purposes, populations, target audiences, etc. I present an overview of some of the key types of Web surveys currently being implemented and do so using the language of survey errors. In order to judge the quality of a particular survey (be it Web or any other type), one needs to do so within the context of its stated aims and the claims it makes. By offering a typology of Web survey designs, the intent of this article is to facilitate the task of evaluating and improving Web surveys. Web surveys represent a double-edged sword for the survey industry. On the one hand, the power of Web surveys is that they make survey data collection (as opposed to survey participation) available to the masses. Not only can researchers get access to undreamed of numbers of respondents at dramatically lower costs than traditional methods, but members of the general <s> BIB001 </s> FIELD Crawford METHODS / WEB SURVEY DATA COLLECTION SYSTEMS REVIEWS Evaluation of Web Survey Data Collection Systems <s>  <s> Many claims are being made about the advantages of conducting surveys on the Web. However, there has been little research on the effects of format or design on the levels of unit and item response or on data quality. In a study conducted at the University of Michigan, a number of experiments were added to a survey of the student pop- ulation to assess the impact of design features on resulting data quality. A sample of 1,602 students was sent an e-mail invitation to participate in a Web survey on attitudes toward affirmative action. Three experi- ments on design approaches were added to the survey application. One experiment varied whether respondents were reminded of their progress through the instrument. In a second experiment, one version presented several related items on one screen, while the other version presented one question per screen. In a third experiment, for one series of questions a random half of the sample clicked radio buttons to indicate their answers, while the other half entered a numeric response in a box. This article discusses the overall implementation and outcome of the survey, and it describes the results of the imbedded design experiments. <s> BIB002 </s> FIELD Crawford METHODS / WEB SURVEY DATA COLLECTION SYSTEMS REVIEWS Evaluation of Web Survey Data Collection Systems <s>  <s> Web surveys appear to be attaining lower response rates than equivalent mail surveys. One reason may be that there is currently little information on effective strategies for increasing response to Internet-based surveys. Web users are becoming more impatient with high-burden Web interactions. The authors examined the decision to respond to a Web survey by embedding a series of experiments in a survey of students at the University of Michigan. A sample of over 4,500 students was sent an e-mail invitation to participate in a Web survey on affirmative action policies. Methodological experiments included using a progress indicator, automating password entry, varying the timing of reminder notices to nonrespondents, and using a prenotification report of the anticipated survey length. Each of these experiments was designed to vary the burden (perceived or real) of the survey request. Results of these experiments are presented. <s> BIB003 </s> FIELD Crawford METHODS / WEB SURVEY DATA COLLECTION SYSTEMS REVIEWS Evaluation of Web Survey Data Collection Systems <s>  <s> Preface to the 2007 Update.Preface to the Second Edition.Acknowledgments.Part One: ELEMENTS OF THE TAILORED DESIGN METHOD.1 Introduction to Tailored Design.2 Writing Questions.3 Constructing the Questionnaire.4 Survey Implementation.5 Reduction of Coverage and Sampling Errors.Part Two: TAILORING TO THE SURVEY SITUATION.6 Mixed-Mode Surveys.7 Alternative Questionnaire Delivery: In Person, to Groups, and through Publications.8 When Timing Is Critical: Diary, Customer Satisfaction, and Election Forecast Surveys.9 Household and Individual Person Surveys by Government.10 Surveys of Businesses and Other Organizations.11 Internet and Interactive Voice Response Surveys.12 Optical Scanning and Imaging, and the Future of Self-Administered Surveys.References.2007 Appendix: Recent Developments in the Design of Web, Mail, and Mixed-Mode Surveys.Appendix References.Index. <s> BIB004
The World Wide Web has emerged as a viable means of collecting survey responses, and software developers have jumped at the chance to design the gold-standard Web-based data collection system. Such a system, it has been thought, could bring Web-based surveying to the common researcher. A comprehensive, self-contained Web survey system that meets most needs has yet to be developed. This is for a good reason: The Web survey research industry has not developed well-documented standards for how to do good-quality Web-based research. Various researchers have begun to explore the intricacies of doing research online and how various designs affect data quality  BIB001 BIB002 BIB003 BIB004  . However, until a basic framework for design standards has been developed, the specifications for a good-quality Web-based data collection product will not be set. Until then, software quality will hinge greatly on the ability of the researcher to use the tools provided by the software to create the best possible solution, given the resources available for each survey project. The three products selected for this review represent the three main types of Web survey packages available today: a basic off-the-shelf package, an intermediate application service provider package, and an advanced survey system. The terms basic, intermediate, and advanced will become clearer as the evaluation develops. They correspond positively with cost; however, cost should not be used as a sole indicator of capabilities or quality. With Web-based survey systems, as with most Web development environments, there are almost always additional products, services, and systems required to make the product fully functional. These additional pieces influence overall costs and resource requirements. Contact the software vendors for detailed costs as they pertain to your specific survey needs. Regardless of which package you choose to invest in for your research needs, I recommend that you work with a network specialist to make certain that your available network systems (including hardware, software, operating systems, and network connections) can accommodate the software for its intended use as a twenty-four-hour-a-day, seven-day-a-week platform. Each product in this review represents a category of product that exists in the marketplace today. Inclusion was primarily based on my experience using these systems. I conducted the review through actual product use, product documentation, and with any additional information made available through the product Web site. I reviewed each product as a researcher who implements Web surveys on a daily basis. While the review does not exhaustively explore every feature and capability, it gives a broad idea of the level of functionality each product and product group may offer. Researchers considering purchasing any of these products should thoroughly review the one they feel may best fit their needs to ensure that it actually does. Three products were selected for this review. The first is the following: SurveySolutions for the Web v4.0 is the latest version of the low end of the Perseus product line. This package has received much attention for its ability to bring Web surveys to low-budget research. In many respects, it is a teaser product for the more comprehensive Professional and Enterprise versions also distributed by Perseus. SurveySolutions is clearly designed for the researcher who does not possess a high level of technical skills. The interface looks much like the interface in Microsoft Word. Surveys can be created in a word processor and uploaded into the survey system. Figure 1 shows the look of the design interface.