 use disproportionate stratified sample to make sure that they have enough respondents from the relatively small group to get an accurate assessment of the range of views in the smaller group. A researcher surveying with genders as strata might use a sample of 300 males and 300 females; a researcher stratifying a population by ethnicity in which 48% were one ethnicity, 48% a second ethnicity, and the remaining 4% a third, might use a sample with 300 from each group. This does not mess up results if the goal is to compare attitudes within each group to attitudes within each other group. It only becomes a problem if someone forgets how the samples were drawn and assumes that the set of responses were collected from a random sample and can be used to draw conclusions about the attitudes of the population as a whole rather than about the separate groupings.
Formulating the Questions
Having a good random or proportionate stratified sample of respondents is important to a good research project. However the data is generated by asking the respondents for their answers to one or more questions. The data cannot be any better than the questions.
Pollsters seeking to predict election results have a very standard set of questions. Early in the nomination process (during primaries for example) they often try to assess the relative popularity of various candidates by asking how a person would vote if different pairs of candidates were running against each other. You will see this in early 2008 as people start thinking about which Republican hopeful can or cannot defeat which Democratic hopeful. At this point in the campaign season they can also likely voters in each party who they plan to support in the primary. You saw lots of polls of this type in late August and early September as O’Reilly, Gabrielli, and Patrick were all contending for the Democratic nomination. As the general election comes closer, the questions which to “which candidate will you vote for?” or, for the ballot questions, “will you vote yes or no on Question __?”
Survey questionnaires are less standardized because the questions they are designed to answer vary far more. Developing a good survey questionnaire requires careful work on six distinct steps: a) defining the purpose of the survey and what will be included, b) selecting an appropriate form of questionnaire, c) deciding whether to use open-ended questions, closed-ended questions, or some mix of both, d) setting the order of questions, e) setting the wording of each question, and f) doing a “pretest” and “pilot” of the questionnaire to be sure that actual respondents understand and react to it as the researchers assume they will.
Defining the purpose starts from realization that surveys can be used to attain a number of research goals. Some surveys are used to develop descriptive information about a population of individuals or some other unit. Questions must be focused on getting the information desired. Other surveys are used to test one or more causal hypotheses about why people hold particular views or act in some way. The hypothesis will determine the question content since the researcher will want data conforming challenging the hypothesis. If a student of voting behavior claims that veterans in a country vote for conservative parties and there are no lists of veterans available, then the questionnaire must include questions about whether the respondent is a vateran as well as on how the respondent voted in the last election. In both types of project, the questions can range from the very factual (“when were you born?” “have you been to the UK in the past five years?” “when do you get up on a workday?”) to the very judgmental (“how would you rate GW Bush’s performance as president?” “does the Israel lobby have too much influence over US policy in the Middle East?).
There are two main types of questionnaires: self-completion questionnaires handout, mail (or e-mail) surveys that respondents fill out and return to the researcher, and interview questionnaires that an interviewer follows in direct phone or 