rom the fact most people want their answers to be consistent with each other. This effect operates most strongly when questions on a similar topic appear close together In 1944, while the USA was involved in World War II, D. Riggs and H. Cantrill explored the consistency effect by taking two surveys of different sets of randomly-selected respondents. One set of respondents was asked:
Should the United States allow its citizens to join the British and French armies?
The other was asked:
Should the United States allow its citizens to join the German army?
In the first version, 45% said yes to the first question and 31% said yes to the second question. When the questions were reversed, only 22% agreed with letting US citizens join the German army (“The Wording of Questions,” in H.Cantrill, editor, Gauging Public Opinion, Princeton University Press, 1944). 22% in the midst of World War II is pretty amazing, but the point of the research was that putting the allied armies first set up a pressure to be consistent and agree with allowing citizens to join an enemy army as well. Another sort of influence, the part-whole effect, occurs when one question refers to a broad topic and another to some more specific matter that can be seen as within the broad topic. H. Shuman and S. Presser tried reverse ordering of two questions with two sets of respondents in 1996. First they asked:
When they reversed the order, fewer respondents said there should be laws permitting abortion. They argued that respondents thought the more specific question asked about a special case of the broader question if the general question came first, but that the specific question was about a different topic if it came first. Thus the respondents asked about the strong likelihood of severe birth defects redefined “abortion” as not covering that situation (Questions and Answers in Attitude Surveys, Sage Publications, 1996).
Large surveys often use questions that have been used successfully in earlier surveys for either or both of two reasons. First, they are proven to work, sparing the researcher from having to do pretesting (see below). Second, asking the same question allows tracking attitudes or information over time.
Writing new questions requires considerable thought and care. Researcher need to make the question as clear as possible because ambiguous wording can lead some respondents to understand the question one way and others to understand it a different way. If they do, their answers will not be comparable because they “agree” or “disagree” with different statements. Using an interviewer rather than a self-completion questionnaire may not prevent this because many people are reluctant to admit that they do not understand a question. It is also important to avoid double-barrelled questions, questions that ask about two possibilities simultaneously. Any question with the word “or” – as in “Should the government increase the gas tax or leave it at the present level?” – is double-barrelled because a data coder cannot tell whether “yes” or “no” refers to raising the tax or leaving it where it is. However, any question that covers two options – for instance, “would you use the library more if it were open later on weekdays or on the weekend?” – causes the same problem because the coder can’t tell whether a “yes” answer means the respondent would use the library more if it were open later on weekdays, if it were open on weekends, or both. The library budget committee would certainly want to know so it could attract as many users as possible for the additional money spend of staff pay if the library is to be open more hours. Loaded questions, questions in which the first words use language that will encourage respondents to view an issue, problem, group, or individual in a positive or negative light, should also be avoided because they can affect answers. “Should deadbeat dads be put in jail?” has a very different effect on respondents than “Should parents who fail to pay child support be punished?”
Unless they are only using previously-asked questions, pollst