What is an index? When might it be used?

Question

What will be an ideal response?

Answer 1

Measuring variables with single questions is very popular. Public opinion polls based on answers to single questions are reported frequently in newspaper articles and TV newscasts: “Do you favor or oppose U.S. policy for...?” “If you had to vote today, for which candidate would you vote?” The primary problem with using a single question is that if respondents misunderstand the question or have some other problem with the phrasing, there is no way to tell. Single questions are prone to this idiosyncratic variation, which occurs when individuals’ responses vary because of their reactions to particular words or ideas in the question. Differences in respondents’ background, knowledge, and beliefs almost guarantee that they will understand the same question differently. If a number of respondents do not know some of the words in a question, we may misinterpret their answers—if they answer at all. If a question is too complex, respondents may focus on different parts of the question. If prior experiences or culturally biased orientations lead different groups in the sample to interpret questions differently, answers will not have a consistent meaning because the question meant something different to each respondent.

As noted above, if just one question is used to measure a variable, the researcher may not realize respondents had trouble with a particular word or phrase in the question. Although writing carefully worded questions will help reduce idiosyncratic variation, when measuring concepts, the best option is to devise an index of multiple rather than single questions.

When several questions are used to measure one concept, the responses may be combined by taking the sum or average of the responses. A composite measure based on this type of sum or average is called an index or scale. The idea is that idiosyncratic variation in response to single questions will average out, so the main influence on the combined measure will be the concept focused on by the questions. In addition, the index can be considered a more complete measure of the concept than can any one of the component questions.

Idiosyncratic variation: Variation in responses to questions that is caused by individuals’ reactions to particular words or ideas in the question instead of by variation in the concept that the question is intended to measure.

Index: A composite measure based on summing, averaging, or otherwise combining the responses to multiple questions that are intended to measure the same variable.

Scale: A composite measure of one concept created from a series of two or more questions.

Creating an index, however, is not just a matter of writing a few questions that seem to focus on one concept. Questions that seem to you to measure a common concept might seem to respondents to concern several different issues. The only way to know that a given set of questions does effectively form an index is to administer the questions in a pretest to people similar to the sample you plan to study. If a common concept is being measured, people’s responses to the different questions should display some consistency. Special statistics called reliability measures help researchers decide whether responses are consistent. Most respondent attitudes are complex and consist of many elements.

Reliability measures: Special statistics that help researchers decide whether responses are consistent.

Be aware of response sets when constructing an index measuring attitudes. For example, some people tend to agree with almost everything asked of them, whereas others tend to disagree. Still others are prone to answer neutrally to everything if given the option. To decrease the likelihood of this happening, it is a good idea to make some statements both favorable and unfavorable to a particular attitude to vary the response choices and still reach an understanding of an individual’s opinion. In this way, respondents are forced to be more careful in their responses to individual items. Exhibit 7.4 displays a hypothetical set of questions designed to solicit respondents’ attitudes toward police in their community.

When scoring an index or scale made up of both favorable and unfavorable statements, you must remember to reverse code the unfavorable items. For example, marking “strongly agree” on the first item in Exhibit 7.4 should not be scored the same as a “strongly agree” response to the second item.

Exhibit 7.4 Items in an “Attitude Toward Police” Index