Is it possible to achieve both reliability and validity? If so, how?

Question

What will be an ideal response?

Answer 1

We must always assess the reliability of a measure if we hope to be able to establish its validity. Remember that a vary measure is not necessarily a valid measure, as Exhibit 4.8 illustrates. This discrepancy is a common flaw of self-report measures of substance abuse. The multiple questions in self-report indexes of substance abuse are answered by most respondents in a consistent way, so the indexes are reliable. However, a number of respondents will not admit to drinking, even though they drink a lot. Their answers to the questions are consistent, but they are consistently misleading. So the indexes based on self-report are reliable but invalid. Such indexes are not useful and should be improved or discarded. Unfortunately, many measures are judged to be worthwhile on the basis only of a reliability test.
The reliability and validity of measures in any study must be tested after the fact to assess the quality of the information obtained. But then if it turns out that a measure cannot be considered reliable and valid, little can be done to save the study. Thus, it is supremely important to select in the first place measures that are likely to be reliable and valid. In studies that use interviewers or observers, careful training is often essential to achieving a consistent approach. In most cases, however, the best strategy is to use measures that have been used before and whose reliability and validity have been established in other contexts. However, know that the selection of “tried and true” measures still does not absolve researchers from the responsibility of testing the reliability and validity of the measure in their own studies.
It may be possible to improve the reliability and validity of measures in a study that already has been conducted if multiple measures were used. For example, in a study of housing for homeless mentally ill persons, residents’ substance abuse was assessed with several different sets of direct questions as well as with reports from subjects’ case managers and others (Goldfinger et al., 1996). It was discovered that the observational reports were often inconsistent with self-reports and that different self-report measures were not always in agreement and were thus unreliable. A more reliable measure of substance abuse was initial reports of lifetime substance abuse problems. This measure was extremely accurate in identifying all those who subsequently abused substances during the project.