A researcher administers a test of hopelessness to 50 teen mothers in a continuation high school, in order to norm it for a study planned for the subsequent semester. This norming process is part of the approved research protocol

Question

Retesting in two weeks reveals different values for many of the subjects' paired tests. What should the researcher do now, and why? (Select all that apply.)
a. Administer the test a third time and see whether there is a trend. The subjects may be becoming more/less hopeless because of an external factor or a Hawthorne effect.
b. Calculate statistical tests to determine whether there are about the same amounts of positive change and negative change for the pooled sample, and whether the means are very similar for the two measurements. If so, this represents normal fluctuation of mood.
c. After obtaining IRB approval for study revision, ask several of the 50 subjects with large changes about their recent experiences and discover whether there is an underlying reason for the change.
d. Use the second value, not the first, for all data analysis. The subjects have evidently re-examined their feelings and discovered new insights.
e. Consider the multitrait-multimethod technique for this complex concept. A single measurement doesn't seem to be capturing the concept very well.

Answer 1

ANS: A, B, C, E
Stability reliability is concerned with the consistency of repeated measures of the same attribute with the use of the same scale or instrument over time. It is usually referred to as test-retest reliability. Test-retest may include an additional retest, especially when an unanticipated event has occurred. A third measurement may capture the Hawthorne effect. To capture a complex concept, researchers use the multimethod-multitrait technique, or even a mixed methods design, employing a variety of data collection methods, such as scale, interview, and observation. These techniques allow researchers to measure more dimensions of abstract concepts, and the effect of the systematic error on the composite observed score decreases. Subjects' knowledge of a study could influence their behavior and possibly alter the research outcomes. This threatens the validity or accuracy of the measurements. This change could be due to normal random measurement error. If so, the sum of the error (the measured change) for all subjects should be zero, and the means on the two days should be the same. Random error does not influence the mean to be higher or lower but, rather, increases the amount of unexplained variance around the mean.
On the other hand, test-retest reliability requires the assumption that the factor being measured has not changed between the measurement points. Many of the phenomena studied in nursing, such as hope, coping, pain, and anxiety, do change over short intervals. Perhaps true change has occurred.