Discuss two distinct methods of evaluating validity coefficients. Explain and give examples of the differences in information that the two methods provide.

Question

What will be an ideal response?

Answer 1

• The first question to ask about a validity coefficient is “How likely is it that the correlation between the test and the criterion resulted from chance or sampling error?” In other words, if the test scores (e.g., SAT scores) and the criterion (e.g., college GPA) are completely unrelated, then their true correlation is zero. If we conducted a study to determine the relationship between these two variables and found that the correlation was .4, one question we would need to ask is “What is the probability that our study would have yielded the observed correlation by chance alone, even if the variables were truly unrelated?” If the probability that the correlation occurred by chance is low—less than 5 chances out of 100, we can be reasonably sure that the test and its criterion (in this example, SAT scores and college GPA) are truly related. This process is called a test of significance.

• Another way to evaluate the validity coefficient is to determine the amount of variance that the test and the criterion share. We can determine the amount of shared variance by squaring the validity coefficient to obtain r2—called the coefficient of determination. For example, if the correlation (r) between a test and a criterion is .30, the coefficient of determination (r2) is .09. This means that the test and the criterion have 9% of their variance in common.