What is cross-validation and what is its purpose and importance? Describe two methods for finding the criterion-related validity coefficient for cross-validation.

Question

What will be an ideal response?

Answer 1

• If the final revision of a test yields scores with sufficient evidence of reliability/precision and validity, the test developers conduct a final analysis called cross-validation for tests that rely on criterion-related evidence of validity to make predictions.

• The process of cross-validation involves a final round of test administration to another sample of test takers representative of the target audience.

• It is necessary to ensure that the validity coefficient that was found in the initial validation study was not overly influenced by chance factors that were unique to the population included in the original study.

• The purpose of cross validation is to ensure that the results of the validation study will generalize to other populations.

• The process of cross-validation involves a final round of test administration to another sample of test takers representative of the target audience.

• Cross-validation can also be conducted without having to administer the test to a second group of test takers if the original group is large enough.
o The first step is to randomly split the data set into two parts
o Then, a regression is performed on one half of the sample using the test scores as the predictor and the actual criterion scores as the criteria. The object of this step is to create an equation that can be used to calculate a set of “predicted” criterion scores for the other half of the sample.
o Those predicted scores are then correlated with the actual criterion scores for that group. The higher that correlation, the better the results of the cross validation study.