Explain how validity evidence is gathered and the two issues that validity is trying to define/explain.

Question

What will be an ideal response?

Answer 1

Traditionally, validity was viewed as the extent to which a measurement procedure actually measures what it is designed to measure. On the contrary, a thorough knowledge of the interrelationships between scores from a particular procedure and other variables typically requires many investigations. The investigative processes of gathering or evaluating the necessary data are called validation. Various methods of validation revolve around two issues: (1) what a test or other procedure measures (i.e., the hypothesized underlying trait or construct) and (2) how well it measures (i.e., the relationship between scores from the procedure and some external criterion measure). Thus, validity is a not a dichotomous variable (i.e., valid or not valid); rather, it is a matter of degree.
Validity is also a unitary concept (Landy, 1986). There are not different “kinds” of validity, only different kinds of evidence for analyzing validity. Although evidence of validity may be accumulated in many ways, validity always refers to the degree to which the evidence supports inferences that are made from the scores. Validity is neither a single number nor a single argument, but an inference from all of the available evidence (Guion, 2002). It is the inferences regarding the specific uses of a test or other measurement procedure that are validated, not the test itself (AERA, APA, & NCME, 2014; SIOP, 2018). Hence, a user first must specify exactly why he or she intends to use a measure (i.e., what inferences are to be made from it). While there are numerous procedures available for evaluating validity, Standards for Educational and Psychological Measurement (AERA, APA, & NCME, 2014) describes three principal strategies: content-related evidence, criterion-related evidence (predictive and concurrent), and construct-related evidence. These strategies for analyzing validity differ in terms of the kinds of inferences that may be drawn. Although we discuss them independently for pedagogical reasons, they are interrelated operationally and logically