What is meant by validity generalization? Describe three methods that can be used to generalize evidence of validity.

Question

What will be an ideal response?

Answer 1

• One issue that frequently arises in organizations that wish to use a pre-employment test for selecting employees is demonstrating that the test is valid for use in that organization. In practice, however, data gathering and validation studies may be seen as too expensive and time-consuming for an organization to undertake. For example, hiring new employees is often an immediate business need, and conducting validity research can be time-consuming.

• One of the ways that organizations can deal with this issue is via something referred to as the generalization of validity evidence. This involves making the case that a test is valid for a particular use based on evidence for validity that has been provided using methods other than a validity study actually conducted in the organization that wishes to use the test.

• There are three strategies that can be useful when it becomes necessary to make a case that evidence of validity will generalize from one situation to another. The strategies are meta-analysis, synthetic validity, and transportability.
o Meta-analysis is a statistical technique that accumulates the results of multiple studies on comparable or similar constructs into a single result. The goal is to provide a better estimate of the true population value of a predictor–criterion relationship than that provided by any one individual study.
? The idea is that by combining the results of multiple studies into a single, larger analysis, one can reduce the impact that statistical artifacts have on the results.
? In the organizational psychology literature, meta-analysis has been generically referred to as validity generalization. This is because the procedure is often used to address the question of why a certain construct (like personality test scores) seem to be valid for predicting criteria (like job performance) in one study but not in another.
? The question that needs to be answered is whether the different results across the studies is due to statistical artifacts or to true differences in the relationship between the predictors and criteria in the population. If the results can be shown to be largely attributable to statistical artifacts, then one might be able to conclude that the validity of the test scores really does generalize across the populations.
o A second strategy for generalizing validity evidence is called synthetic validity, or job component validity.
? In this approach, existing evidence of validity is generalized from one group of jobs to another if it can be shown that the same job components are present in both
? Job components are defined as clusters of work behaviors that are similar across multiple jobs or job families, such as problem-solving skills, mechanical aptitude skills, clerical skills, and the like.
? Therefore, a test that validly measures an attribute important for job success in one job or job family should be equally valid for any job for which the same attribute (job component) is also required.
? Once the job components for a job or job family are identified along with the tests that have been previously been validated to measure these components, a synthetic validity coefficient can be computed.
? This is different from the traditional criterion-related approach to calculating validity coefficients because the calculations don’t rely directly on the correlations between individuals’ scores on the tests with their performance criteria scores in the actual job being validated.
o A third and more common strategy used to generalize validity evidence is transportability.
? Transportability occurs when a case is made that because a test has been shown to be valid for a particular use, it can be presumed to also be valid for another, similar use.
? The need for this strategy can occur when a test that has been validated for selection of employees into one job is proposed to select employees into a different but similar job.
? There are at least two criteria that should be present before deciding that the validity evidence for a test will generalize. First, test users need a study that demonstrates evidence of validity for the original use of the test. Second, there must be evidence that there is a strong similarity between the content and requirements of the job for which the test was originally validated and the new job.