Describe how we collect and interpret data for a qualitative item analysis. Discuss the types of information the test developer should seek and give examples of questions.
What will be an ideal response?
• Potential areas of exploration by means of qualitative item analysis:
o Cultural sensitivity: Did you feel that any item or aspect of this test was discriminatory with respect to any group of people?
o Face validity: Did the test appear to measure what you expected it would measure?
o Test administrator: Did the presence of the test administrator affect your performance on this test in any way?
o Test environment: Did any conditions in the room affect your performance on this test in any way? If so, how?
o Test fairness: Do you think the test was a fair test of what it sought to measure?
o Test language: Were there any instructions or other written aspects of the test that you had difficulty understanding?
o Test length: How did you feel about the length of the test with respect to (a) the time it took to complete and (b) the number of items?
o Test taker’s guessing: Did you guess on any of the test items? About what percentage of the items would you estimate you guessed on?
o Test taker’s integrity: Do you think that there was any cheating during this test?
o Test taker’s mental/physical state upon entry: How would you describe your physical/ mental state going into this test? Do you think that your mental state in any way affected the test outcome?
o Test taker’s overall impressions: How would you describe your overall impression of this test?
o Test taker’s preferences: Was there any part of the test that you found entertaining or otherwise rewarding? What specifically did you like or dislike about the test? Was there any part of the test that you found anxiety provoking, condescending, or otherwise upsetting?
o Test taker’s preparation: How did you prepare for this test?
o Expert Panels: Test developers also find the information provided by experts to be helpful in understanding and improving test results. One group of experts would be people who are knowledgeable about the test’s constructs.
o Review by another test developer who has not been involved in the development process can provide a fresh look at issues of test length, administration instructions, and other issues not related to test content.
You might also like to view...
Although several studies have linked youth sports participation to ____, outcomes are usually positive when activities involve adults who emphasize skill development
a. the development of attention-deficit hyperactivity disorder b. delinquent and antisocial behavior c. stunted physical growth d. obsessive-compulsive disorder
The cerebellum controls vital reflexes
Indicate whether the statement is true or false
Our workforce will become less diverse racially and ethnically because we are a melting pot.
Answer the following statement true (T) or false (F)
When a belief is judged as positive or negative, it becomes ______.
A. attitude object B. dual attitude C. evaluative belief D. informational belief