Why is a true experimental design the strongest method for conducting impact evaluations?
What will be an ideal response?
The Risk Skills Training Program (RSTP) was designed to target multiple risk behaviors such as drinking and drug use along with changing adolescents’ personal beliefs about such behavior. Elizabeth D’Amico and Kim Fromme (2002) studied the impact of the RSTP and how it compared to the Drug Abuse Resistance Education-Abbreviated (DARE-A) program as well as how both of these programs affected outcomes compared to students who received no programming (a control group).
To evaluate the efficacy of these programs in reducing drinking and drug use, they randomly selected 150 students to participate in their study. Then students were randomly assigned to one of the three conditions: 75 students received RSTP programming, 75 students received the DARE-A programming, and another 150 students were randomly selected to participate but received no programming. The students received a pretest assessment and then posttest assessments, which took place at both two and six months after the programs.
The impacts (dependent variables) D’Amico and Fromme (2002) examined included positive and negative “alcohol expectancies” (the anticipated effects of drinking) as well as perceptions of peer risk taking and actual alcohol consumption. D’Amico and Fromme found that negative alcohol expectancies increased for the RSTP group in the posttest but not for the DARE-A group or the control group, while weekly drinking and “positive expectancies” for drinking outcomes actually increased for the DARE-A group and/or the control group by the six-month follow-up but not for the RSTP group (see Exhibit 10.3).
You should recognize the design used by D’Amico and Fromme as a true experimental design (see Chapter 6). This is the preferred method for maximizing internal validity—that is, for making sure your causal claims about program impact are justified. Cases are assigned randomly to one or more experimental treatment groups and to a control group so that there is no systematic difference between the groups at the outset. The goal is to achieve a fair, unbiased test of the program itself so that the judgment about the program’s impact is not influenced by differences between the types of people who are in the different groups. It can be a difficult goal to achieve, because the usual practice in social programs is to let people decide for themselves whether they want to enter a program and also to establish eligibility criteria that ensure that people who enter the program are different from those who do not (Boruch, 1997). In either case, a selection bias is introduced.
Impact analysis is an important undertaking that fully deserves the attention it has been given in government program funding requirements. However, you should realize that more rigorous evaluation designs are less likely to conclude that a program has the desired effect; as the standard of proof goes up, success is harder to demonstrate.
You might also like to view...
Which of the following is the best entrapment defense to any given crime?
A) I committed the crime because the government caused me to commit the crime. B) I did not commit the crime because the government caused me to commit conduct that is considered criminal. C) I committed the crime because I thought my act was not criminal. D) I did not commit the crime because I thought my actions were justifiable.
The Fourth Amendment "follows the flag."
Indicate whether the statement is true or false
What is the flight/fight system?
What will be an ideal response?
In which Supreme Court case did the court determine that inmate privacy was not a valid reason to refuse hiring women as correctional officers?
a. Morrissey v. Brewer b. Mempa v. Rhay c. Dothard v. Rawlinson d. Gunther v. Iowa