What are two challenges to using experimental designs?

Question

What will be an ideal response?

Answer 1

The Interaction of Testing and Treatment – a variation of the problem of external validity occurs when the experimental treatment is effective only when particular conditions created by the experiment occur. For example, if subjects have had a pretest, it may sensitize them to a particular issue so when they are exposed to the treatment, their reaction is different from what it would have been if they had not taken the pretest. In other words, testing and treatment interact to produce the outcome.
Suppose you were interested in the effects of a diversity training film on prejudicial attitudes. After answering questions in a pretest about their attitudes on various topics related to diversity (e.g., racial or sexual prejudice), the subjects generally became more sensitive to the issue of prejudice without seeing the training film. On the posttest, then, their attitudes may be different from pretest attitudes simply because they have become sensitized to the issue of diversity through pretesting. In this situation, the treatment may actually have an effect, but it would be difficult to determine how much of the effect was attributable to the sensitizing pretest and how much was due to seeing the film.
This possibility can be tested with what is called the Solomon four-group design. In this version of a true experimental design, subjects are randomly assigned to at least two experimental groups and at least two comparison groups. One experimental group and one comparison group will have a pretest, and the other two groups will not have a pretest. If testing and treatment do interact, the difference in outcome scores between the experimental and comparison groups will differ between the subjects who took the pretest and those who did not.