Identify and describe the major potential sources of noise in research

Question

What will be an ideal response?

Answer 1

There are several important sources of noise, each of which can be considered a threat to conclusion validity. One important threat is low reliability of measures. This can be caused by many factors, including poor question wording, bad instrument design or layout, illegibility of field notes, and so on. In studies where you are evaluating a program, you can introduce noise through poor reliability of treatment implementation. If the program doesn't follow the prescribed procedures or is inconsistently carried out, it will be harder to see relationships between the program and other factors like the outcomes. Noise caused by random irrelevancies in the setting can also obscure your ability to see a relationship. For example, in a classroom context, the traffic outside the room, disturbances in the hallway, and countless other irrelevant events can distract the researcher or the participants. The types of people you have in your study can also make it harder to see relationships. The threat here is due to the random heterogeneity of respondents. If you have a diverse group of respondents, group members are likely to vary more widely on your measures or observations. Some of their variability may be related to the phenomenon you are looking at, but at least part of it is likely to constitute individual differences that are irrelevant to the relationship you observe. All of these threats add variability into the research context and contribute to the noise relative to the signal of the relationship you are looking for.