Consider each of the scenarios below and decide which of the following threats to internal validity are most apparent: extraneous events, instrumentation, mortality, regression, or selection. A group of second graders scoring at the bottom 10% on a reading test were targeted for intensive daily instruction for two weeks. They were retested after the remediation using the same instrument (but with clearer instructions) and scored significantly higher. A comparison group identified at another school as needing—but not yet receiving—remediation was also tested using the same instrument. They scored lower than the remediated group. This finding, coupled with the group's significant gain, led the researcher to conclude that the intensive instruction was effective.
When we are talking about threats to internal validity in experimental social sciences and educational research, we are essentially asking if we are measuring what we want to claim to be measuring (external validity refers to our ability to generalize findings to other groups). There are many potential threats that could threaten the validity of an experimental intervention, such as the one described in this scenario.
In this case, the issue most likely to affect the claim made by the researchers is selection. In experimental research, ideally the experimental group and the control group are identical. However, the researchers in this study opted for the experimental group to be housed in one school, whereas the control group exists at a separate school. It appears that the researchers did not do anything to determine that the groups were homogeneous (equal). Perhaps, for instance, one of the schools was a charter school or a school for special populations of students. These variables might affect the way that students learn and the way that they receive the intervention, and thus this may be a variable that is not being measured in the intervention.
There may be another issue at hand as well. The directions mention that after the intervention, students were retested with clearer instructions. This is an extraneous event that may affect the purity of the data. In this case, if the initial instructions were not clear, it means that the results from the pre-test could be a measure of how well students understood the directions, not a measure of how well they were able to perform on the test.
For clarity, mortality refers to losing participants over the course of time, which does not seem to be a factor here, unless a large influx of students begins moving or is pulled out of school for various reasons. Regression (toward the mean) simply refers to the fact that if we provide multiple tests, each student will generally move closer toward the mean score. Validity issues with instrumentation suggests that the instrument may not have questions designed to measure the construct that it claims to measure, although in this quantitative study based on study scores, we do not have enough information to say either way whether the instrument is appropriate.
check Approved by eNotes Editorial