Compare the strengths of, threats to internal validity of and similarities and differences between i) One group pre-test/post-test designs ii) Classic experimental designs iii) Non-equivalent...

Compare the strengths of, threats to internal validity of and similarities and differences between

i) One group pre-test/post-test designs

ii) Classic experimental designs

iii) Non-equivalent comparison group designs

Asked on by lskohleffel

1 Answer | Add Yours

mathsworkmusic's profile pic

mathsworkmusic | (Level 2) Educator

Posted on

i) One group pre-test/post-test design

Here, the experiment is carried out on only one group. The main advantage to having one group is that variability between individuals' results in the treatment and control group is eliminated as the pre- and post-test measurements are made within the same individual. If pre-tests weren't made and only post results were compared between groups, there would be more variability in outcomes. The efficiency of estimating the effect of treatment is reduced if individuals post-test measurements are compared to their own baseline measurements.

This design has disadvantages though, including confounding factors such as another external event affecting outcome rather than the administered treatment (they get supplementary treatment), or simply the natural development of individuals towards better scores (eg children maturing). Also, the results might just demonstrate regression to the mean where exceptionally poor or good initial test results regress towards a standard result, because the patient or candidate did well or badly by chance (e.g., in multiple choice tests), or were having a good or bad day. Further, the individuals given the intervention (all of them here) might learn to the test so that they do better the second time just because they have become more familiar with the testing procedure. The results might be biased if data are only kept for participants who complete the whole program if those who find the tests difficult drop out before the second test. This is called informative missingness. Finally, the test procedure might change so when comparing the test results one isn't comparing like with like. 

ii) Non-equivalent comparison group design versus iii) Classical experimental design

The NEGD does compare two groups so problems associated with having one group mentioned above are avoided, as does the classical experimental design. However, where the treatment and control groups are assigned at random for the latter, they are assigned according to convenience for the NEGD. They are chosen to be broadly similar ideally, but systematic differences might effect the internal validity of results. The results then might suffer from selection bias and present an inaccurate picture of the effectiveness of the treatment. If assignment is truly random this bias is avoided. The disadvantage to randomising individuals to treatment or placebo is that random allocation is viewed as unethical by some, particularly if good treatment is essential for survival of an ill patient or if some patients are more ill than others. In either case, results might be biased by the phenomenon where individuals think that they will improve just because they have been allocated to the treatment. This is called the placebo effect and some argue that this is the only way alternative remedies work. The 'gold standard' of the randomised control trial or the classical experimental design is for the experiment to be double-blind. This means that neither the experimenter nor the patient knows which group they are in. This is thought to reduce bias due to the placebo effect. If individuals are measured at baseline then between patient variability can be eliminated and differences between post treatment and pre treatment measured. if compared to the control arm there is a greater improvement, the treatment appears to have been successful. The sample size needs to be sufficiently large however to make this conclusion reasonably certain.

We’ve answered 318,945 questions. We can answer yours, too.

Ask a question