How would one evaluate the article "Cognitive Effects of Riperidone in children with Autism and Irritable Behavior" by Michael Aman et al. (2008) and critique the statistical analysis employed in the study?
When evaluating an article stating published research using a critique of statistical analysis, one is taking steps to determine if the conclusions the scientists reached were valid or if the conclusions cannot be deemed valid due to inappropriate analyses of the data. There are several steps to take. First, you want to identify the hypothesis/hypotheses of the scientists, and the hypothesis/hypotheses should first be clearly asserted in the article's abstract. To find the hypothesis/hypotheses, first consider the exact purpose of the study. As authors Michael G. Aman et al. state in the abstract of their article titled "Cognitive Effects of Risperidone in Children with Autism and Irritable Behavior," the purpose of their study was to find out if the drug risperidone has any effects on the cognitive processes of children with behavior problems, like autistic children or children with irritable behavior. The abstract further states their conclusion, which was that 3.5 mg doses of risperidone given to children for up to 8 weeks "appears to have no detrimental effect on cognitive performance" (Aman et al.). It should be noted that the statement draws no conclusions concerning whether or not the drug has positive effects; therefore, we can deduce from the abstract that their hypothesis was that risperidone administered at that dosage will not have dire effects on the children's cognitive abilities. However, now that we have deduced the hypothesis based on the abstract, you will want to find it stated in the actual article and stat the hypothesis in your own evaluation.
Once you determine the hypothesis, you next want to determine and state in your evaluation how the study was designed, such as what types of groups of people were used and what treatments were applied. Again, we see this information first stated in the abstract under the method section.
Once you've determined the study design, you next want to determine the type of statistical test that was used, and there are several options, such as the "two-way, mixed model ANOVA, with 'factor A' fixed and 'factor B' random" ("Critique of Statistical Analysis in Published Papers"). You may want to review the different types of statistical tests in your course material, but consider that if the study involves two groups, then it is obviously a two-way ANOVA study with the two groups being divided between taking risperidone and a placebo to determine effects on cognition.
Once you've determined the statistical methods, especially noting the dependent and independent variables, you are then in a better position to critique the statistical analysis. To do that, you first want to ask yourself if the statistical method the researchers used was appropriate, or do you see a more appropriate method? Next, you want to actually perform the calculations yourself. We already know again from the abstract that the researchers used ANOVA to measure the changes in the children's performance. ANOVA stands for Analysis of Variance, and through it, we compare the "averages of two or more groups" ("Selecting Appropriate Inferential Statistical Techniques"). So, if we know the data and the averages, we can calculate our own averages to see if the researchers' calculations are correct. Performing the calculations allows you to be able to analyze the statistical data and assess whether or not the data is valid.