Please help with these research questions in regard to the study below:
- Describe the research question for this experiment.
- What were the null and alternative hypotheses?
- Were the results of this test statistically significant?
- If so, why were they significant?
- Would the researchers reject or fail to reject the null hypothesis?
- Do the results provide sufficient evidence to support the alternative hypothesis?
- Was the sample appropriate for this study? Explain your answer.
- What are some possible limitations to this study?
- Discuss how you would conduct a follow up study to this one. Explain your answer.
- Describe the difference between practical and statistical significance.
A group of researchers conducted an experiment to determine which vaccine is more effective for preventing getting the flu. They tested two different types of vaccines: a shot and a nasal spray. To test the effectiveness, 1000 participants were randomly selected with 500 people getting the shot and 500 the nasal spray. Of the 500 people were treated with the shot, 80 developed the flu and 420 did not. Of the people who were treated with the nasal spray, 120 people developed the flu and 380 did not. The level of significance was set at .05. The proportion of people who were treated with the shot who developed the flu = .16, and the proportion of the people who were treated with the nasal spray was .24. The calculated p value = .0008.
(1) The null hypothesis is that there is no difference between the treatments; the alternative hypothesis is that there is some difference.
Since p=.0008 is less than the level of significance alpha=.05, so the results are statistically significant.
(2) The researchers would reject the null hypothesis. If the null hypothesis were true, a sample with results like this would only occur 8 out of 10000 trials. Since this is highly unlikely ( and far below the 5 in a hundred level of significance required) we would believe that the null hypothesis was incorrect.
(3) The sample is large enough. We are not told what the population is. (All US adults, US adults who are taking a vaccine, etc...) Assuming the samples are truly randomly chosen from a well-defined population the sample seems adequate.
(4) Possible limitations include an improperly defined population or the selection of the sample not being truly random. Also, the experiment could not be double-blind (it is hard to disguise whether you are getting a shot or the mist.) There is no control group to define the likelihood of an unvaccinated individual contracting the flu, though this might be known for the given population. Was the type of influenza considered? (Were the test subjects exposed to a particular strain or were they tracked during the flu season and only reported if they got the flu?)
(5) First identify any problems in the first test (improper population, etc...) and fix those. Make sure to control for any confounding variables including flu strain, age and health of participants. Insure the experiment is double-blind by including a control group who are issued a placebo (salt-water spray or inert shot) to reduce the placebo/nocebo effects or give everyone two treatments-- one placebo and the other that actual treatment. That is, if I am scheduled to get the shot, I also get a saline mist, where those destined to get the mist also get an inert shot.
(6) Assuming that the study results are correct it appears that receiving the vaccine as a shot is more effective than through nasal application. But we are not told what the percentage of untreated people who would contract the flu is. Suppose 1/4 of untreated people exposed to the flu actually become infected. (So 125 out of 500.) Then the small advantage of nasal protection might not be worth the time and expense. Indeed, while the difference between shot and mist might be statistically significant, the results might not be practically significant in that the cost (time, money, pain) might not outweigh the benefit (reduced chance of getting the flu.) In the case of immunosuppressed patients the risks of getting the flu are high, but for most healthy people the flu is typically an irritation so a small reduction in risk is not practically significant. (If the vaccine virtually guaranteed no infection this would be different.)