Worksheet 20: Potential Outcomes Model#
Your name:
Your student ID number:
Your name:
Your student ID number:
In the potential outcomes model, what is the null hypothesis for the experiment?
What is the \(p\)-value?
For our class experiment about retrieval practice, what was the difference in means between the treatment and control groups? What was the \(p\)-value for this difference in means?
We’ll consider a few experimental designs, and critique each. Try to answer the following questions: 1. What are the strengths of the experimental design? 2. What are the weaknesses? 3. Does the experiment have reliable conclusions?
Consider the following questions: Is there a sensible control group? Are you worried about confounding variables? How large is the sample size? Are you worried about sample bias? Are the measurements relevant to the hypothesis?
An overweight podcaster wants to determine if eating breakfast will help him lose weight. The podcaster normally does not eat breakfast. He weighs himself on January 1, then eats breakfast every day for 6 months, and weighs himself again on July 1, and checks whether he lost weight. The podcaster lost 7 lbs. He releases an episode about the benefits of breakfast.
What are the pros of this experimental design?
What are the cons?
Does the experiment have reliable conclusions?
A health insurance provider conducts a survey of its \(10,000\) customers. The survey includes the following questions: How much do you weigh? How many days per week do you typically eat breakfast? A data scientist looks at the standardized scatterplot of body weight vs. days of breakfast per week. The correlation coefficient is \(R = -.1\), with \(p\)-value \(.03\). The average weight among those who eat breakfast 4+ days per week is 15 lbs lower than the average weight among those who eat breakfast < 4 days per week (sadly, the average in both categories is still overweight).
What are the pros of this experimental design?
What are the cons?
Does the experiment have reliable conclusions?
A medical researcher at a university hospital recruits 50 female participants between the ages of 25-50 who are habitual non-breakfast eaters, and meet the minimal requirements of regularly sleeping at least 6 hours per night and have stable body weight for the last 3 months. They are randomized into two groups: The treatment group is assigned to eat breakfast every day for a month consuming at least 15% of recommended daily calorie intake within 90 minutes of waking. The control group does not change their behavior. At the end of the month the researcher weighs the women and analyzes the change in weight using the potential outcomes model. The mean difference in weight gain between the breakfast eaters and non-eaters is 1 lbs (breakfast eaters weighed more), with \(p\)-value 0.03.
What are the pros of this experimental design?
What are the cons?
Does the experiment have reliable conclusions?