{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Discussion 6: Statistical significance\n", "\n", "STATS 60 / STATS 160 / PSYCH 10\n", "\n", "
\n", "\n", "
\n", "\n", "**Today's section**\n", "\n", "\n", "- Recap of lecture material.\n", "- Week 6 practice quiz 1.\n", "- Spotting AI images.\n", "\n", "\n", "
\n", "
\n", "\n", "\n", "
\n", "
\n", "\n", "# Recap\n", "\n", "## Hypotheses and p-values\n", "\n", "- A **p-value** is the probability of finding a result *at least* as extreme/surprising, if outcomes happened by random chance alone.\n", "- The **null hypothesis** corresponds to \"just chance\" or \"no effect.\"\n", "- The **alternative hypothesis** corresponds to \"better than chance\" or \"an effect.\"\n", "- Small p-value \u2192 evidence against the null hypothesis.\n", "- \"Small\" means less than 0.05.\n", "\n", "## p-value visualization\n", "\n", "\n", "
\"\"
\n", "\n", "The p-value is represented by the red area in the histogram.\n", "\n", "## Computing p-values\n", "\n", "- To compute a p-value, we need a model for what the results would have looked like if there was no effect.\n", "- Three questions:\n", " 1. If the null was true, what would be the \"probability of success\"? \n", " 2. What should be the \"number of trials\" (also called the sample size)?\n", " 3. What value will you compare the simulated data to?\n", "\n", "\n", "# Week 6 practice quiz\n", "\n", "## Background\n", "\n", "
\n", "\n", "
\n", "\n", "- Suppose that the league average for a soccer player scoring a penalty is 78%. A new player just scored 18 out of their last 20 penalty kicks.\n", "\n", "- You will investigate whether the number of goals scored by this new player is significantly different from the league average.\n", "\n", "
\n", "
\n", "\n", "
\"\"
Courtney Vine at the 2023 World Cup.
\n", "\n", "
\n", "
\n", "\n", "\n", "\n", "\n", "## Question 1\n", "\n", "- What are the null and alternative hypotheses? Describe them both in English and in mathematical symbols.\n", "\n", "- **Answer**\n", "\n", "- The null hypothesis is that the new player is just as good at scoring penalties as the league average. The alternative hypothesis is that the new player is better at scoring penalties.\n", "\n", "- In symbols, let $\\pi$ be the probability of the new player scoring on a penalty. The null hypothesis is $H_0 : \\pi = 0.78$ and the alternative hypothesis is $H_A : \\pi > 0.78$.\n", "\n", "## Question 2\n", "\n", "- Describe how you would do a simulation to compute a p-value. \n", "- If the null was true, what would be the \"probability of success\"? \n", "- What would be the \"number of trials\"?\n", "- What value would you compare the simulated data to?\n", "\n", "- **Answer**\n", "\n", "- A \"success\" corresponds to the goal being scored. If the new player was just as good as the league average then they would score with probability 0.78. Therefore, the \"probability of success\" is 0.78.\n", "- The number of trails is 20 (the number of penalties taken).\n", "- The value to compare to is 18 (the number of goals scored in the sample).\n", "\n", "## Question 3\n", "\n", "- The p-value for the observed results (18 out of 20 goals) is 0.15. What do you conclude about the null hypothesis?\n", "\n", "- **Answer**\n", "\n", "\n", "- Since the p-value is greater than 0.05, we do not have evidence against the null hypothesis that the player is just as good as the league average. \n", "\n", "\n", "# Spotting AI images\n", "\n", "## Spotting AI images\n", "\n", "\n", "- For each of the following images (1 through 7) write down which face (left or right) you think is real and which face you think is AI generated.\n", "- You will then see if your results are statistically significant.\n", "- We will look at the first image together.\n", "- The images are taken from [whichfaceisreal.com](https://www.whichfaceisreal.com/).\n", "\n", "## Which face is real? - Practice\n", "\n", "\n", "\n", "\n", "
\"Image
Which face is real?
\n", "\n", "## Which face is real? - Solution\n", "\n", "\n", "\n", "\n", "
\"Image
The left face is real.
\n", "\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "
\"Image
Image 1: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 2: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 3: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 4: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 5: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 6: Which face is real?
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 7: Which face is real?
\n", "\n", "\n", "\n", "# Results\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "
\"Image
Image 1: Left real.
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 2: Left real.
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 3: Left real.
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 4: Right real.
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 5: Right real.
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 6: Right real
\n", "\n", "## Which face is real? \n", "\n", "\n", "\n", "\n", "
\"Image
Image 7: Left real.
\n", "\n", "\n", "\n", "## p-value computation\n", "\n", "Use the One Proportion applet to compute a p-value for your results.\n", "\n", "\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "\n", "- Keep in mind the following:\n", " - Probability of success.\n", " - Number of trails.\n", " - What value to compare the data to.\n", "- State your conclusion in terms of a null hypothesis.\n", "\n", "
\n", "
\n", "\n", "
\"\"
\n", "\n", "
\n", "
" ] } ], "metadata": { "kernelspec": { "display_name": "Phython (JB)", "language": "python", "name": "jb-python" } }, "nbformat": 4, "nbformat_minor": 4 }