MATH 1051
Trent University
So the analogy: if we report a point estimate, we probably won’t hit the exact population parameter. If we report a range of plausible values we have a good shot at capturing the parameter.
As discussed before reading week, there is inherent sampling variability that occurs when we sample from a population. We can leverage this to get a sense of how variable we would expect results to be when taken from an assumed distribution.
Thus a confidence interval is a range of plausible values where we may find the true population value. The process for creating a confidence interval is based on understanding how a statistic (here the sample proportion) varies around the parameter (here the population proportion) when many different statistics are calculated from many different samples.
This is the distribution from 10,000 bootstrap simulations. They vary from 0 to 0.15. This variability leads us to believe that the true probability of complication (the underlying parameter \(p\)) is likely between 0 and 11.3% (the two dashed lines).
library(infer)
dat_df <- data.frame(Outcome = c(rep("C", 3),
rep("NC", 59)))
p_hat <- dat_df |>
observe(response = Outcome, success = "C", stat = "prop")
boot_dist <- dat_df |>
specify(response = Outcome, success = "C") |>
generate(reps = 10000, type = "bootstrap") |>
calculate(stat = "prop")
percentile_ci <- get_ci(boot_dist)
percentile_ci# A tibble: 1 × 2
lower_ci upper_ci
<dbl> <dbl>
1 0 0.113This range of values, \((0, 0.1129)\) is known as a 95% bootstrap percentile confidence interval. Why 95%?
If we want to be more certain that we capture the population parameter, i.e., increase our confidence level, should we use a wider interval or a smaller interval?
If we want to be more certain that we capture the population parameter, i.e., increase our confidence level, should we use a wider interval or a smaller interval?
A wider interval.
If we want to be more certain that we capture the population parameter, i.e., increase our confidence level, should we use a wider interval or a smaller interval?
A wider interval.
Can you see any drawbacks to using a wider interval?
If we want to be more certain that we capture the population parameter, i.e., increase our confidence level, should we use a wider interval or a smaller interval?
A wider interval.
Can you see any drawbacks to using a wider interval?
If the interval is too wide it may not be very informative.1
Two scientists want to know if a certain drug is effective against high blood pressure. The first scientist wants to give the drug to 1000 people with high blood pressure and see how many of them experience lower blood pressure levels. The second scientist wants to give the drug to 500 people with high blood pressure, and not give the drug to another 500 people with high blood pressure, and see how many in both groups experience lower blood pressure levels. Which is the better way to test this drug?
Two scientists want to know if a certain drug is effective against high blood pressure. The first scientist wants to give the drug to 1000 people with high blood pressure and see how many of them experience lower blood pressure levels. The second scientist wants to give the drug to 500 people with high blood pressure, and not give the drug to another 500 people with high blood pressure, and see how many in both groups experience lower blood pressure levels. Which is the better way to test this drug?
The General Social Survey (GSS) collects information and keep a historical record of the concerns, experiences, attitudes, and practices of residents of the United States. Since 1972, the GSS has been monitoring societal change and studying the growing complexity of American society. Canada has been running a similar survey since 19851.
The GSS asks the question from the previous slide. Below is the distribution of responses from the 2010 survey:
| All 1000 get the drug | 99 |
|---|---|
| 500 get the drug, 500 don’t | 571 |
| Total | 670 |
Parameter of Interest: Proportion of all Americans who have good intuition about experimental design
p: a population proportion
Parameter of Interest: Proportion of all Americans who have good intuition about experimental design
p: a population proportion
Point Estimate: proportion of sampled Americans who have good intuition about experimental design.
\(\hat{p}\): a sample proportion
What percent of all Americans have good intuition about experimental design, i.e., would answer “500 get the drug 500 don’t”?
We can answer this question using a confidence interval, computed with a bootstrap. We are given:
and told that the data are sampled randomly (so independent).
dat_df <- data.frame(Outcome = c(rep("Y", 571),
rep("N", 670 - 571)))
p_hat <- dat_df |>
observe(response = Outcome, success = "Y", stat = "prop")
boot_dist <- dat_df |>
specify(response = Outcome, success = "Y") |>
generate(reps = 10000, type = "bootstrap") |>
calculate(stat = "prop")
percentile_ci <- get_ci(boot_dist)
percentile_ci# A tibble: 1 × 2
lower_ci upper_ci
<dbl> <dbl>
1 0.825 0.879This range of values, \((0.825, 0.879)\) is a 95% bootstrap percentile confidence interval for the true underlying population proportion, \(p\), of Americans who have correct intuition about experimental design. Or, roughly, \((82.5, 87.9)\) %.
Can you see how 95% of the shaded area is in between the two green lines?
binom.testWe can also compute confidence intervals for \(p\) using the binom.test() function we mentioned recently. There’s on trick to it: unlike the case of hypothesis tests, all confidence intervals we compute in this class will be two-tailed. You have to set this correctly!
This means if you use binom.test() for both a hypothesis test and confidence interval in the same question, you might need to run it twice to get the right information!
Exact binomial test
data: 571 and 670
number of successes = 571, number of trials = 670, p-value < 2.2e-16
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
0.8230751 0.8782558
sample estimates:
probability of success
0.8522388 [1] 0.8230751 0.8782558
attr(,"conf.level")
[1] 0.95Thus a 95% exact binomial confidence interval for \(p\), the true underlying population proportion of Americans who have correct intuition about experimental design, is (0.823, 0.878).
So we have two different ways to access a confidence interval for \(p\):
binom.test().infer to create a bootstrap distribution, then extractThe contents of the latter part of this lecture come exclusively from Chapter 12 of Introduction to Modern Statistics (2e).
