BMR 617: Statistical Techniques for the Biomedical Sciences

Confidence Intervals for Proportions

Last time:

We saw how to compute confidence intervals for the mean of a data set, assuming the data were sampled from a normal distribution.

The interpretation of the confidence interval, with a confidence level of 95%, is that if we were to repeat the experiment over and over again, and compute the confidence intervals the same way, then 95% of those confidence intervals would capture the "true mean" value.

It is difficult to say this precisely. We can't really talk about "probability" here. The "true mean" is a fixed value that doesn't change. The interval we calculate (in reality we only calculate one interval) either does or does not contain that true value. The "probability the interval contains the true value" is either 1 or 0, accordingly (but we don't know which). The only correct interpretation here is based around imagining repeating the experiment infinitely often. The semantics are difficult to get right.

Nevertheless, confidence intervals are useful to compute. They are arguably the best tool we have for quantifying our uncertainty in estimating the "true value" by using the value from our sample. When interpreting data, it's good practice to think about how you would interpret it if the true value were the value you measured in the sample, as well as if the true value were at either end of the confidence interval.

Confidence Intervals for Proportions

Recall the Pfizer vaccine trial:

		Treatment
		Placebo	Vaccine	Total
SARS-CoV-2 status	Infected	162	8	170
	Not Infected	18163	18190	36353
	Total	18325	18198	36523

We calculated the risk for the placebo group \[\operatorname{Risk}_{\operatorname{placebo}} = \frac{162}{18325} = 0.00884 = 0.884\%\] and the risk for the vaccine group \[\operatorname{Risk}_{\operatorname{vaccine}} = \frac{8}{18198} = 0.000440 = 0.044\%\]

These risks are calculated from samples (a sample of size 18,325 for the placebo risk and a sample of 18,198 for the vaccine risk). What we really want to know is the risk for the entire population. As we did with the mean cholesterol level for our experimental mice, we treat the risk calculated for the sample as an estimate of the risk of the population from which the sample was drawn.

We can, at least in theory, compute a 95% confidence interval for this risk. The interpretation, say for the placebo group, is that if we were to repeatedly take a sample of 18,325 adults, give them the placebo, follow them for two weeks, compute the proportion who contracted COVID-19, and then compute the confidence interval, 95% of those confidence interval would include the "true population proportion" who would contract COVID-19 in two weeks with the placebo.

When working with proportions, however, there is no nice closed formula for computing these, as there is for means of normally-distributed data. There are many methods for approximating the confidence interval.

Confidence Intervals for Proportions in R

The R package binom assembles various methods for computing confidence intervals for proportions. Install the package (you will only need to do this once, unless you reinstall R itself):

	install.packages("binom")
	

and load the library (you need to do this once per R session in which you use the package):

	library(binom)
	

We can compute a confidence interval for the a proportion using binom.confint(x, n, conf.level). The default conf.level is 0.95. So, for example, to compute the 95% confidence interval for the risk of COVID-19 infection for the placebo group, we can do

	
	binom.confint(162, 18325)
	
	

How many different methods does this use?

How well do the methods agree on the confidence interval?

Choosing the method

Understanding all the different methods here is highly technical and beyond the scope of this course.

One common method is the asymptotic method. This relies on the central limit theorem, and essentially approximates the proportion using a normal distribution. This works well, particularly for large sample sizes. However, normally distributed data can take any positive or negative value. Proportions can only have values between 0 and 1. So the asymptotic method can be inaccurate if the proportion is close to 0 or close to 1.

The "exact" method computes a confidence interval that is guaranteed to be correct for some confidence level greater than or equal to the requested one. So, despite its name, it is still an approximation, but it is guaranteed always to be a conservative approximation (i.e. the intervals may be wider than the "correct" ones).

As a general rule of thumb, the asymptotic method is the most common choice, unless the sample proportion is less than 0.1 or greater than 0.9. In those cases use the "exact" method.

Specifying the method

You can specify the method using

	binom.confint(8, 18198, methods="exact")
	

The value for methods can be any of the 11 different methods listed if you don't provide a value for that parameter.

Interpreting the confidence intervals

Using the exact method, the risk for the placebo group is 0.882% with a 95% confidence interval of \([0.752\%, 1.03\%]\). The risk for the vaccine group is 0.0440% with a 95% confidence interval of \([0.0190\%, 0.0866\%]\). In the samples, the risk for the placebo group is about 20 times higher than the risk for the vaccine group. Even if we take the extremes of the 95% confidence intervals, the lowest risk for the placebo group is still 8.7 times higher than the highest risk for the vaccine group. Even considering the extremes of the 95% confidence intervals, it is clear that the vaccine reduces the risk of contracting COVID-19.

Confidence intervals for other statistics

We can compute confidence intervals for any statistic we calculate from our sample. In theory, we could compute a 95% confidence interval for the relative risk. However, in this example, this is particularly tricky as the individual risks are close to zero. We will revisit this later in the course when we look at hypothesis testing for these data, and consider a related quantity called the odds ratio.