BMR 617: Statistical Techniques for the Biomedical Sciences

ANOVA and linear regression: Review and Summary

This review includes statistical concepts and R commands from the following lectures:

• One-Way ANOVA
• Two-Way ANOVA
• Linear Regression
• Multiple Linear Regression
• Linear Models
• Sample Size and Power

Summary of hypothesis tests

This is a list of all the hypothesis tests we have seen to date. Note how they are classified by the type of explanatory and response variables.

Test	Response variable	Explanatory variable(s)
T-test	Quantitative	One categorical variable with two levels
χ2 test or Fisher's Exact Test	Categorical	One categorical variable
One-Way ANOVA	Quantitative	One categorical variable with any number of levels
Two-Way ANOVA	Quantitative	Two categorical variables with any number of levels
Linear Regression	Quantitative	Any number of quantitative variables

Two-Way ANOVA

For Two-Way ANOVA we saw that we could include an interaction term. This tests whether or not the two explanatory variables act independently on the response variable. Equivalents, it tests if there is (positive or negative) synergy between the two explanatory variables.

Linear Regression

For linear regression, we also saw that we could handle categorical variables by creating "dummy variables" which take on the value 0 or 1. This way we can turn t-tests and ANOVAs into linear regression.

While the underlying mathematics is essentially the same, t-tests and ANOVAs are used to answer the question "Is there a relationship between the explanatory and response variables?". Linear regression is used to answer the question "Given certain values of the explanatory variables, what is the best prediction for the value of the explanatory variables?"

Care should be taken to only predict within a reasonable range, that is covered by the sample data.

Types of Error

In hypothesis testing, we always run the risk of coming to the wrong conclusion. This is because we are inferring information about the entire population from a sample, which is subject to random effects.

Two different things can go wrong:

1. A Type I Error, or false positive, occurs when we reject the null hypothesis (conclude that our hypothesis of interest is true) when in fact the null hypothesis is true. The chances of a Type I Error are represented by α.
2. A Type II Error, or false negative, occurs when we fail to reject the null hypothesis (our experiment fails to provide sufficient evidence that our hypothsis of interest is true) when in fact the null hypothesis is false. The chances of a Type II Error are represented by β.

Sample Size and Power Calculations

There is a mathematical relationship between the following five quantities:

1. α, the p-value threshold for determining whether or not to reject the null hypothesis. This is the chance of a Type I Error, if the null hypothesis is true.
2. The statistical power, \(1-\beta\). This is the probability our experiment provides sufficient evidence to reject the null hypothesis, assuming the null hypothesis is false.
3. The effect size. This is the difference between the groups in a t-test, or the size of the effect of the parameters in a linear regression, or the odds ratio in a χ² or Fisher's Exact Test.
4. The amount of variability in the data.
5. The sample size.

Therefore, given the value of four of these quantities, we can (at least in theory) compute the other one.

In a sample size calculation, we specify α, the statistical power, and estimates for the variability and effect size. We can then calculate the number of samples needed to attain the specified statistical power.

In a power calculation, we specify α, the sample size, and estimate for the variabilty and effect size. We can then calculate the statistical power (the chances of rejecting the null hypothesis) using the specified sample size.

The estimated effect size should be the smallest effect size that would be considered scientifically interesting. The variability can be estimated from similar experiments, for example pilot studies or similar experiments in the literature.

A good strategy is to make reasonable estimates for the effect size and variability, and compute a sample size. Then, using that sample size, compute the statistical power over a range of effect sizes and variabilities around the one previously used.

Note that sample size and power calculations should be performed as part of the experimental design. They are of no real use after the experiment has been performed.