BMR 617: Statistical Techniques for the Biomedical Sciences

Inference: Matched Pairs t-test (Paired t-test)

This type of t-test is used to compare two dependent means of quantitative variables. Here "dependent" means that the underlying values are dependent on some other variable. It is also called repeated-measures t-test, paired samples t-test, or matched samples t-test. Paired t-test assumes that the population standard deviation of the paired differences is unknown and will be estimated through the data.

Recall the pooled and unpooled two-sample t-tests.

We will use the following variables in this section:

H₀: null hypothesis
H_a: alternative hypothesis
μ₁: mean of population 1
μ₂: mean of population 2
x₁ and x₂: computed (observed) sample means of groups 1 and 2, respectively
n₁ and n₂: sample sizes of groups 1 and 2, respectively
s₁² and s₂²: computed sample variances of groups 1 and 2, respectively
s_p²: computed common sample variance (estimator of the pooled variance of groups 1 and 2)
s_p: computed common sample standard deviation

Classical t-test ("pooled two-sample t-test") is used when the variance of two groups being compared are equivalent.

The test statistic value t, i.e., [(sample mean difference – population mean difference)/standard error], would be $$ {t = \frac{\bar{x_1} - \bar{x_2} - Δ_0}{\sqrt{s_p^2({\frac{1}{n_1} + \frac{1}{n_2}})}} = \frac{\bar{x_1} - \bar{x_2}}{s_p\sqrt{{\frac{1}{n_1} + \frac{1}{n_2}}}}} $$ where μ₁-μ₂ is replaced by the null value Δ₀, which is equal to zero.

H₀: μ₁ = μ₂ or μ₁ - μ₂ = 0
H_a: μ₁ ≠ μ₂ or μ₁ - μ₂ ≠ 0

H₀: μ₁ ≤ μ₂ or μ₁ - μ₂ ≤ 0
H_a: μ₁ > μ₂ or μ₁ - μ₂ > 0

H₀: μ₁ ≥ μ₂ or μ₁ - μ₂ ≥ 0
H_a: μ₁ < μ₂ or μ₁ - μ₂ < 0

Alternative Hypothesis	Rejection Region for Level α Test
H_a: μ₁ ≠ μ₂ or μ₁ - μ₂ ≠ 0	either t ≥ t_α/2,n1+n2-2 or t ≤ -t_α/2,n1+n2-2 (two-tailed test)
H_a: μ₁ > μ₂ or μ₁ - μ₂ > 0	t ≥ t_α,n1+n2-2 (upper-tailed test)
H_a: μ₁ < μ₂ or μ₁ - μ₂ < 0	t ≤ t_α,n1+n2-2 (lower-tailed test)

Welch t-test ("unpooled two-sample t-test") is used when the variance of two groups being compared are different from each other.

The test statistic is a Welch t-statistic $$ {t = \frac{\bar{x_1} - \bar{x_2}}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}} $$ H₀: μ₁ = μ₂ or μ₁ - μ₂ = 0
H_a: μ₁ ≠ μ₂ or μ₁ - μ₂ ≠ 0

H₀: μ₁ ≤ μ₂ or μ₁ - μ₂ ≤ 0
H_a: μ₁ > μ₂ or μ₁ - μ₂ > 0

H₀: μ₁ ≥ μ₂ or μ₁ - μ₂ ≥ 0
H_a: μ₁ < μ₂ or μ₁ - μ₂ < 0

In some references, they use ν to represent the degrees of freedom of an “unpooled” (two-sample) t-test.

Alternative Hypothesis	Rejection Region for Level α Test
H_a: μ₁ ≠ μ₂ or μ₁ - μ₂ ≠ 0	either t ≥ t_α/2,df or t ≤ -t_α/2,df (two-tailed test)
H_a: μ₁ > μ₂ or μ₁ - μ₂ > 0	t ≥ t_α,df (upper-tailed test)
H_a: μ₁ < μ₂ or μ₁ - μ₂ < 0	t ≤ t_α,df (lower-tailed test)

Paired t-test

As said earlier, paired t-test is used to compare two dependent means of quantitative variables. It is very useful in comparing results from one experimental unit.

The paired t-test is used when the data in the two different conditions to be compared for pairs in a way determined naturally by the experimental design. This can be very useful for "before and after" experimental designs, where some measurement is taken from the same experimental subjects at two different time points.

Notice that this means that you must have the same number of samples in both groups. However, that is not sufficient to use a paired t-test: you must also have some natural pairing between each sample in one group and each sample the other group.

Formally speaking, the null and alternative hypotheses would be:

Null Hypothesis	Alternative Hypothesis	Rejection Region for Level α Test
H₀: μ_D = Δ₀	H_a: μ_D ≠ Δ₀	either t ≥ t_α/2,n-1 or t ≤ -t_α/2,n-1 (two-tailed test)
H₀: μ_D ≤ Δ₀	H_a: μ_D > Δ₀	t ≥ t_α,n-1 (upper-tailed test)
H₀: μ_D ≥ Δ₀	H_a: μ_D < Δ₀	t ≤ t_α,n-1 (lower-tailed test)

where
H₀: null hypothesis
H_a: alternative hypothesis
D = X_pre - X_post = difference between the pre (1^st) and post (2^nd) observations within a pair
μ_D = mean difference between pairs of observations
Δ₀ = null hypothesized value
t = test statistic

The test statistic is a t-statistic $$ {t = \frac{\bar{d} - \Delta_0}{\frac{s_d}{\sqrt{n}}}} $$ where
t = test statistic
d = computed mean difference
Δ₀ = null hypothesized value
s_d = computed sample standard deviation
n = sample size

Untangling the formal language here, if $x_1,...x_n$ are the values in one group, and $y_1,...y_n$ are the values in the other group, and each $x_i$ is paired with $y_i$, we compute the differences, $d_i=y_i-x_i$ and then compare the average of the differences $d_1,...d_n$ to zero.

In other words, a paired t-test is a one-class t-test on the differences between the pairs.

Notice that this effectively divides our sample size in half! This means a paired t-test can lose statistical power, so it is only worth doing if there is some variability between the underlying experimental subjects.

Paired t-test in R using a Real Data Set

We'll use some more of the Renal cancer data to investigate paired t-tests. This data set contains quantified Western blot data from seven patients, where the tumor samples and adjacent normal kidney samples were collected from each patient. Western blots were used to measure the protein level for each sample (data were quantified via image density from the blots and them normalized to β-actin, which is assumed to have fairly constant expression for across tissues.

Download the data using


rwd <- read_csv("https://denvirlab.marshall.edu/BMR617-2024/data/RenalWesternData.csv")

And filter (for now) for just the data for the mTOR protein:


mTor <- filter(rwd, Protein=="mTOR")

View the data. You should see


# A tibble: 14 × 4
   Expression Protein Patient TissueType
                    
 1   3148978  mTOR    N1      Normal    
 2   4637102. mTOR    N1      Tumor     
 3   5217165. mTOR    N2      Normal    
 4   4313804. mTOR    N2      Tumor     
 5   6260963. mTOR    N3      Normal    
 6  15465791. mTOR    N3      Tumor     
 7   5923589. mTOR    N4      Normal    
 8   6668831. mTOR    N4      Tumor     
 9   2492781. mTOR    N5      Normal    
10  10879041. mTOR    N5      Tumor     
11   3825333. mTOR    N6      Normal    
12   3842864. mTOR    N6      Tumor     
13   1053525. mTOR    N7      Normal    
14   4273393. mTOR    N7      Tumor

Typically, this is actually a good format in which to analyze the data (it is tidy). For our exploration, though, it will be convenient for us to arrange these data into two columns, one for tumor expression and one for normal expression. You can do this using pivot_wider, which can be used to separate a column into multiple columns (making the table wider) based on the value in a different column:


mTor <- pivot_wider(mTor, names_from=TissueType, values_from=Expression)

View mTor again. You should see


# A tibble: 7 × 4
  Protein Patient   Normal     Tumor
                
1 mTOR    N1      3148978   4637102.
2 mTOR    N2      5217165.  4313804.
3 mTOR    N3      6260963. 15465791.
4 mTOR    N4      5923589.  6668831.
5 mTOR    N5      2492781. 10879041.
6 mTOR    N6      3825333.  3842864.
7 mTOR    N7      1053525.  4273393.

Let's also add another column, with the difference between expression in the tumor and expression in the adjacent normal tissue:


mTor <- mutate(mTor, Diff=Tumor-Normal)

View the data again to make sure it looks the way you expect.

First, let's run a regular t-test between the tumor and normal expression, ignoring the fact that the values are naturally paired because each pair of values comes from a single patient. We'll use pull(mTor, Tumor) to get the values from the Tumor column (and similarly for the normal tissue). There are other ways to do this if you prefer.


t.test(pull(mTor, Tumor), pull(mTor, Normal))

What is the p-value?

Now, instead, let's run a paired t-test. Here the t-test will pair the first tumor expression value with the first normal expression value, and the second tumor expression value with the second normal expression value, etc.


t.test(pull(mTor, Tumor), pull(mTor, Normal), paired=TRUE)

What has happened to the p-value?

To convince yourself that what is really happening here is that we are doing a one-class t-test on the differences, let's explicitly perform that test and see that we get the same thing:


t.test(pull(mTor, Diff), mu=0)

Note you would not really do this in real life; it just helps to see what is happening under the hood.

Visualizing paired data

To visualize paired data, we need to plot the data, but we need a way of visualizing which data points are paired. A geom_line layer, creating lines connecting the pairs of points, can be useful for this.

Let's return to our "tidy" version of the data:


mTor <- filter(rwd, Protein=="mTOR")

We can create a scatter plot of the data using


ggplot(mTor, aes(x=TissueType, y=Expression)) + geom_point()

To join corresponding points, we can add a geom_line layer. The x-value for the points in the line will be the tissue type, and the y-value will be the expression; these aesthetics will be inherited from the ggplot layer.

To create a separate line for each pair, we need to add a group aesthetic to geom_line. The pairs in the data set are defined by the Patient column, so we can do:


ggplot(mTor, aes(x=TissueType, y=Expression)) + geom_point() + 
  geom_line(aes(group=Patient))

Does this give a reasonable visualization of the changes in expression between tumor and normal samples, per patient? How consistent are those changes? Does it make sense to you, looking at the graph, that the result of the t-test is not statistically significant?

Exploration

Analyze the remaining proteins using paired t-tests.
See if you can create a visualization of these data for all proteins in a single graph.

References

Devore, J.L. (2010). Probability and Statistics for Engineering and the Sciences (Eighth ed). Cengage Learning, Boston, MA, USA. https://faculty.ksu.edu.sa/sites/default/files/probability_and_statistics_for_engineering_and_the_sciences.pdf

Motulsky, H. (2018). Intuitive biostatistics : a nonmathematical guide to statistical thinking (Fourth edition. ed.). New York: Oxford University Press. pp. 318-328.

Elston, R.C. and Johnson, W.D. (2008). Basic Biostatistics for Geneticists and Epidemiologists: A Practical Approach. John Wiley & Sons Ltd, West Sussex, UK.