All About t-Tests

Author

Dr. Anisha Singh

Dataset

This script uses a dataset of reading scores from 72 participants assigned to one of two intervention types. Download it and save it in the same folder as your R script before running any code.

↓ scores.csv

Variables: scores (reading score), intervention (1 or 2) · 72 observations, 36 per group

Exploring the Data

Always start by loading and inspecting the dataset before running any tests.

library(readr)
scores <- read_csv("scores.csv")

head(scores)
summary(scores)

# A tibble: 6 × 2
  scores intervention
   <dbl>        <dbl>
1    170            1
2    175            1
3    165            1
4    180            1
5    160            1
6    158            1

     scores       intervention
 Min.   :152.0   Min.   :1.0
 1st Qu.:170.8   1st Qu.:1.0
 Median :182.5   Median :1.5
 Mean   :184.5   Mean   :1.5
 3rd Qu.:198.2   3rd Qu.:2.0
 Max.   :228.0   Max.   :2.0

One-Sample t-Test

A one-sample t-test asks whether a sample mean is significantly different from a known or hypothesized population mean (\(\mu_0\)). The test statistic is:

\[t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}\]

The key difference from the z-test is that we use the sample standard deviation \(s\) instead of a known population \(\sigma\). This introduces more uncertainty, which is accounted for by the t-distribution (discussed further at the end of this script).

One-Tailed Test

A one-tailed test has a directional alternative hypothesis: we predict the mean is either greater than or less than \(\mu_0\).

Is the sample mean significantly higher than a national average reading score of 150?

## H0: mean = 150
## H1: mean > 150  (one-tailed, upper)
t.test(scores$scores, mu = 150, alternative = "greater")

    One Sample t-test

data:  scores$scores
t = 16.41, df = 71, p-value < 2.2e-16
alternative hypothesis: true mean is greater than 150
95 percent confidence interval:
 180.9963      Inf
sample estimates:
mean of x
    184.5

The t-statistic of 16.41 is far into the upper tail, and the p-value is essentially zero. We reject H₀ and conclude that the sample mean (184.5) is significantly higher than 150.

Two-Tailed Test

A two-tailed test has a non-directional alternative: we only predict that the mean is different from \(\mu_0\).

## H0: mean = 150
## H1: mean ≠ 150  (two-tailed)
t.test(scores$scores, mu = 150, alternative = "two.sided")

    One Sample t-test

data:  scores$scores
t = 16.41, df = 71, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 150
95 percent confidence interval:
 180.3081 188.6919
sample estimates:
mean of x
    184.5

The t-statistic is the same (16.41) — only the CI and p-value calculation differ. The 95% CI [180.31, 188.69] does not include 150, confirming the result.

One-tailed vs. two-tailed: which to use?

Use a two-tailed test unless you have a strong, pre-registered directional prediction. Two-tailed tests are more conservative and are the default in most psychological research. Switching to one-tailed after seeing your data inflates Type I error.

Two-Sample t-Test (Independent Samples)

A two-sample independent t-test compares the means of two separate, unrelated groups. We look at the sampling distribution of the difference between means.

There are two versions depending on whether we assume the two groups have equal population variances.

Equal Variances Assumed

## H0: mean(group 1) = mean(group 2)
## H1: means are not equal
t.test(scores$scores ~ scores$intervention, var.equal = TRUE)

    Two Sample t-test

data:  scores$scores by scores$intervention
t = 4.5757, df = 70, p-value = 2e-05
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
  9.590143 24.409857
sample estimates:
mean in group 1 mean in group 2
            193             176

Unequal Variances (Welch’s t-Test)

## Welch correction: does not assume equal variances
t.test(scores$scores ~ scores$intervention, var.equal = FALSE)

    Welch Two Sample t-test

data:  scores$scores by scores$intervention
t = 4.5757, df = 66.636, p-value = 2.125e-05
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
  9.58356 24.41644
sample estimates:
mean in group 1 mean in group 2
            193             176

Equal vs. unequal variances: Welch’s correction

When variances are unequal, the Welch version adjusts the degrees of freedom downward (here, from 70 to 66.636) to compensate for the additional uncertainty. The t-statistic stays the same, but the p-value changes slightly.

In practice: Welch’s t-test (var.equal = FALSE) is recommended as the default. It performs just as well as the equal-variance test when variances are equal, and it is more accurate when they are not. Always run Levene’s test first (see below) to check the assumption.

Paired Samples t-Test

A paired t-test is used when the same participants (or matched pairs) provide scores under two conditions. Instead of comparing two independent groups, we compute the difference score for each participant and test whether the mean difference is zero.

The formula is the same as the one-sample t-test, applied to the difference scores:

\[t = \frac{\bar{d}}{s_d / \sqrt{n}}\]

Here we use data from the mindfulness stress study on the slides. Stress was measured before and after one month of mindfulness practice for 10 participants.

before <- c(195, 198, 201, 187, 205, 155, 187, 190, 234, 187)
after  <- c(194, 190, 200, 200, 152, 185, 190, 220, 180, 180)

## H0: mean difference = 0
## H1: stress is higher before than after (one-tailed: before > after)
t.test(before, after, alternative = "greater", paired = TRUE)

    Paired t-test

data:  before and after
t = 0.52277, df = 9, p-value = 0.3069
alternative hypothesis: true mean difference is greater than 0
95 percent confidence interval:
 -12.0314      Inf
sample estimates:
mean difference
            4.8

The mean stress score dropped by 4.8 points on average, but this difference is not statistically significant (t = 0.52, p = .31). With only 10 participants and substantial variability in stress change across individuals, we do not have enough evidence to conclude that mindfulness reduced stress.

Levene’s Test: Can We Assume Equal Variances?

Before choosing between the equal-variance and Welch versions of the independent t-test, check whether the two groups have similar variance using Levene’s test.

H₀: The variances are equal (homogeneity of variance holds)
H₁: The variances are not equal

A significant result means the equal-variance assumption is violated — use Welch’s t-test.

## Install lawstat once if you haven't already:
## install.packages("lawstat")
library(lawstat)

levene.test(scores$scores, group = scores$intervention)

Reading the Levene output

Look at the p-value. If p > .05, you fail to reject H₀ and may proceed with the equal-variance t-test. If p < .05, variances differ significantly — use Welch’s (var.equal = FALSE).

For this dataset, the group SDs are 17.44 (Intervention 1) and 13.88 (Intervention 2). The groups have similar spreads, so we would expect the Levene test to be non-significant.

Visualizing the Results

Box Plot

A box plot shows the median, interquartile range, and any outliers for each group.

with(scores, boxplot(scores ~ intervention,
                     xlab = "Intervention Type",
                     ylab = "Reading Scores",
                     main = "Reading Scores by Intervention"))

Bar Graph with Error Bars

Bar graphs with error bars (showing ± 1 standard error) are standard for reporting group means in psychology.

## First, compute the descriptive statistics needed for the plot
aggregate(scores$scores, list(scores$intervention), FUN = mean)
aggregate(scores$scores, list(scores$intervention), FUN = sd)
table(scores$intervention)

  Group.1   x
1       1 193
2       2 176

  Group.1        x
1       1 17.44379
2       2 13.87907

 1  2
36 36

## SE = SD / sqrt(n)
17.44379 / sqrt(36)   # SE for Intervention 1
13.87907 / sqrt(36)   # SE for Intervention 2

[1] 2.907298
[1] 2.313178

## Build the bar graph
library(ggplot2)

intervention.names <- c("Intervention 1", "Intervention 2")
mean.scores        <- c(193, 176)
se.scores          <- c(2.907298, 2.313178)
scores.dataframe   <- data.frame(intervention.names, mean.scores, se.scores)

ggplot(scores.dataframe, aes(x = intervention.names, y = mean.scores)) +
  geom_bar(stat = "identity", width = 0.4, color = "black", fill = "white") +
  geom_errorbar(aes(ymin = mean.scores - se.scores,
                    ymax = mean.scores + se.scores),
                width = 0.1) +
  labs(x = "Intervention Type", y = "Reading Scores")

Reading error bars

Each error bar extends ±1 standard error above and below the bar. When error bars from two groups do not overlap, this is a visual signal (though not a formal test) that the group means may differ significantly.

t-Critical Values and the Link to z

Use qt() to find the t-critical value for any combination of alpha and degrees of freedom — the value your t-statistic must exceed to reach significance.

## qt(p, df, lower.tail = FALSE)
## p = alpha/2 for a two-tailed test (0.025 at alpha = .05)
## df = degrees of freedom

qt(0.025,    5, lower.tail = FALSE)
qt(0.025,   10, lower.tail = FALSE)
qt(0.025,   25, lower.tail = FALSE)
qt(0.025,   30, lower.tail = FALSE)
qt(0.025,  100, lower.tail = FALSE)
qt(0.025, 1000, lower.tail = FALSE)

[1] 2.570582
[1] 2.228139
[1] 2.059539
[1] 2.042272
[1] 1.983972
[1] 1.962339

As df increases, t approaches z

Notice the pattern: as degrees of freedom grow, the t-critical value falls toward 1.96 — the familiar z-critical value for a two-tailed test at α = .05.

df	t-critical (α = .05, two-tailed)
5	2.571
10	2.228
25	2.060
30	2.042
100	1.984
1000	1.962
∞ (z)	1.960

This is not a coincidence. The t-distribution has heavier tails than the normal distribution to account for the extra uncertainty introduced by estimating the population SD from a small sample. As n (and therefore df) increases, that estimate becomes more precise and the t-distribution gradually narrows until it is indistinguishable from z.

You can verify this with qnorm():

## The z-critical value (equivalent to t with infinite df)
qnorm(0.025, lower.tail = FALSE)

[1] 1.959964

With 1000 degrees of freedom the t-critical value (1.962) is already within 0.002 of the z-critical value (1.960). By the time you have a few hundred participants, the practical difference between a t-test and a z-test is negligible — the reason we use t is that for small samples, those extra decimal points matter.