3. Correlations, confidence intervals, and t-tests

Learning goals

You will learn to:

match a correlation or t-test to the research question and design;
distinguish independent from paired samples;
distinguish two-sided from one-sided hypotheses;
understand why the direction of a one-sided test must be chosen before seeing the result;
interpret the estimated difference, confidence interval, p-value, and effect size together.

Management questions: Are workload and burnout associated? Does management training change performance? If our theory predicts improvement specifically, when would a directional test be defensible?

First choose the design

Two continuous variables

Correlation

Two separate groups

Independent-samples t-test

Same people measured twice

Paired-samples t-test

Correlation

library(tidyverse)
library(effectsize)

employees <- read_csv(
  "data/employee_survey.csv",
  show_col_types = FALSE
)

cor.test(
  employees$workload,
  employees$burnout,
  use = "complete.obs"
)


    Pearson's product-moment correlation

data:  employees$workload and employees$burnout
t = 19.429, df = 478, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 0.6111010 0.7114717
sample estimates:
    cor 
0.66427

Pearson’s (r) summarizes the strength and direction of a linear association.

A correlation does not by itself establish causation.

The management-training experiment

experiment <- read_csv(
  "data/training_experiment.csv",
  show_col_types = FALSE
)

experiment |>
  group_by(group) |>
  summarise(
    n = n(),
    mean_post = mean(post_score),
    sd_post = sd(post_score),
    .groups = "drop"
  )

# A tibble: 2 × 4
  group        n mean_post sd_post
  <chr>    <int>     <dbl>   <dbl>
1 Control    115      65.1    11.3
2 Training   105      68.5    10.1

Always look at the groups before testing them.

Two-sided test

A two-sided test asks whether the means differ in either direction.

[ H_0: - = 0 ]

[ H_1: - ]

two_sided <- t.test(
  post_score ~ group,
  data = experiment,
  alternative = "two.sided"
)

two_sided


    Welch Two Sample t-test

data:  post_score by group
t = -2.3536, df = 217.85, p-value = 0.01948
alternative hypothesis: true difference in means between group Control and group Training is not equal to 0
95 percent confidence interval:
 -6.2358887 -0.5517717
sample estimates:
 mean in group Control mean in group Training 
              65.10522               68.49905

R uses Welch’s two-sample t-test by default when var.equal = FALSE.

One-sided test

A one-sided test asks a directional question. Suppose the hypothesis before collecting or inspecting the outcome is:

[ H_1: > ]

To make the direction unambiguous, create the two vectors explicitly:

training_post <- experiment |>
  filter(group == "Training") |>
  pull(post_score)

control_post <- experiment |>
  filter(group == "Control") |>
  pull(post_score)

one_sided <- t.test(
  training_post,
  control_post,
  alternative = "greater"
)

one_sided


    Welch Two Sample t-test

data:  training_post and control_post
t = 2.3536, df = 217.85, p-value = 0.009742
alternative hypothesis: true difference in means is greater than 0
95 percent confidence interval:
 1.01182     Inf
sample estimates:
mean of x mean of y 
 68.49905  65.10522

Here "greater" means the mean of the first vector is greater than the mean of the second.

Never choose a one-sided test after seeing which direction the sample difference goes. Doing so changes the hypothesis after observing the evidence and invalidates the intended Type-I error control.

When is a one-sided test defensible?

A one-sided test can make sense when:

the theory or decision problem is genuinely directional;
the opposite direction would not lead to the same scientific conclusion;
the direction is specified before looking at the relevant result;
the choice is reported transparently.

When in doubt, a two-sided test is the safer default.

Effect size

cohens_d(
  post_score ~ group,
  data = experiment
)

Cohen's d |         95% CI
--------------------------
-0.32     | [-0.58, -0.05]

- Estimated using pooled SD.

A p-value addresses evidence relative to a null model. Cohen’s (d) describes the magnitude of the group difference in standardized units.

Paired t-test

If the same participants are measured before and after training:

paired_training <- experiment |>
  filter(group == "Training")

paired_result <- t.test(
  paired_training$post_score,
  paired_training$pre_score,
  paired = TRUE,
  alternative = "two.sided"
)

paired_result


    Paired t-test

data:  paired_training$post_score and paired_training$pre_score
t = 19.965, df = 104, p-value < 2.2e-16
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
 6.501155 7.935035
sample estimates:
mean difference 
       7.218095

The pairing is not a software option chosen for convenience; it represents the research design.

Confidence intervals

A confidence interval describes a range of effect sizes compatible with the data and model assumptions. Narrow intervals indicate greater precision than wide intervals.

For a one-sided test, R reports a one-sided confidence bound, which is another reason to be explicit about the direction of the research question.

What should be reported?

A strong results sentence combines:

group means or the estimated difference;
confidence interval;
t statistic and degrees of freedom;
p-value;
effect size;
study design and substantive context.

Statistical significance is not the same as managerial importance. A small effect can be statistically detectable in a large sample, while an important effect can be estimated imprecisely in a small sample.

Practice

Test whether post-training scores differ between groups using a two-sided test.
State a directional hypothesis that would justify "greater".
Run the corresponding one-sided test using explicit vectors.
Compare the p-values and confidence intervals.
Run a paired t-test for the Training group.
Explain why the paired and independent tests answer different questions.

Takeaway

Choose the t-test from the design, choose one-sided versus two-sided from the pre-specified hypothesis, and interpret magnitude and uncertainty—not only the p-value.

Reference

R documentation: t.test()