6. Linear regression and diagnostics

Learning goals

You will learn to:

translate a management question into a regression formula;
interpret coefficients in the original business units;
distinguish bivariate from conditional relationships;
interpret confidence intervals, R-squared, and model fit;
inspect linearity, residual patterns, multicollinearity, and influential observations;
distinguish model association from causal claims.

Management question: Which marketing activities are associated with weekly sales after accounting for other activities and market conditions?

From question to model

Outcome+Predictors+Design→Model→Diagnostics→Interpretation

For this case:

outcome: sales_k_eur;
marketing predictors: search, social, email advertising and discount;
market condition: competitor price index.

Import and inspect

library(tidyverse)
library(broom)
library(car)
library(performance)

marketing <- read_csv(
  "data/marketing_mix.csv",
  show_col_types = FALSE
)

glimpse(marketing)

Rows: 156
Columns: 8
$ week                   <dbl> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, …
$ date                   <date> 2023-01-02, 2023-01-09, 2023-01-16, 2023-01-23…
$ search_ads_k_eur       <dbl> 38.5, 42.0, 55.5, 31.6, 48.2, 48.7, 48.3, 48.8,…
$ social_ads_k_eur       <dbl> 41.0, 36.4, 53.8, 23.4, 54.7, 26.6, 34.4, 31.1,…
$ email_ads_k_eur        <dbl> 13.5, 8.6, 14.8, 9.9, 17.9, 8.5, 8.6, 15.0, 6.1…
$ discount_pct           <dbl> 7.3, 5.1, 10.5, 10.3, 5.6, 6.8, 9.0, 4.6, 5.2, …
$ competitor_price_index <dbl> 103.2, 95.0, 99.6, 100.1, 95.7, 102.7, 101.7, 1…
$ sales_k_eur            <dbl> 256.2, 255.2, 322.0, 248.2, 319.7, 270.0, 258.4…

Start with a graph

ggplot(marketing, aes(x = search_ads_k_eur, y = sales_k_eur)) +
  geom_point() +
  geom_smooth(method = "lm", se = TRUE) +
  labs(
    title = "Search advertising and weekly sales",
    x = "Search advertising (thousand EUR)",
    y = "Sales (thousand EUR)"
  )

Bivariate model

model_1 <- lm(
  sales_k_eur ~ search_ads_k_eur,
  data = marketing
)

summary(model_1)


Call:
lm(formula = sales_k_eur ~ search_ads_k_eur, data = marketing)

Residuals:
    Min      1Q  Median      3Q     Max 
-38.067 -14.939  -0.473  11.431 119.275 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)      182.2160     8.1349   22.40   <2e-16 ***
search_ads_k_eur   2.3654     0.1663   14.23   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 20.17 on 154 degrees of freedom
Multiple R-squared:  0.5679,    Adjusted R-squared:  0.5651 
F-statistic: 202.4 on 1 and 154 DF,  p-value: < 2.2e-16

Multiple regression

model_2 <- lm(
  sales_k_eur ~
    search_ads_k_eur +
    social_ads_k_eur +
    email_ads_k_eur +
    discount_pct +
    competitor_price_index,
  data = marketing
)

tidy(model_2, conf.int = TRUE)

# A tibble: 6 × 7
  term                  estimate std.error statistic  p.value conf.low conf.high
  <chr>                    <dbl>     <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
1 (Intercept)            272.       35.6        7.64 2.40e-12 201.       342.   
2 search_ads_k_eur         1.96      0.196      9.99 2.49e-18   1.57       2.35 
3 social_ads_k_eur         0.793     0.203      3.90 1.42e- 4   0.392      1.19 
4 email_ads_k_eur          1.12      0.404      2.78 6.19e- 3   0.323      1.92 
5 discount_pct             0.914     0.490      1.86 6.43e- 2  -0.0548     1.88 
6 competitor_price_ind…   -1.24      0.346     -3.58 4.70e- 4  -1.92      -0.554

glance(model_2)

# A tibble: 1 × 12
  r.squared adj.r.squared sigma statistic  p.value    df logLik   AIC   BIC
      <dbl>         <dbl> <dbl>     <dbl>    <dbl> <dbl>  <dbl> <dbl> <dbl>
1     0.670         0.659  17.9      61.0 1.93e-34     5  -668. 1350. 1371.
# ℹ 3 more variables: deviance <dbl>, df.residual <int>, nobs <int>

A coefficient now describes the estimated difference in the outcome associated with a one-unit predictor change holding the other included predictors constant.

Interpret in business units

If search_ads_k_eur has coefficient (b):

Holding the other included variables constant, an additional €1,000 of search advertising is associated with an estimated (b) thousand euros difference in weekly sales.

Do not replace “associated with” by “causes” unless the design supports that interpretation.

Diagnostics

vif(model_2)

      search_ads_k_eur       social_ads_k_eur        email_ads_k_eur 
              1.780005               1.751556               1.028917 
          discount_pct competitor_price_index 
              1.020504               1.025562

check_model(model_2)

Ask:

Does the relationship appear approximately linear?
Do residuals show a systematic pattern?
Is residual spread reasonably stable?
Are predictors highly redundant?
Is one observation unusually influential?

Influential observations

diagnostics <- augment(model_2) |>
  mutate(row_number = row_number()) |>
  arrange(desc(.cooksd))

diagnostics |>
  select(row_number, .fitted, .resid, .std.resid, .cooksd) |>
  slice_head(n = 5)

# A tibble: 5 × 5
  row_number .fitted .resid .std.resid .cooksd
       <int>   <dbl>  <dbl>      <dbl>   <dbl>
1        121    306.  114.        6.44  0.132 
2        149    334.   28.3       1.64  0.0349
3        156    357.   30.1       1.74  0.0346
4         46    277.   20.0       1.19  0.0293
5         37    304.  -34.3      -1.96  0.0230

Investigate unusual observations rather than deleting them automatically.

A regression table is not the end of an analysis. The substantive conclusion depends on the research question, coefficient size, uncertainty, model specification, diagnostics, and design.

Practice

Fit a model with only the three advertising channels.
Add discount and competitor price index.
Compare the coefficient for search advertising.
Identify the most influential observation.
Write a three-sentence managerial interpretation with one limitation.

Takeaway

Regression estimates conditional relationships. Credible interpretation requires both statistical output and design-aware reasoning.