5. PCA, EFA, and CFA: from many items to a measurement model

Learning goals

You will learn to:

  • explain the difference between PCA, EFA, and CFA;
  • use PCA for dimension reduction;
  • use EFA to explore plausible latent dimensions;
  • use CFA to evaluate a pre-specified factor structure;
  • understand why EFA finds latent dimensions, not hidden groups of people;
  • use separate exploratory and confirmatory samples in a teaching workflow.

Measurement question: Twelve survey items were designed to capture leadership, engagement, and role clarity. Do the data support a simpler structure, what latent structure appears plausible, and does the proposed three-factor theory fit a separate sample?

PCA, EFA, and CFA answer different questions

PCA — reduce

Question: Can many observed variables be summarized by fewer weighted components?

  • uses total variance;
  • components are mathematical combinations of observed variables;
  • useful for compression and exploratory summaries;
  • does not posit latent common causes.

EFA — explore

Question: What latent factor structure could explain the correlations among the observed items?

  • models shared/common variance;
  • number and pattern of factors are explored;
  • rotation helps interpretation;
  • theory still matters.

CFA — test a specified structure

Question: Does a measurement model defined in advance fit the data reasonably well?

  • researcher specifies which items load on which factors;
  • estimates loadings and factor correlations;
  • evaluates model fit and residual evidence;
  • does not “prove” the theory.

Important vocabulary: EFA explores latent dimensions/factors among variables. If the question is “which employees belong to hidden groups?”, that is closer to clustering or mixture modelling, not factor analysis.

A pedagogical sequence, not a compulsory statistical pipeline

This lesson uses PCA → EFA → CFA because the progression is intuitive for beginners:

Reduce→Explore structure→Evaluate theory

In real research, PCA is not a required prerequisite for EFA. Choose the method from the question you are trying to answer.

Prepare the measurement items

library(tidyverse)
library(psych)
library(lavaan)

employees <- read_csv(
  "data/employee_survey.csv",
  show_col_types = FALSE
) |>
  mutate(
    leadership_4 = 6 - leadership_4_reverse,
    engagement_4 = 6 - engagement_4_reverse
  )

item_names <- c(
  "leadership_1", "leadership_2", "leadership_3", "leadership_4",
  "engagement_1", "engagement_2", "engagement_3", "engagement_4",
  "role_clarity_1", "role_clarity_2", "role_clarity_3", "role_clarity_4"
)

measurement_data <- employees |>
  select(all_of(item_names)) |>
  drop_na()

For this teaching exercise we use complete item responses so the measurement concepts remain visible.

Split exploration from confirmation

Using the same data to discover a structure and then “confirm” that structure is circular.

set.seed(2026)

explore_rows <- sample(
  seq_len(nrow(measurement_data)),
  size = floor(nrow(measurement_data) / 2)
)

explore_data <- measurement_data[explore_rows, ]
confirm_data <- measurement_data[-explore_rows, ]

Part A — PCA for dimension reduction

pca_fit <- prcomp(
  explore_data,
  center = TRUE,
  scale. = TRUE
)

summary(pca_fit)
Importance of components:
                         PC1    PC2    PC3     PC4     PC5     PC6     PC7
Standard deviation     2.388 1.2532 1.1163 0.74414 0.70257 0.65685 0.65367
Proportion of Variance 0.475 0.1309 0.1038 0.04615 0.04113 0.03595 0.03561
Cumulative Proportion  0.475 0.6059 0.7098 0.75590 0.79704 0.83299 0.86860
                           PC8     PC9    PC10    PC11    PC12
Standard deviation     0.60261 0.59243 0.57230 0.52889 0.50540
Proportion of Variance 0.03026 0.02925 0.02729 0.02331 0.02129
Cumulative Proportion  0.89886 0.92811 0.95540 0.97871 1.00000

The summary reports the proportion of total standardized variance represented by each principal component.

Scree plot

plot(
  pca_fit,
  type = "l",
  main = "PCA scree plot"
)

Component weights

round(
  pca_fit$rotation[, 1:3],
  2
)
                 PC1   PC2   PC3
leadership_1   -0.32  0.24 -0.30
leadership_2   -0.29  0.25 -0.34
leadership_3   -0.29  0.21 -0.32
leadership_4   -0.31  0.28 -0.28
engagement_1   -0.29  0.13  0.37
engagement_2   -0.30  0.09  0.36
engagement_3   -0.30  0.09  0.33
engagement_4   -0.28  0.19  0.47
role_clarity_1 -0.26 -0.43 -0.12
role_clarity_2 -0.27 -0.42 -0.05
role_clarity_3 -0.27 -0.42 -0.03
role_clarity_4 -0.27 -0.38 -0.05

Large positive or negative weights indicate variables that contribute strongly to a component. The sign itself can flip without changing the substantive component.

PCA is useful when the goal is compression. It does not tell us that a latent psychological or organizational construct caused the observed responses.

Part B — EFA for latent structure

Before extracting factors, inspect whether the correlation structure is usable.

KMO(explore_data)
Kaiser-Meyer-Olkin factor adequacy
Call: KMO(r = explore_data)
Overall MSA =  0.9
MSA for each item = 
  leadership_1   leadership_2   leadership_3   leadership_4   engagement_1 
          0.89           0.90           0.93           0.90           0.91 
  engagement_2   engagement_3   engagement_4 role_clarity_1 role_clarity_2 
          0.92           0.91           0.87           0.90           0.89 
role_clarity_3 role_clarity_4 
          0.89           0.91 
cortest.bartlett(
  cor(explore_data),
  n = nrow(explore_data)
)
$chisq
[1] 1381.526

$p.value
[1] 2.913495e-245

$df
[1] 66

These are diagnostics, not automatic decision rules.

How many factors?

Parallel analysis is usually more defensible than retaining factors only because an eigenvalue exceeds one.

fa.parallel(
  explore_data,
  fa = "fa",
  plot = FALSE
)
Parallel analysis suggests that the number of factors =  3  and the number of components =  NA

Estimate an oblique three-factor solution

efa_fit <- fa(
  explore_data,
  nfactors = 3,
  rotate = "oblimin",
  fm = "minres"
)

print(
  efa_fit$loadings,
  cutoff = .30
)

Loadings:
               MR1    MR2    MR3   
leadership_1    0.875              
leadership_2    0.813              
leadership_3    0.726              
leadership_4    0.820              
engagement_1                  0.700
engagement_2                  0.673
engagement_3                  0.637
engagement_4                  0.900
role_clarity_1         0.773       
role_clarity_2         0.775       
role_clarity_3         0.788       
role_clarity_4         0.680       

                 MR1   MR2   MR3
SS loadings    2.639 2.319 2.171
Proportion Var 0.220 0.193 0.181
Cumulative Var 0.220 0.413 0.594

We use an oblique rotation because leadership, engagement, and role clarity can plausibly be correlated in management settings.

When reading the loadings, look for:

  • several meaningful items per factor;
  • strong primary loadings;
  • limited cross-loadings;
  • theoretically coherent factors.

Do not choose a factor solution only because it “looks cleaner”.

Part C — CFA for a pre-specified theory

Now specify the model before looking at the confirmatory results:

cfa_model <- '
  Leadership =~ leadership_1 + leadership_2 + leadership_3 + leadership_4
  Engagement =~ engagement_1 + engagement_2 + engagement_3 + engagement_4
  RoleClarity =~ role_clarity_1 + role_clarity_2 + role_clarity_3 + role_clarity_4
'

Because the items are 1–5 ordinal responses, we use an estimator suitable for ordered indicators:

cfa_fit <- cfa(
  cfa_model,
  data = confirm_data,
  ordered = item_names,
  estimator = "WLSMV",
  std.lv = TRUE
)

summary(
  cfa_fit,
  fit.measures = TRUE,
  standardized = TRUE
)
lavaan 0.7-2 ended normally after 20 iterations

  Estimator                                       DWLS
  Optimization method                           NLMINB
  Number of model parameters                        63

  Number of observations                           222

Model Test User Model:
                                              Standard      Scaled
  Test Statistic                                20.757      44.047
  Degrees of freedom                                51          51
  P-value (Unknown)                                 NA       0.744
  Scaling correction factor                                  0.662
  Shift parameter                                           12.703
    simple second-order correction                                

Model Test Baseline Model:

  Test statistic                              5069.102    2246.313
  Degrees of freedom                                66          66
  P-value                                           NA       0.000
  Scaling correction factor                                  2.295

User Model versus Baseline Model:

  Comparative Fit Index (CFI)                    1.000       1.000
  Tucker-Lewis Index (TLI)                       1.008       1.004
                                                                  
  Robust Comparative Fit Index (CFI)                         1.000
  Robust Tucker-Lewis Index (TLI)                            1.007

Root Mean Square Error of Approximation:

  RMSEA                                          0.000       0.000
  90 Percent confidence interval - lower         0.000       0.000
  90 Percent confidence interval - upper         0.000       0.032
  P-value H_0: RMSEA <= 0.050                    1.000       0.997
  P-value H_0: RMSEA >= 0.080                    0.000       0.000
                                                                  
  Robust RMSEA                                               0.000
  90 Percent confidence interval - lower                     0.000
  90 Percent confidence interval - upper                     0.046
  P-value H_0: Robust RMSEA <= 0.050                         0.966
  P-value H_0: Robust RMSEA >= 0.080                         0.000

Standardized Root Mean Square Residual:

  SRMR                                           0.031       0.031

Goodness of Fit Index:

  Goodness of Fit Index (GFI)                    1.000            
  90 Percent confidence interval - lower         1.000            
  90 Percent confidence interval - upper         1.000            
                                                                  
  Robust GFI                                                 1.000
  90 Percent confidence interval - lower                     0.982
  90 Percent confidence interval - upper                     1.000

Parameter Estimates:

  Parameterization                               Delta
  Standard errors                           Robust.sem
  Information                                 Expected
  Information saturated (h1) model        Unstructured

Latent Variables:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
  Leadership =~                                                         
    leadership_1      0.854    0.029   29.848    0.000    0.854    0.854
    leadership_2      0.796    0.032   25.165    0.000    0.796    0.796
    leadership_3      0.810    0.032   25.421    0.000    0.810    0.810
    leadership_4      0.811    0.031   25.919    0.000    0.811    0.811
  Engagement =~                                                         
    engagement_1      0.775    0.034   22.654    0.000    0.775    0.775
    engagement_2      0.793    0.034   23.375    0.000    0.793    0.793
    engagement_3      0.828    0.032   25.746    0.000    0.828    0.828
    engagement_4      0.804    0.034   23.585    0.000    0.804    0.804
  RoleClarity =~                                                        
    role_clarity_1    0.810    0.036   22.559    0.000    0.810    0.810
    role_clarity_2    0.756    0.043   17.737    0.000    0.756    0.756
    role_clarity_3    0.833    0.039   21.368    0.000    0.833    0.833
    role_clarity_4    0.731    0.047   15.676    0.000    0.731    0.731

Covariances:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
  Leadership ~~                                                         
    Engagement        0.633    0.053   11.924    0.000    0.633    0.633
    RoleClarity       0.447    0.067    6.648    0.000    0.447    0.447
  Engagement ~~                                                         
    RoleClarity       0.521    0.065    7.972    0.000    0.521    0.521

Thresholds:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
    leadershp_1|t1   -1.023    0.102   -9.983    0.000   -1.023   -1.023
    leadershp_1|t2   -0.382    0.087   -4.409    0.000   -0.382   -0.382
    leadershp_1|t3    0.274    0.085    3.210    0.001    0.274    0.274
    leadershp_1|t4    0.898    0.098    9.170    0.000    0.898    0.898
    leadershp_2|t1   -0.967    0.100   -9.642    0.000   -0.967   -0.967
    leadershp_2|t2   -0.298    0.086   -3.477    0.001   -0.298   -0.298
    leadershp_2|t3    0.357    0.086    4.143    0.000    0.357    0.357
    leadershp_2|t4    1.042    0.103   10.094    0.000    1.042    1.042
    leadershp_3|t1   -0.864    0.097   -8.928    0.000   -0.864   -0.864
    leadershp_3|t2   -0.216    0.085   -2.543    0.011   -0.216   -0.216
    leadershp_3|t3    0.310    0.086    3.610    0.000    0.310    0.310
    leadershp_3|t4    0.832    0.096    8.683    0.000    0.832    0.832
    leadershp_4|t1   -0.832    0.096   -8.683    0.000   -0.832   -0.832
    leadershp_4|t2   -0.251    0.085   -2.943    0.003   -0.251   -0.251
    leadershp_4|t3    0.357    0.086    4.143    0.000    0.357    0.357
    leadershp_4|t4    0.915    0.098    9.289    0.000    0.915    0.915
    engagemnt_1|t1   -1.062    0.104  -10.203    0.000   -1.062   -1.062
    engagemnt_1|t2   -0.494    0.088   -5.599    0.000   -0.494   -0.494
    engagemnt_1|t3    0.136    0.085    1.607    0.108    0.136    0.136
    engagemnt_1|t4    0.832    0.096    8.683    0.000    0.832    0.832
    engagemnt_2|t1   -0.950    0.100   -9.525    0.000   -0.950   -0.950
    engagemnt_2|t2   -0.382    0.087   -4.409    0.000   -0.382   -0.382
    engagemnt_2|t3    0.310    0.086    3.610    0.000    0.310    0.310
    engagemnt_2|t4    0.915    0.098    9.289    0.000    0.915    0.915
    engagemnt_3|t1   -1.123    0.107  -10.519    0.000   -1.123   -1.123
    engagemnt_3|t2   -0.456    0.088   -5.203    0.000   -0.456   -0.456
    engagemnt_3|t3    0.113    0.085    1.339    0.181    0.113    0.113
    engagemnt_3|t4    0.898    0.098    9.170    0.000    0.898    0.898
    engagemnt_4|t1   -0.848    0.096   -8.806    0.000   -0.848   -0.848
    engagemnt_4|t2   -0.274    0.085   -3.210    0.001   -0.274   -0.274
    engagemnt_4|t3    0.298    0.086    3.477    0.001    0.298    0.298
    engagemnt_4|t4    0.898    0.098    9.170    0.000    0.898    0.898
    rol_clrty_1|t1   -1.102    0.106  -10.416    0.000   -1.102   -1.102
    rol_clrty_1|t2   -0.382    0.087   -4.409    0.000   -0.382   -0.382
    rol_clrty_1|t3    0.274    0.085    3.210    0.001    0.274    0.274
    rol_clrty_1|t4    0.950    0.100    9.525    0.000    0.950    0.950
    rol_clrty_2|t1   -1.082    0.105  -10.310    0.000   -1.082   -1.082
    rol_clrty_2|t2   -0.251    0.085   -2.943    0.003   -0.251   -0.251
    rol_clrty_2|t3    0.228    0.085    2.676    0.007    0.228    0.228
    rol_clrty_2|t4    0.816    0.095    8.560    0.000    0.816    0.816
    rol_clrty_3|t1   -0.986    0.101   -9.757    0.000   -0.986   -0.986
    rol_clrty_3|t2   -0.274    0.085   -3.210    0.001   -0.274   -0.274
    rol_clrty_3|t3    0.216    0.085    2.543    0.011    0.216    0.216
    rol_clrty_3|t4    0.950    0.100    9.525    0.000    0.950    0.950
    rol_clrty_4|t1   -1.167    0.109  -10.718    0.000   -1.167   -1.167
    rol_clrty_4|t2   -0.431    0.087   -4.939    0.000   -0.431   -0.431
    rol_clrty_4|t3    0.263    0.085    3.077    0.002    0.263    0.263
    rol_clrty_4|t4    0.864    0.097    8.928    0.000    0.864    0.864

Variances:
                   Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
   .leadership_1      0.270                               0.270    0.270
   .leadership_2      0.367                               0.367    0.367
   .leadership_3      0.344                               0.344    0.344
   .leadership_4      0.342                               0.342    0.342
   .engagement_1      0.399                               0.399    0.399
   .engagement_2      0.371                               0.371    0.371
   .engagement_3      0.314                               0.314    0.314
   .engagement_4      0.353                               0.353    0.353
   .role_clarity_1    0.344                               0.344    0.344
   .role_clarity_2    0.428                               0.428    0.428
   .role_clarity_3    0.305                               0.305    0.305
   .role_clarity_4    0.465                               0.465    0.465
    Leadership        1.000                               1.000    1.000
    Engagement        1.000                               1.000    1.000
    RoleClarity       1.000                               1.000    1.000

Extract a compact set of fit measures:

fitMeasures(
  cfa_fit,
  c("cfi", "tli", "rmsea", "srmr")
)
  cfi   tli rmsea  srmr 
1.000 1.008 0.000 0.031

How to read CFA without threshold worship

CFI, TLI, RMSEA, and SRMR are useful pieces of evidence. They should be interpreted alongside:

  • standardized loadings;
  • residuals;
  • factor correlations;
  • sample size and estimator;
  • theory and measurement purpose.

Do not treat “CFI > .95” or another single cutoff as proof of a valid construct. Fit indices summarize particular discrepancies under particular assumptions.

PCA versus EFA: the key mathematical idea

PCA decomposes total variance into components.

Common-factor analysis represents each observed variable as a combination of:

  • common variance associated with latent factors;
  • unique variance specific to the item and error.

That is why PCA and EFA can look similar in output while answering different questions.

Practice

  1. Run PCA and inspect the first three components.
  2. Run EFA with two factors and with three factors. Which is more interpretable?
  3. Identify one item with a strong loading on each factor.
  4. Fit the three-factor CFA in the confirmatory sample.
  5. Report CFI, TLI, RMSEA, and SRMR.
  6. Write three sentences: what PCA tells you, what EFA adds, and what CFA adds.

Takeaway

PCA reduces. EFA explores. CFA evaluates a specified structure. None of them replaces theory, measurement quality, or careful interpretation.

References and further reading