2. Prepare, describe, and visualize data

Learning goals

You will learn to:

  • check structure, ranges, categories, and missing values;
  • select and filter data;
  • calculate overall and group-level descriptive statistics;
  • interpret mean, median, standard deviation, counts, and proportions;
  • use a histogram and boxplot to see patterns the mean alone can hide.

Management question: What does the employee survey tell us about the workforce before we run any formal statistical test?

Step 1 — Import and inspect

library(tidyverse)
library(janitor)

employees <- read_csv(
  "data/employee_survey.csv",
  show_col_types = FALSE
) |>
  clean_names()

glimpse(employees)
Rows: 480
Columns: 26
$ employee_id            <chr> "E000001", "E000002", "E000003", "E000004", "E0…
$ age                    <dbl> 37, 32, 22, 44, 42, 41, 48, 23, 32, 25, 51, 39,…
$ gender                 <chr> "Man", "Man", "Man", "Woman", "Woman", "Woman",…
$ department             <chr> "Operations", "Operations", "Operations", "Tech…
$ remote_days            <dbl> 3, 3, 1, 1, 3, 0, 3, 0, 3, 4, 2, 0, 4, 1, 2, 2,…
$ tenure_years           <dbl> 9.2, 7.0, 0.7, 4.2, 7.4, 1.6, 1.6, 2.9, 10.0, 6…
$ manager                <chr> "No", "No", "No", "No", "No", "Yes", "No", "Yes…
$ training_group         <chr> "Training", "Training", "Control", "Training", …
$ leadership_1           <dbl> 1, 3, 5, 3, 3, 2, 3, 3, 5, 2, 4, 5, 1, 2, 1, 2,…
$ leadership_2           <dbl> 1, 1, 5, 2, 4, 3, 3, 4, 5, 1, 5, 5, 1, 1, 1, 2,…
$ leadership_3           <dbl> 1, 1, 5, 3, 5, 4, 4, 2, 5, 2, 5, 5, 1, 1, 2, 2,…
$ leadership_4_reverse   <dbl> 5, 5, 1, 4, 1, 3, 3, 4, 1, 5, 2, 1, 3, 4, 4, 3,…
$ engagement_1           <dbl> 2, 3, 5, 4, 5, 3, 5, 2, 5, 3, 4, 5, 3, 2, 4, 5,…
$ engagement_2           <dbl> 2, 1, 3, 5, 4, 4, 5, 1, 5, 2, 4, 5, 4, 2, 3, 1,…
$ engagement_3           <dbl> 1, 1, 4, 4, 4, 4, 5, 2, 5, 2, 5, 5, 4, 1, 3, 3,…
$ engagement_4_reverse   <dbl> 3, 4, 1, 2, 2, 4, 1, 3, 1, 5, 1, 1, 3, 5, 2, 2,…
$ role_clarity_1         <dbl> 1, 1, 4, 4, 5, 4, 3, 2, 3, 3, 5, 3, 2, 1, 1, 2,…
$ role_clarity_2         <dbl> 1, 3, 3, 5, 4, 4, 5, 3, 4, 2, 5, 5, 1, 1, 1, 1,…
$ role_clarity_3         <dbl> 1, 2, 3, 4, 4, 3, 4, 3, 5, 4, 5, 2, 2, 1, 1, 2,…
$ role_clarity_4         <dbl> 1, 4, 3, 3, 3, 2, 5, 5, 5, 3, 4, 4, 2, 1, 1, 3,…
$ workload               <dbl> 3.44, 4.68, 4.68, 4.09, 4.26, 2.25, 2.79, 4.16,…
$ organizational_support <dbl> 2.50, 3.92, 5.09, 3.72, 3.63, 3.97, 5.58, 4.25,…
$ job_satisfaction       <dbl> 2.52, 4.17, NA, 4.87, 3.17, 5.68, 4.86, 3.77, 5…
$ performance            <dbl> 45.1, 43.8, 52.2, 60.3, 59.9, NA, 64.3, 48.1, 6…
$ burnout                <dbl> 49.8, 48.5, 47.4, 47.1, 51.8, 36.3, 32.1, 46.6,…
$ turnover_intention     <dbl> 4.84, 4.18, 3.66, 4.12, 4.62, 2.83, 3.16, 5.26,…

Do not start by calculating a p-value. First ask: what variables do we have, what values are plausible, and what is missing?

Step 2 — Count missing values

employees |>
  summarise(
    missing_performance = sum(is.na(performance)),
    missing_satisfaction = sum(is.na(job_satisfaction))
  )
# A tibble: 1 × 2
  missing_performance missing_satisfaction
                <int>                <int>
1                   8                    8

NA means missing. Missingness is part of the data-generating process and should be investigated rather than silently ignored.

Step 3 — Select variables

employees |>
  select(
    employee_id,
    department,
    training_group,
    job_satisfaction,
    performance
  )
# A tibble: 480 × 5
   employee_id department training_group job_satisfaction performance
   <chr>       <chr>      <chr>                     <dbl>       <dbl>
 1 E000001     Operations Training                   2.52        45.1
 2 E000002     Operations Training                   4.17        43.8
 3 E000003     Operations Control                   NA           52.2
 4 E000004     Technology Training                   4.87        60.3
 5 E000005     Operations Control                    3.17        59.9
 6 E000006     Technology Control                    5.68        NA  
 7 E000007     Finance    Training                   4.86        64.3
 8 E000008     Technology Training                   3.77        48.1
 9 E000009     Operations Training                   5.77        66  
10 E000010     Operations Control                    3.79        50  
# ℹ 470 more rows

select() chooses columns.

Step 4 — Filter observations

employees |>
  filter(department == "Marketing")
# A tibble: 100 × 26
   employee_id   age gender          department remote_days tenure_years manager
   <chr>       <dbl> <chr>           <chr>            <dbl>        <dbl> <chr>  
 1 E000013        37 Woman           Marketing            4          3.6 Yes    
 2 E000015        46 Woman           Marketing            2          1.3 No     
 3 E000017        54 Man             Marketing            3          4.2 No     
 4 E000022        44 Man             Marketing            3          4.2 No     
 5 E000023        43 Woman           Marketing            3          9.1 No     
 6 E000025        36 Woman           Marketing            0          1.8 Yes    
 7 E000039        23 Man             Marketing            5          2.3 Yes    
 8 E000041        45 Non-binary / p… Marketing            2          6.9 No     
 9 E000043        39 Woman           Marketing            0          4   No     
10 E000050        46 Man             Marketing            3          3.9 Yes    
# ℹ 90 more rows
# ℹ 19 more variables: training_group <chr>, leadership_1 <dbl>,
#   leadership_2 <dbl>, leadership_3 <dbl>, leadership_4_reverse <dbl>,
#   engagement_1 <dbl>, engagement_2 <dbl>, engagement_3 <dbl>,
#   engagement_4_reverse <dbl>, role_clarity_1 <dbl>, role_clarity_2 <dbl>,
#   role_clarity_3 <dbl>, role_clarity_4 <dbl>, workload <dbl>,
#   organizational_support <dbl>, job_satisfaction <dbl>, performance <dbl>, …

filter() chooses rows that satisfy a condition.

Step 5 — Count categories

employees |>
  count(department, sort = TRUE)
# A tibble: 5 × 2
  department     n
  <chr>      <int>
1 Operations   125
2 Technology   104
3 Marketing    100
4 Finance       97
5 HR            54

For categorical variables, counts and proportions are often more informative than a mean.

Step 6 — Describe a numeric variable

employees |>
  summarise(
    n = sum(!is.na(performance)),
    mean = mean(performance, na.rm = TRUE),
    median = median(performance, na.rm = TRUE),
    sd = sd(performance, na.rm = TRUE),
    min = min(performance, na.rm = TRUE),
    max = max(performance, na.rm = TRUE)
  )
# A tibble: 1 × 6
      n  mean median    sd   min   max
  <int> <dbl>  <dbl> <dbl> <dbl> <dbl>
1   472  53.9   54.2  9.17    25  82.4

Mean

Arithmetic average. Sensitive to extreme values.

Median

Middle value. Useful when the distribution is skewed.

Standard deviation

Typical spread around the mean, expressed in the original unit.

Step 7 — Compare groups descriptively

employees |>
  group_by(department) |>
  summarise(
    n = n(),
    mean_performance = mean(performance, na.rm = TRUE),
    sd_performance = sd(performance, na.rm = TRUE),
    .groups = "drop"
  )
# A tibble: 5 × 4
  department     n mean_performance sd_performance
  <chr>      <int>            <dbl>          <dbl>
1 Finance       97             53.6           9.56
2 HR            54             51.7           8.35
3 Marketing    100             55.2           8.33
4 Operations   125             52.7           9.81
5 Technology   104             55.7           8.85

group_by() changes the level at which the following summarise() is calculated.

Step 8 — Look at the distribution

ggplot(employees, aes(x = performance)) +
  geom_histogram(binwidth = 5, boundary = 0) +
  labs(
    title = "Distribution of employee performance",
    x = "Performance score",
    y = "Number of employees"
  )

A histogram helps you see shape, centre, spread, gaps, and unusual observations.

Step 9 — Compare distributions, not only means

ggplot(
  employees,
  aes(x = training_group, y = performance)
) +
  geom_boxplot() +
  labs(
    title = "Performance by training group",
    x = NULL,
    y = "Performance score"
  )

A boxplot makes it easier to compare medians and spread across groups.

A descriptive group difference is not yet evidence that the groups differ beyond sampling variation, and it is not automatically causal. Formal inference comes next.

Optional: create a new category

Only after the basic workflow is comfortable:

employees <- employees |>
  mutate(
    work_mode = case_when(
      remote_days == 0 ~ "On-site",
      remote_days <= 3 ~ "Hybrid",
      TRUE ~ "Mostly remote"
    )
  )

Practice

  1. Count employees by department.
  2. Calculate mean and median job_satisfaction.
  3. Calculate mean burnout by department.
  4. Draw a histogram of job_satisfaction.
  5. Draw a boxplot of burnout by manager.
  6. Write one descriptive claim and one claim these results cannot establish.

Common mistakes

  • Treating a coded category as a continuous quantity without justification.
  • Reporting a mean without checking the distribution.
  • Ignoring missing values.
  • Describing a visible graph difference as a causal effect.
  • Writing five transformations in one pipe before understanding any of them.

Takeaway

Good quantitative analysis begins with disciplined inspection and description. Formal models are easier to interpret when you already understand the data.