---
session: "Session 3"
title: "Exploratory Data Analysis"
session_id: session-03
format:
  html: default
---


In this exercise, we cover:

| Exercise part | Time (min) |
|---|---:|
| [Business context](#context) | 5 |
| [Clustering tasks](#clustering) | 30 |
| [Wrap-up](#wrap-up) | 5 |
| **Overall** | **40** |

Work directly in this Quarto (`.qmd`) exercise using R in RStudio, Positron, or VS Code. Work through the tasks in R and compare your work with the solution.

## Package setup

Before starting this exercise, make sure the required R packages are installed.

Run this command once in the R console. You do not need to run it every time you open the exercise. You can skip this step if these packages are already installed.

```{r}
#| eval: false
install.packages("tidyverse")
```


## Business context {#context}

An organization wants to understand whether employees show recurring **work profiles**.
Instead of defining employee types in advance, we use clustering to identify groups with similar working patterns.

Dataset: `employee_work_profiles.csv`

```{r}
#| output: false

library(tidyverse)

employee_work_df <- read_csv(
  "https://raw.githubusercontent.com/fs-ise/machine-learning-for-big-data/main/exercises/data/employee_work_profiles.csv",
  show_col_types = FALSE
)

employee_work_df |>
  slice_head(n = 5)
```

The dataset contains measures such as weekly meeting hours, weekly focus hours, the share of work performed remotely, performance, and job satisfaction.


## Clustering tasks {#clustering}

### 1. Prepare the data for clustering

We want the clusters to represent **how employees work**, rather than their outcomes.
Use the following three variables for clustering:

- `meeting_hours`
- `focus_hours`
- `remote_share`

Create a data frame `X` containing these variables:

```{r}
X <- employee_work_df |>
  select(meeting_hours, focus_hours, remote_share)

# Print X at the end.
X
```

Before clustering, do a quick exploratory check so you know what kind of patterns you are looking for.

```{r}
summary(X)
```

```{r}
X |>
  pivot_longer(
    cols = everything(),
    names_to = "variable",
    values_to = "value"
  ) |>
  ggplot(aes(x = value)) +
  geom_histogram(bins = 30) +
  facet_wrap(~ variable, scales = "free") +
  labs(
    x = NULL,
    y = "Count"
  )
```

::: {.callout-note}
`pivot_longer()` puts the variable names into one column so that
`facet_wrap(~ variable)` can create one panel per variable.
:::

Considering that k-means is sensitive to different measurement scales, we need to standardize the values.
This ensures that the three variables have approximately equal influence on the clusters. Base R's `scale()` centers each variable around zero and scales it to unit standard deviation.

Run the following code and observe how the scale of the variables changes:

```{r}
X_scaled <- scale(X)
head(X_scaled)
```

Before running a clustering algorithm, view two of the standardized variables in a scatterplot:

```{r}
X_scaled_df <- as.data.frame(X_scaled)

ggplot(
  X_scaled_df,
  aes(x = meeting_hours, y = focus_hours)
) +
  geom_point() +
  labs(
    title = "Standardized work-pattern data (no clustering)",
    x = "Meeting hours (scaled)",
    y = "Focus hours (scaled)"
  )
```

How many clusters would you expect? Briefly justify your answer based on the plot.

**Your answer:** *Write your answer here.*

### 2. Apply k-means clustering

Next, run the k-means algorithm for different values of `k`.
Use a `for` loop to test values from **2 to 10**. For each value of `k`, create a scatterplot showing the resulting clusters.

Remember that the clustering itself uses all three variables in `X_scaled`, even though the plot displays only `meeting_hours` and `focus_hours`.

```{r}
# YOUR CODE HERE
# Use a for loop to try k = 2, 3, ..., 10.
# For each k:
# 1. run kmeans()
# 2. add the cluster labels to X_scaled_df
# 3. create and print a scatterplot
```

Which number of clusters seems appropriate? Consider both the separation of the groups and whether adding more clusters reveals meaningful additional structure.

**Your answer:** *Write your answer here.*

### 3. Add cluster labels to the dataset

Based on your result above, fit a final k-means model and assign the cluster labels returned by `kmeans()` to the original data frame (`employee_work_df`).
Store the labels as a factor.

```{r}
# YOUR CODE HERE
```

### 4. Compare outcomes across the employee profiles

The clusters were created only from work-pattern variables. We can now ask whether the resulting employee profiles also differ in outcomes.

Create grouped boxplots to compare **performance** and **job satisfaction** across the clusters.
Try to complete the task using the [`ggplot2` boxplot documentation](https://ggplot2.tidyverse.org/reference/geom_boxplot.html).

```{r}
# YOUR CODE HERE
```

What do the boxplots tell you about the employee profiles represented by the clusters?

**Your answer:** *Write your answer here.*




## Wrap-up {#wrap-up}

🎉🎈 You have completed the exercise—good work! 🎈🎉

In this exercise, we have learned to

* Explore distributions and relationships in employee work data
* Select and standardize variables for clustering
* Compare different numbers of clusters using a `for` loop
* Apply and interpret k-means clustering
* Use variables not included in the clustering to characterize the resulting profiles