## Why is modern analytics so successful?

1. [More data](#more-data-for-the-analysis)
2. [More computing power](#computing-power)
3. [New algorithms](#new-algorithms)
4. [New analytics processes](#analytics-processes)

# More data for the analysis {data-stack-name="Data"}

## The explosion of data

::: columns

:::  {.column width="50%"}

<br><br>

```{python}
#| echo: false

import matplotlib.pyplot as plt
import numpy as np

# Known datapoints from literature (IDC Global Datasphere summaries)
years_known = np.array([2010, 2012, 2014, 2016, 2018, 2020, 2022, 2024, 2025])

structured_known = np.array([0.6, 1.0, 1.6, 3.8, 7, 12, 18, 26, 32])
unstructured_known = np.array([1.4, 2.5, 4.4, 12.2, 26, 52, 79, 121, 149])

# Sources referenced in literature:
#
# - Global data volume was ~2 ZB in 2010
# - It increased to ~33 ZB by 2018
# - IDC Global Datasphere reports forecast ~175 ZB by 2025
# - Some summaries report ~181 ZB by 2025
#
# These figures illustrate strongly exponential growth.

# Create full yearly range
years = np.arange(2010, 2026)

# Interpolate missing years
structured = np.interp(years, years_known, structured_known)
unstructured = np.interp(years, years_known, unstructured_known)

totals = structured + unstructured

# Identify interpolated years
is_interpolated = ~np.isin(years, years_known)
is_known = np.isin(years, years_known)

# Fit exponential trendline
log_totals = np.log(totals)
coeffs = np.polyfit(years, log_totals, 1)
trend_raw = np.exp(np.poly1d(coeffs)(years))

# Scale trend so it reaches ~190 ZB in 2025
target_2025 = 190
scale_factor = target_2025 / trend_raw[years == 2025]
trend = trend_raw * scale_factor

plt.figure(figsize=(9,4))

# --- Structured data bars ---
plt.bar(
    years[is_known],
    structured[is_known],
    label="Structured data"
)

plt.bar(
    years[is_interpolated],
    structured[is_interpolated],
    color="gray",
    hatch="//"
)

# --- Unstructured data bars ---
plt.bar(
    years[is_known],
    unstructured[is_known],
    bottom=structured[is_known],
    label="Unstructured data"
)

plt.bar(
    years[is_interpolated],
    unstructured[is_interpolated],
    bottom=structured[is_interpolated],
    color="gray",
    hatch="//"
)

# Exponential trendline
plt.plot(years, trend, color="black", linewidth=2, label="Estimated exponential trend")

plt.xlabel("Year")
plt.ylabel("Zettabytes")
plt.title("Growth of Global Data Volume")

plt.xticks(years, rotation=45)
plt.grid(axis="y")

plt.legend()

plt.show()
```

:::

:::  {.column width="50%"}

<br><br>

| Unit | Equivalent | Approximate meaning |
|-----|------------|---------------------|
| Gigabyte (GB) |  | A HD movie file or a few hundred photos |
| Terabyte (TB) | 1,000 GB | Storage of a modern laptop or external drive |
| Petabyte (PB) | 1,000 TB | Data of a large company or several large data centers |
| Exabyte (EB) | 1,000 PB | Roughly the yearly internet traffic of a small country |
| Zettabyte (ZB) | 1,000 EB | ≈ 1 trillion gigabytes; global data creation scale |

:::

:::

## Data production

**Enterprise and transactional data**  
Enterprise systems such as ERP, CRM, and supply chain platforms generate large volumes of structured data through everyday business transactions.

**E-commerce**  
Every search, click, purchase, and review creates behavioral and transactional data used for recommendations and personalized marketing.

**Social media and user-generated content**  
Platforms such as TikTok, YouTube, and Instagram generate enormous data volumes through uploads, interactions, and live-streaming.

**IoT and smart devices**  
Connected devices—from wearables to industrial sensors—continuously produce real-time data across interconnected systems.

**Digital transactions**  
Online banking, mobile payments, and blockchain systems generate detailed financial records for transactions and security monitoring.

**AI-generated data**  
Machine learning and generative AI create large datasets during training and operation, further accelerating global data growth.


# Computing power {data-stack-name="Computing power"}

## Growth of computing power

The rapid acceleration of computing power—driven by advances in hardware, cloud infrastructure, and parallel processing—has enabled modern analytics and machine learning to scale to massive datasets.

<!-- TODO : could include an illustration of a modern AI data center -->

<br>

![]({{< var shared-materials >}}/images/analytics_machines_development.png){fig-align="center" width="70%"}

## Evolution of computing power

The growth of modern analytics is enabled by **changing strategies for increasing computing power**.

<br>

| Era | Main strategy | Explanation |
|-----|---------------|-------------|
| 1970s–2000s (*) | **Moore’s Law & miniaturization** | Smaller transistors → more components per chip → faster processors |
| 2005–today | **Parallel computing** | Performance increases by using multiple processors simultaneously |
| 2010s–today | **Specialized hardware** | Chips optimized for specific workloads (GPUs, TPUs, AI accelerators) |
| Emerging | **New computing paradigms** | Alternative computing models such as quantum computing |

::: aside
(*) Miniaturization continues today, but progress has slowed as transistors approach physical limits.
Advances such as extreme ultraviolet (EUV) lithography are required to continue scaling.
See how $400 million machines are constructed to build modern chips: [https://www.youtube.com/watch?v=MiUHjLxm3V0](https://www.youtube.com/watch?v=MiUHjLxm3V0).
:::

# New algorithms {data-stack-name="Algorithms"}

## Algorithms

An algorithm is a step-by-step procedure for performing a computation and thereby solving a problem.

Algorithms determine how efficiently computers can process data and solve tasks.

Examples:

- **Linux Scheduling Algorithms** *(1990s–present)* — Efficient process scheduling enabling operating systems to run tasks concurrently
- **RSA Encryption** *(1977)* — Public-key cryptography algorithm enabling secure internet communication
- **PageRank** *(1998)* — Algorithm ranking webpages based on link structure, enabling scalable web search
- **Blockchain** *(2008)* — Distributed consensus algorithm enabling decentralized digital ledgers and cryptocurrencies
- **Deep Learning** *(2010s)* — Improved neural network training enabling major advances in vision, speech, and language AI
- **Gradient Boosting** *(2014)* — High-performance ensemble learning algorithm widely used in predictive analytics and data science
- **GPT / Transformer Models** *(2017–present)* — Transformer architecture enabling large language models and generative AI

::: {.learning_note}

**Learning note**

No need to memorize everything.  
Be able to give a few illustrative examples.  
This also applies to the data production areas.

:::

## Algorithms enable more powerful analytical models

<br><br>

::: columns
::: {.column width="30%"}
![]({{< var shared-materials >}}/images/analytics_traditional_regression.png){width="90%"}

Traditional Regression

:::

::: {.column width="30%"}

![]({{< var shared-materials >}}/images/analytics_decision_tree.png){width="90%"}

Decision Tree
:::

::: {.column width="30%"}
![]({{< var shared-materials >}}/images/analytics_neural_network.png){width="90%"}

Neural Network
:::

:::


## Algorithmic breakthroughs drive AI progress

Recent breakthroughs in artificial intelligence (AI)^[AI systems are based on algorithms; in the following, we use the broader term AI when referring to such algorithm-based systems.] show how new algorithms can rapidly surpass human performance.
**DeepMind** provides good examples.

::: columns

::: column

AlphaGo (2016)

- Uses **deep neural networks + reinforcement learning**
- Defeated world champion **Lee Sedol** in the game of Go

Go was long considered too complex for computers due to the enormous search space.

![]({{< var shared-materials >}}/images/analytics_alphago.png){width=50% fig-align="center"}

:::

::: column

AlphaFold (2020–2022)

- Uses deep learning to predict **protein structures**
- Achieved breakthrough performance in the **CASP competition**^[Hassabis and colleagues were awarded the 2024 Nobel Prize in Chemistry, in part for the development of AlphaFold.]

Predicting protein folding had been a major unsolved problem in biology for decades.

![]({{< var shared-materials >}}/images/analytics_casp.png){width=50% fig-align="center"}

:::

:::


<!--
    ## AlphaGo

    > Go is one of the hardest games in the world for AI because of the huge number of different game scenarios and moves. The number of potential legal board positions is greater than the number of atoms in the universe.

    > The core of AlphaGo is a deep neural network. It was initially trained to learn playing by using a database of around 30 million recorded historical moves. After the training, the system was cloned and it was trained further playing large numbers of games against other instances of itself, using reinforcement learning to improve its play. During this training AlphaGo learned new strategies which were never played by humans.

    > A newer version named AlphaGo Zero skips the step of being trained and learns to play simply by playing games against itself, starting from completely random play.

    ::: {.notes}

    Chess 2^64 legal positions, Go 2^120!!!

    :::

    ## Libratus

    > An artificial intelligence called Libratus has beaten four of the world’s best poker players in a grueling 20-day tournament in January 2017.

    > Poker is more difficult because it’s a game with imperfect information. With chess and Go, each player can see the entire board, but with poker, players don’t get to see each other’s hands. Furthermore, the AI is required to bluff and correctly interpret misleading information in order to win.

    > “We didn’t tell Libratus how to play poker. We gave it the rules of poker and said ‘learn on your own’.” The AI started playing randomly but over the course of playing trillions of hands was able to refine its approach and arrive at a winning strategy.

-->

## Jagged frontier of AI

:::: {.columns}

::: {.column width="60%"}

<br><br>

```{python}
#| echo: false

import numpy as np
import matplotlib.pyplot as plt

# Categories (axes)
labels = [
    "Knowledge",
    "Reading & Writing",
    "Math",
    "Reasoning",
    "Working Memory",
    "Memory Storage",
    "Memory Retrieval",
    "Visual",
    "Auditory",
    "Speed"
]

# Example scores (approximate values from the illustration)
gpt4 = [8, 6, 4, 0, 2, 0, 4, 0, 0, 3]
gpt5 = [9, 10, 10, 7, 4, 0, 4, 4, 6, 3]

# Close the polygon
gpt4 = gpt4 + [gpt4[0]]
gpt5 = gpt5 + [gpt5[0]]

# Compute angles
angles = np.linspace(0, 2*np.pi, len(labels), endpoint=False).tolist()
angles += angles[:1]

# Create figure
# fig, ax = plt.subplots(figsize=(6,6), subplot_kw=dict(polar=True))
fig, ax = plt.subplots(figsize=(6,6), subplot_kw=dict(polar=True))

ax.set_theta_offset(np.pi / 2)   # rotate so first label is at top
ax.set_theta_direction(-1)       # make axes go clockwise

# Plot data
ax.plot(angles, gpt5, linewidth=2, label="GPT-5 (2025)")
ax.fill(angles, gpt5, alpha=0.15)

ax.plot(angles, gpt4, linewidth=2, label="GPT-4 (2023)")
ax.fill(angles, gpt4, alpha=0.15)

# Axis labels
ax.set_xticks(angles[:-1])
ax.set_xticklabels(labels)

# Radial limits
ax.set_ylim(0, 10)

# Grid and legend
ax.set_title("AI Capabilities", pad=20)
ax.legend(loc="upper right", bbox_to_anchor=(1.3, 1.1))

plt.show()
```
:::


::: {.column width="40%"}
<br><br>

::: {.highlight_must_learn}

AI progress often occurs through **algorithmic breakthroughs**, enabling machines to outperform humans in increasingly complex tasks.

Recent studies suggest that AI creates a **“jagged frontier.”** [@DellAcquaEtAl2023], i.e., some tasks are well suited to AI, while others that appear similar remain outside its capabilities. 

- **Within this frontier, AI can strongly enhance knowledge work**, improving productivity and the quality of outputs. 
- **Outside the frontier, AI can reduce performance**, especially when users rely too heavily on its outputs without verification. 

:::

<!--
https://www.agidefinition.ai/
https://www.oneusefulthing.org/p/the-shape-of-ai-jaggedness-bottlenecks
https://www.youtube.com/watch?v=d95J8yzvjbQ
-->

:::

::::

::: aside
Figure based on the work of @HendrycksEtAl2025 on *Artificial General Intelligence*.
:::

# Analytics processes {data-stack-name="Analytics processes"}

## Maturing analytical capabilities


:::: {.columns}

::: {.column width="50%"}

::: {.highlight_must_learn}


**Descriptive analytics: What happened?**  

- Summarizes historical data to understand patterns and trends.  
- *Example: Sales reports, dashboards, KPIs*

**Predictive analytics: What will happen?**  

- Uses statistical models and machine learning to forecast future outcomes.  
- *Example: Demand forecasting, churn prediction*

**Prescriptive analytics: What should we do?**  

- Recommends actions based on predictions and optimization.  
- *Example: Pricing optimization, recommendation systems*

:::

:::


::: {.column width="50%"}

<br><br>
![]({{< var shared-materials >}}/images/analytics_maturity.png){width=100% fig-align=center}

:::

::::
