Dimensionality reduction is a fundamental concept in machine learning used to reduce the number of input features (dimensions) in a dataset while preserving as much important information as possible.

Principal Component Analysis (PCA)

Principal Component Analysis (PCA) is a linear dimensionality reduction technique that transforms data into a new coordinate system.

Instead of using the original features, PCA creates new variables called principal components, which are:

Linear combinations of the original features

Ordered by importance (variance explained)

PCA works by identifying directions (called principal axes) where the data varies the most.

The first principal component captures the maximum variance while the second principal component captures the next highest variance.

This allows us to:

Keep only the most informative components

Discard less important ones

How PCA Works

Standardize the data - Features must be scaled (very important for PCA)

Compute the covariance matrix - Shows…