Principal Component Analysis (PCA)

1. Formula
   Observation

   A pink point is labeled $x$ in the projection diagram.

2. Audio
   Observation

   The narrator says, "When you project a point x on a unit vector u..."

x

A point or data vector being projected.

A vector in the ambient space shown by the 3D axes; the video does not state an explicit dimension for this symbol.

1. Formula
   Observation

   A yellow arrow from the origin is labeled $u$.

2. Audio
   Observation

   The narrator calls it "a unit vector u".

u

The unit direction vector defining the line of projection.

Unit vector; later the video explicitly imposes $u^T u = 1$.

1. Formula
   Observation

   The projected point on the line through $u$ is labeled $x'$.

2. Audio
   Observation

   The narrator says the projection gives "a new point x prime".

x'

The orthogonal projection of $x$ onto the line spanned by $u$.

A vector lying on the line determined by $u$.

1. Formula
   Observation

   The board shows $(x^T u)^2$ with the annotation "(info preserved)".

2. Audio
   Observation

   The narrator says, "we can think of the square of this inner product as the amount of information about x that is preserved after projection on u."

($x^T u$)^2

The squared inner product between $x$ and the unit vector $u$, interpreted in the video as preserved information after projection.

Scalar quantity associated with a single point $x$ and direction $u$.

1. Formula
   Observation

   The optimization board shows $\max \sum_i (x_i^T u)^2$ subject to $u^T u = 1$.

2. Audio
   Observation

   The narrator says PCA "maximizes the sum over all countries of the square of the inner product."

$\sum_i (x_i^T u)^2$

Total preserved information across all data points when projecting onto direction $u$.

Sum over indexed data points $x_i$ in the dataset.

1. Formula
   Observation

   The board displays $u^T u = 1$ under the maximization problem.

2. Audio
   Observation

   The narrator describes the first component as "a unit vector".

$u^T u = 1$

Normalization constraint forcing the chosen direction to be a unit vector.

Constraint on the vector $u$.

1. Formula
   Observation

   The board defines $C = \frac{1}{n} \sum_i x_i x_i^T$ and labels it "(cov matrix)".

2. Audio
   Observation

   The narrator says, "where C is known as the covariance matrix of the data."

C

The covariance matrix used to rewrite the PCA objective.

Matrix built from the data vectors $x_i$; the video does not discuss centering or sample-versus-population normalization beyond the displayed $1/n$ factor.

1. Formula
   Observation

   The Lagrangian and final eigenvalue equation use $\lambda$.

2. Audio
   Observation

   The narrator refers to the "Lagrange multipliers method" and then concludes with the equation $Cu = \lambda u$.

$\lambda$

The Lagrange multiplier associated with the unit-length constraint; in the final equation it appears as the scalar multiplying $u$.

Scalar introduced by the constrained optimization derivation.

1. Formula
   Observation

   The board shows $C u = \lambda u$ and labels $C$ as "(cov matrix)".

2. Audio
   Observation

   The narration refers to "the covariance matrix of the data" and later to "every symmetric matrix like C".

C

Covariance matrix of the data; in this example it is a symmetric matrix whose eigenvectors are PCA components.

A square matrix acting on vectors in the feature space spanned by GDP, Social, and Life.

1. Formula
   Observation

   The equation $C u = \lambda u$ is displayed on screen.

2. Audio
   Observation

   The speaker says this is "exactly the equation of the eigenvectors and eigenvalues of C".

u

Eigenvector of $C$; in PCA it gives a component direction onto which data are projected.

Vector in the same feature space as the data points.

1. Formula
   Observation

   The equation $C u = \lambda u$ is shown with $\lambda$ as an unknown scalar.

2. Audio
   Observation

   The narrator calls $\lambda$ "some unknown scalar" and identifies it as the eigenvalue corresponding to the eigenvector.

$\lambda$

Eigenvalue associated with an eigenvector $u$ of $C$; here interpreted as the amount of information preserved after projection onto that eigenvector.

Scalar.

1. Formula
   Observation

   The covariance formula is shown as $C = \frac{1}{n}\sum_i x_i x_i^T$.

1. The video does not explicitly state whether the data have already been centered before this formula is used.

$x_i$

Data vector for observation $i$ used in the displayed covariance-matrix formula.

Column vector in the feature space.

1. Audio
   Observation

   The narrator states that PCA takes all factors, combines them in a smart way, and produces new factors that are uncorrelated and ranked from most important to least important.

2. Animation
   Observation

   The original axes GDP, Social, Life are transformed into numbered axes 1, 2, 3, visually representing new principal components.

The video presents PCA as a method for reducing a multidimensional dataset while retaining as much information as possible. It does this by constructing new factors, called principal components, from the original variables. These components are described as mutually uncorrelated and ordered by importance, so the first few components can give a faithful representation of the data.

1. Applied to a dataset with multiple measured factors or dimensions.

2. The goal is dimensionality reduction while preserving information.

3. The video frames the output as new factors rather than raw original variables.

1. Formula
   Observation

   The board shows $x' = (x^T u) u$.

2. Audio
   Observation

   The narrator explains that projecting a point $x$ on a unit vector $u$ gives a new point $x'$ whose magnitude is given by the inner product between $x$ and $u$.

3. Animation
   Observation

   A pink point $x$ is dropped orthogonally onto the yellow direction $u$, producing $x'$ on the line.

For a point vector $x$ and a unit direction vector $u$, the projection onto the line spanned by $u$ is $x' = (x^T u)u$. The scalar coefficient $(x^T u)$ measures how much of $x$ lies along $u$. The video further interprets $(x^T u)^2$ as the amount of information about $x$ preserved after projection.

1. $u$ must be a unit vector, i.e. $u^T u = 1$.

2. The formula is for orthogonal projection onto the one-dimensional subspace spanned by $u$.

3. The interpretation of $(x^T u)^2$ as preserved information is the video's framing.

1. x
2. u
3. x'

1. Formula
   Observation

   The board shows $\max \sum_i (x_i^T u)^2$ subject to $u^T u = 1$.

2. Audio
   Observation

   The narrator says the first component is a unit vector that tries to preserve as much information as possible, so it maximizes the sum over all countries of the square of the inner product.

The first principal component is defined as the unit direction $u$ that maximizes the total squared projection of all data points onto that direction. In the video's notation, this is the constrained optimization problem $\max \sum_i (x_i^T u)^2$ subject to $u^T u = 1$. This formalizes the idea of choosing the line that preserves the most information from the original cloud of points.

1. The data are represented as points $x_i$.

2. The direction $u$ is constrained to have unit length.

3. The objective sums over all data points in the dataset.

1. Projection of a point onto a unit vector
2. $\sum_i (x_i^T u)^2$
3. $u^T u = 1$

1. Formula
   Observation

   The board defines $C = \frac{1}{n} \sum_i x_i x_i^T$ and labels it "(cov matrix)".

2. Audio
   Observation

   The narrator says, "where C is known as the covariance matrix of the data."

1. The video does not explicitly discuss whether the data have been centered before forming $C$, even though it calls the displayed matrix the covariance matrix.

The video introduces the matrix $C = \frac{1}{n} \sum_i x_i x_i^T$ and names it the covariance matrix of the data. This matrix is used to rewrite the PCA objective in compact quadratic form. The presentation treats $C$ as the central object that summarizes the second-order structure of the dataset for the purpose of choosing principal directions.

1. Built from data vectors $x_i$.

2. The displayed normalization uses $1/n$.

3. The video does not state additional assumptions such as centering.

1. C

1. Formula
   Observation

   The board rewrites the objective as $\max u^T C u$ with the constraint $u^T u = 1$.

2. Audio
   Observation

   The narrator says that with some manipulations the expression can be rewritten as $u$ times the matrix $C$ times $u$.

1. The intermediate algebraic steps are skipped in the video.

The sum of squared projections is rewritten in matrix form as $u^T C u$, where $C$ is the covariance matrix introduced just before. This converts the original summation-based objective into a compact quadratic form, making the subsequent constrained optimization easier to handle.

1. Uses the previously defined matrix $C$.

2. Equivalent to the earlier objective $\sum_i (x_i^T u)^2$ under the video's derivation.

3. The detailed manipulation is not shown on screen.

1. First principal component as an optimization problem
2. Covariance matrix used in PCA

1. Formula
   Observation

   The board shows $u^T C u - \lambda(u^T u - 1)$.

2. Audio
   Observation

   The narrator says, "We then form the Lagrange function, take the gradient with respect to u, and set it equal to zero."

3. Animation
   Observation

   The text "Lagrange Multipliers" appears before the derivation.

To solve the constrained maximization of $u^T C u$ subject to $u^T u = 1$, the video forms the Lagrangian $u^T C u - \lambda(u^T u - 1)$. The multiplier $\lambda$ enforces the unit-length constraint, and the method turns the constrained problem into an unconstrained stationarity condition.

1. Applies to the constrained optimization problem for the first principal component.

2. The constraint is $u^T u = 1$.

3. The video uses the Lagrange multipliers method without proving its general validity.

1. Rewriting the PCA objective as a quadratic form
2. $\lambda$

1. Formula
   Observation

   The board shows $2Cu - 2\lambda u = 0$ and then $Cu = \lambda u$.

2. Audio
   Observation

   The narrator concludes that the direction preserving the most information satisfies $Cu = \lambda u$.

1. The video does not explicitly state that the maximizing direction corresponds to the largest eigenvalue, only that the stationary condition is $Cu = \lambda u$.

Differentiating the Lagrangian with respect to $u$ and setting the result to zero yields $2Cu - 2\lambda u = 0$, which simplifies to $Cu = \lambda u$. Thus the optimal projection direction is characterized as an eigenvector of the covariance matrix $C$. The video presents this as the key conclusion of the derivation.

1. $C$ is the covariance matrix defined earlier.

2. $u$ is the direction vector being optimized.

3. The equation is obtained from the stationarity condition of the Lagrangian.

1. Lagrangian for the constrained PCA problem
2. Covariance matrix used in PCA

1. Formula
   Observation

   The board shows $C u = \lambda u$ together with $C = \frac{1}{n}\sum_i x_i x_i^T$ labeled "(cov matrix)".

2. Audio
   Observation

   The speaker says this is exactly the equation of the eigenvectors and eigenvalues of $C$.

The video identifies PCA directions as eigenvectors of the covariance matrix $C$. A vector $u$ is an eigenvector if multiplying $C$ by $u$ only rescales it by a scalar $\lambda$, giving $C u = \lambda u$. In this context, $u$ is the component direction and $\lambda$ measures how much information is preserved when projecting onto that direction.

1. $C$ is the covariance matrix of the data.

2. $u$ is an eigenvector of $C$.

3. $\lambda$ is the corresponding eigenvalue.

1. Formula
   Observation

   The board transforms $C u = \lambda u$ into $u^T C u = \lambda u^T u$ and then into $u^T C u = \lambda$.

2. Audio
   Observation

   The narrator says the amount of information preserved after projection on an eigenvector is given by the corresponding eigenvalue.

1. The video does not explicitly state the normalization assumption $u^T u=1$ at the moment the simplification is shown, although later it does impose unit length for $u_2$.

For an eigenvector direction $u$, the quadratic form $u^T C u$ gives the variance captured along that direction. When $u$ is normalized so that $u^T u=1$, the identity $C u = \lambda u$ implies $u^T C u = \lambda$. Thus the eigenvalue is the amount of information preserved by projection onto that eigenvector.

1. $u$ is an eigenvector of $C$.

2. $\lambda$ is its eigenvalue.

3. The simplification to $u^T C u = \lambda$ uses unit norm $u^T u=1$.

1. Eigen-equation for PCA components

1. Audio
   Observation

   The speaker says the best direction to pick is the eigenvector with the largest eigenvalue.

2. Formula
   Observation

   The board writes "$u_1 =$ top eigenvector of $C$" and then displays $u_1 = \begin{bmatrix}0.59\\0.56\\0.58\end{bmatrix}$.

The first principal component is chosen as the eigenvector of $C$ with the largest eigenvalue, because that direction preserves the most information under projection. In the example, this top eigenvector is numerically close to an equal mixture of GDP, Social, and Life, so the video names the component "Power".

1. Choose among eigenvectors of $C$.

2. Select the one with the largest eigenvalue.

1. Eigen-equation for PCA components
2. Projected information equals the eigenvalue

1. Formula
   Observation

   The board shows $u_2^T u_2 = 1$, $u_2 \perp u_1$, and $\max u_2^T C u_2$.

2. Audio
   Observation

   The narrator says the second component should be a unit vector that does not contain information already contained in the first component, geometrically lying in the subspace orthogonal to $u_1$, while maximizing the same quantity as before.

The second component is defined as the unit vector $u_2$ orthogonal to the first component $u_1$ that maximizes the quadratic form $u_2^T C u_2$. This makes it the direction of greatest remaining variance after removing what is already captured by $u_1$.

1. $u_2$ must have unit length.

2. $u_2$ must be orthogonal to $u_1$.

3. The objective is to maximize $u_2^T C u_2$.

1. First PCA component is the top eigenvector
2. Projected information equals the eigenvalue

1. Formula
   Observation

   The board writes "$u_2 =$ second eigenvector of $C$" and then displays $u_2 = \begin{bmatrix}0.22\\-0.8\\0.55\end{bmatrix}$.

2. Audio
   Observation

   The speaker says the second component is given by the second eigenvector of the covariance matrix of the data.

Solving the constrained maximization for $u_2$ yields the second eigenvector of $C$. In the example, its coordinates imply a contrast between individual factors and social factors, so the video names this component "Balance".

1. $u_2$ is obtained after choosing $u_1$ as the top eigenvector.

2. $u_2$ corresponds to the next largest eigenvalue direction under the orthogonality constraint.

1. Definition of the second PCA component

1. Audio
   Observation

   The narrator says, "In particular, this quantity is maximum when x is parallel to u, and is minimal when x is orthogonal to u."

2. Animation
   Observation

   The diagram shows $x$ aligned with $u$ giving a long projection, and $x$ perpendicular to $u$ collapsing to the origin.

For a fixed unit vector $u$, the quantity $(x^T u)^2$ is maximal when $x$ is parallel to $u$ and minimal when $x$ is orthogonal to $u$.

1. $u$ is a unit vector.

2. $x$ is a vector in the same space as $u$.

3. The quantity under discussion is $(x^T u)^2$.

For each fixed unit vector $u$, and for varying $x$.

1. Audio
   Observation

   The narrator states that PCA produces new factors that are (1) uncorrelated with each other and (2) ranked from most important to least important.

2. Animation
   Observation

   The original axes are replaced by numbered axes 1, 2, 3.

1. The video asserts these properties verbally but does not prove them within this clip.

The new factors produced by PCA are uncorrelated with each other and are ranked from most important to least important.

1. PCA is applied to the dataset.

2. The outputs are the principal components constructed by PCA.

For the set of principal components produced by PCA.

1. Audio
   Observation

   The narrator says the first component is a unit vector that maximizes the sum of squared inner products and later concludes it satisfies $Cu = \lambda u$.

2. Formula
   Observation

   The board shows $\max \sum_i (x_i^T u)^2$ subject to $u^T u = 1$, then $u^T C u$, then $Cu = \lambda u$.

1. The clip does not explicitly connect $\lambda$ to the largest eigenvalue, although that is the standard interpretation.

The first principal component is a unit vector $u$ that maximizes $\sum_i (x_i^T u)^2$ subject to $u^T u = 1$, and any stationary solution of this constrained problem satisfies $Cu = \lambda u$.

1. The data are represented by vectors $x_i$.

2. $C = \frac{1}{n}\sum_i x_i x_i^T$ is the matrix used in the derivation.

3. The optimization is constrained by $u^T u = 1$.

There exists a maximizing unit direction $u$; the derived condition applies to stationary points of the constrained problem.

1. Audio
   Observation

   The narrator states: "So clearly, the best direction to pick is the eigenvector with the largest eigenvalue."

Among the eigenvectors of the covariance matrix $C$, the best direction for preserving information under projection is the one with the largest eigenvalue.

1. $C$ is the covariance matrix.

2. Projection is onto an eigenvector direction of $C$.

3. Information preserved is measured by the corresponding eigenvalue.

For the set of eigenvectors of $C$, choose the one with maximal eigenvalue.

1. Audio
   Observation

   The speaker says: "More generally, the directions picked by PCA are exactly the eigenvectors of the covariance matrix."

2. Formula
   Observation

   The title card reads "PCA's Components = Eigenvectors of C".

The directions selected by PCA are exactly the eigenvectors of the covariance matrix $C$.

1. $C$ is the covariance matrix of the data.

2. PCA components are defined by successive maximal-variance orthogonal directions.

For all PCA components in this setup.

1. Audio
   Observation

   The narrator says: "You see, every symmetric matrix like C has n eigenvector and eigenvalue pairs that are orthogonal to each other."

2. Formula
   Observation

   The graphic shows $u_1 \perp u_2 \perp \cdots \perp u_n$ alongside $\lambda_1,\lambda_2,\ldots,\lambda_n$.

1. The statement is presented without proof in the clip.

2. The video does not discuss repeated eigenvalues or the need to choose an orthogonal basis within an eigenspace.

Every symmetric matrix like $C$ has $n$ eigenvector-eigenvalue pairs whose eigenvectors are orthogonal to each other.

1. The matrix is symmetric.

2. The ambient dimension is $n$.

For every symmetric matrix $C$ in $n$ dimensions.

1. Audio
   Observation

   The narrator says: "If we divide all the eigenvalues by their sum, we see that the component power explains about 85% of the data, while balance explains about 10%."

2. Formula
   Observation

   The table shows eigenvalues 2.58, 0.29, and 0.13, and the first entry is annotated as 85%.

1. The exact arithmetic is not shown step by step on screen.

2. The phrase "explains about 85% of the data" is an informal description of variance explained.

Dividing each eigenvalue by the sum of all eigenvalues gives the fraction of the data explained by the corresponding component; in this example Power explains about 85% and Balance about 10%.

1. The listed eigenvalues belong to the components of $C$.

2. All eigenvalues are included in the sum.

For the three components shown in the example.

1. Formula
   Observation

   The board shows $x' = (x^T u)u$ and then $(x^T u)^2$ labeled "(info preserved)".

2. Audio
   Observation

   The narrator explains projection and interprets the squared inner product as preserved information.

3. Animation
   Observation

   The point $x$ is projected onto the line spanned by $u$, and the alignment/orientation of $x$ relative to $u$ changes.

1. Expression

   $$x' = (x^T u) u$$
   Explanation

   Project the point $x$ onto the line determined by the unit vector $u$.

   Justification

   Stated directly by the narrator and shown on the board.

   Shown in the video
2. Expression

   $$(x^T u)^2$$
   Explanation

   Take the square of the scalar coefficient of the projection.

   Justification

   The video labels this quantity as the amount of information preserved after projection.

   Shown in the video

The squared inner product $(x^T u)^2$ measures how much of $x$ is retained when projecting onto the direction $u$.

1. Formula
   Observation

   The board successively shows $\max \sum_i (x_i^T u)^2$, $u^T u = 1$, $u^T C u$, $C = \frac{1}{n}\sum_i x_i x_i^T$, $u^T C u - \lambda(u^T u - 1)$, $2Cu - 2\lambda u = 0$, and $Cu = \lambda u$.

2. Audio
   Observation

   The narrator describes maximizing preserved information, rewriting with the covariance matrix, and using Lagrange multipliers.

1. The algebraic manipulation from $\sum_i (x_i^T u)^2$ to $u^T C u$ is asserted but not shown step by step.

2. The differentiation step is summarized rather than fully expanded.

1. Expression

   $$\max \sum_i (x_i^T u)^2 \quad \text{subject to} \quad u^T u = 1$$
   Explanation

   Choose the unit direction $u$ that maximizes the total squared projection of all data points.

   Justification

   This is the optimization problem stated by the narrator for the first principal component.

   Shown in the video
2. Expression

   $$u^T C u$$
   Explanation

   Rewrite the objective as a quadratic form using the matrix $C$.

   Justification

   The narrator says the expression can be rewritten as $u$ times $C$ times $u$ after manipulations.

   Shown in the video
3. Expression

   $$C = \frac{1}{n} \sum_i x_i x_i^T$$
   Explanation

   Define the matrix $C$ that appears in the quadratic form.

   Justification

   Shown on the board and named the covariance matrix by the narrator.

   Shown in the video
4. Expression

   $$u^T C u - \lambda(u^T u - 1)$$
   Explanation

   Form the Lagrangian for the constrained maximization problem.

   Justification

   The narrator explicitly says to form the Lagrange function.

   Shown in the video
5. Expression

   $$2Cu - 2\lambda u = 0$$
   Explanation

   Take the gradient with respect to $u$ and set it equal to zero.

   Justification

   Stated by the narrator and displayed on the board.

   Shown in the video
6. Expression

   $$Cu = \lambda u$$
   Explanation

   Simplify the stationarity condition to obtain the eigenvalue equation.

   Justification

   Algebraic simplification of the previous line, shown on the board.

   Shown in the video

The optimal direction for the first principal component satisfies $Cu = \lambda u$, so it is an eigenvector of the covariance matrix $C$.

1. Formula
   Observation

   The board successively shows $C u = \lambda u$, then $u^T C u = \lambda u^T u$, then $u^T C u = \lambda$.

2. Audio
   Observation

   The narration links projection information to the corresponding eigenvalue while these formulas appear.

1. The step from $\lambda u^T u$ to $\lambda$ implicitly assumes $u^T u=1$, but the video does not verbally spell out that assumption at this instant.

1. Expression

   $$C u = \lambda u$$
   Explanation

   Start from the defining eigen-equation for an eigenvector $u$ of the covariance matrix $C$.

   Justification

   Given definition of eigenvector and eigenvalue.

   Shown in the video
2. Expression

   $$u^T C u = \lambda u^T u$$
   Explanation

   Left-multiply both sides by $u^T$ to obtain a scalar quadratic form on the left and a scalar multiple of $u^T u$ on the right.

   Justification

   Algebraic multiplication of the eigen-equation by $u^T$ on the left.

   Shown in the video
3. Expression

   $$u^T C u = \lambda$$
   Explanation

   If $u$ is normalized to unit length, then $u^T u=1$, so the right-hand side reduces to $\lambda$.

   Justification

   Use of the unit-norm condition $u^T u=1$.

   Derived from the video

For a unit eigenvector $u$, the projected information or variance captured along that direction is exactly the eigenvalue $\lambda$.

1. Formula
   Observation

   The board displays $u_2^T u_2 = 1$, $u_2 \perp u_1$, $\max u_2^T C u_2$, and then identifies $u_2$ as the second eigenvector of $C$.

2. Audio
   Observation

   The narrator explains that the second component should avoid information already in the first component and maximize the same quantity as before.

1. The video does not show the Lagrange-multiplier derivation or the spectral theorem argument in detail; it states the result directly.

1. Expression

   $$u_2^T u_2 = 1$$
   Explanation

   Require the second direction to be a unit vector so that the maximized quantity measures direction rather than scale.

   Justification

   Normalization constraint stated on screen.

   Shown in the video
2. Expression

   $$u_2 \perp u_1$$
   Explanation

   Require orthogonality to the first component so that $u_2$ does not reuse information already captured by $u_1$.

   Justification

   Geometric interpretation given by the narrator: the second component lies in the subspace orthogonal to $u_1$.

   Shown in the video
3. Expression

   $$\max u_2^T C u_2$$
   Explanation

   Among all such admissible directions, choose the one that maximizes the quadratic form $u_2^T C u_2$, i.e. the remaining preserved information.

   Justification

   Same objective used for the first component, now restricted to the orthogonal subspace.

   Shown in the video
4. Expression

   $$u_2 = \text{second eigenvector of } C$$
   Explanation

   The solution of this constrained maximization is the next eigenvector of $C$ after the top one.

   Justification

   Stated result in the video; standard PCA/eigen-decomposition fact.

   Shown in the video

The second PCA component is the second eigenvector of the covariance matrix, obtained by maximizing captured variance subject to unit length and orthogonality to the first component.

1. Formula
   Observation

   The table lists eigenvalues 2.58, 0.29, and 0.13, with 85% shown under Power.

2. Audio
   Observation

   The narrator says to divide all the eigenvalues by their sum to get the percentage of data explained.

1. The clip does not display the intermediate arithmetic line-by-line on screen.

2. The spoken sentence is cut off after introducing Balance's percentage, though the visible number 10% is present.

1. Expression

   $$\lambda_1=2.58,\ \lambda_2=0.29,\ \lambda_3=0.13$$
   Explanation

   Read the three eigenvalues from the summary table.

   Justification

   Directly observed on screen.

   Shown in the video
2. Expression

   $$\lambda_1+\lambda_2+\lambda_3 = 2.58+0.29+0.13 = 3.00$$
   Explanation

   Add the eigenvalues to obtain the total variance represented by the full set of components.

   Justification

   Arithmetic summation of the displayed values.

   Derived from the video
3. Expression

   $$\frac{\lambda_1}{\lambda_1+\lambda_2+\lambda_3}=\frac{2.58}{3.00}=0.86$$
   Explanation

   Normalize the first eigenvalue by the total to estimate the fraction of data explained by Power.

   Justification

   Procedure stated by the narrator: divide all eigenvalues by their sum.

   Derived from the video
4. Expression

   $$\frac{\lambda_2}{\lambda_1+\lambda_2+\lambda_3}=\frac{0.29}{3.00}\approx 0.097$$
   Explanation

   Normalize the second eigenvalue similarly to estimate the fraction explained by Balance.

   Justification

   Same normalization procedure applied to the second component.

   Derived from the video

The normalized eigenvalues show that Power accounts for roughly 85–86% of the data and Balance for about 10%, matching the video's summary.

1. Audio
   Observation

   The narrator introduces a 2021 UN report scoring countries on six factors and says one can pick two or three factors such as GDP, social support, and life expectancy to visualize the data.

2. Diagram
   Observation

   A table lists countries FR, DE, IN, MA, TR, US with columns GDP, SOCIAL, LIFE, FREEDOM, GENEROSITY, CORRUPTION.

3. Animation
   Observation

   Country flags appear as points in a 3D plot with axes GDP, Social, Life.

1. The exact numerical values in the table are visible but not all are needed for the mathematical explanation.

2. The video does not define the units or scaling of the six factors.

Given country-level data with six factors, how can one begin to analyze and visualize the dataset?

1. A dataset of countries with six factors: GDP, SOCIAL, LIFE, FREEDOM, GENEROSITY, CORRUPTION.

2. The example countries shown include FR, DE, IN, MA, TR, US.

3. The video chooses three factors for visualization: GDP, Social, Life.

Represent the data visually in a lower-dimensional plot while acknowledging that some factors are omitted.

1. Expression

   $$$$
   Explanation

   Start from the full six-factor table of country scores.

   Justification

   Shown directly in the opening data display.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Select only three factors, for example GDP, social support, and life expectancy, and plot each country as a point in 3D.

   Justification

   The narrator says we cannot visualize six dimensions at once but can pick two or three factors.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Recognize that this 3D view loses information contained in the remaining factors such as freedom and generosity.

   Justification

   Explicitly stated by the narrator after showing the 3D scatter plot.

   Shown in the video

A 3D scatter plot using GDP, Social, and Life gives a partial visualization of the country data, but it discards information from the other factors.

The video verifies the limitation by pointing out that valuable information in factors like freedom and generosity is lost in the 3D representation.

1. Diagram
   Observation

   A 3D scatter plot uses axes labeled GDP, Social, and Life, with country-flag spheres as data points.

2. Formula
   Observation

   The board displays $u_1 = \begin{bmatrix}0.59\\0.56\\0.58\end{bmatrix}$, $u_1 = 0.59\,\text{GDP} + 0.56\,\text{Social} + 0.58\,\text{Life}$, $u_2 = \begin{bmatrix}0.22\\-0.8\\0.55\end{bmatrix}$, and $u_2 = 0.22\,\text{GDP} - 0.8\,\text{Social} + 0.55\,\text{Life}$.

3. Audio
   Observation

   The narrator interprets the first component as "Power" and the second as "Balance", then discusses Norway, Niger, Singapore, Benin, and Turkmenistan.

1. The underlying dataset is not fully specified beyond being a happiness-related example with GDP, Social, and Life features.

2. The exact numerical projection values for individual countries are not shown.

Use PCA to reduce a 3-feature country dataset (GDP, Social, Life) to interpretable components and examine what the first two components say about countries such as Norway, Niger, Singapore, Benin, and Turkmenistan.

1. Data points are countries plotted in a 3D space with axes GDP, Social, and Life.

2. The covariance matrix is $C$.

3. The top eigenvector is $u_1 = [0.59,0.56,0.58]^T$.

4. The second eigenvector is $u_2 = [0.22,-0.8,0.55]^T$.

Identify the first two PCA components, interpret them, and project the countries onto the plane spanned by them.

1. Expression

   $$u_1 = 0.59\,\text{GDP} + 0.56\,\text{Social} + 0.58\,\text{Life}$$
   Explanation

   Expand the top eigenvector in the original feature basis to interpret the first component.

   Justification

   Directly shown on screen after displaying the coordinate vector.

   Shown in the video
2. Expression

   $$\text{Power}$$
   Explanation

   Name the first component "Power" because all three original factors contribute almost equally.

   Justification

   Narrator's interpretation of the nearly equal coefficients.

   Shown in the video
3. Expression

   $$u_2 = 0.22\,\text{GDP} - 0.8\,\text{Social} + 0.55\,\text{Life}$$
   Explanation

   Expand the second eigenvector in the original feature basis to interpret the second component.

   Justification

   Directly shown on screen after displaying the coordinate vector.

   Shown in the video
4. Expression

   $$u_2 = (0.22\,\text{GDP} + 0.55\,\text{Life}) - 0.8\,\text{Social}$$
   Explanation

   Regroup the terms to highlight that the second component contrasts individual factors against the social factor.

   Justification

   Algebraic rearrangement shown visually on the board.

   Shown in the video
5. Expression

   $$\text{Balance}$$
   Explanation

   Name the second component "Balance" based on that contrast interpretation.

   Justification

   Narrator's explicit naming.

   Shown in the video
6. Expression

   $$\text{Project countries onto the plane spanned by } u_1 \text{ and } u_2$$
   Explanation

   Reduce the 3D data to a 2D representation using the first two components.

   Justification

   Described by the narrator and visualized by rotating to a 2D plane labeled Power and Balance.

   Shown in the video

The first component is Power, approximately an equal blend of GDP, Social, and Life; the second is Balance, contrasting individual factors against Social. In the 2D projection, high-Power countries include Norway, low-Power countries include Niger, Singapore ranks high on Power but lower on overall happiness due to imbalance, and Benin versus Turkmenistan illustrate opposite ends of the Balance axis.

The verification is qualitative and visual: the projected positions of named countries match the narrator's interpretations on the Power-Balance plane.

1. Diagram
   Observation

   A table of countries and six factor columns appears first.

2. Animation
   Observation

   The scene transitions to three colored axes labeled GDP, Social, and Life.

1. Country table with rows FR, DE, IN, MA, TR, US

2. Columns GDP, SOCIAL, LIFE, FREEDOM, GENEROSITY, CORRUPTION

3. Three axes labeled GDP, Social, Life

1. The presentation moves from a full six-column table to a three-axis spatial plot.

2. Only three of the six factors are retained for visualization.

1. The underlying idea remains that each country is represented by numerical factor scores.

The visual transition illustrates the practical need to reduce dimensionality before plotting high-dimensional data.

1. Animation
   Observation

   Flag-covered spheres appear in the 3D plot, including examples such as Norway, the US, China, and Russia.

2. Audio
   Observation

   The narrator names example countries while the points populate the space.

1. Individual point coordinates are not numerically labeled on the plot.

1. Colored spheres marked with national flags

2. Axes GDP, Social, Life

1. The empty coordinate system becomes a scatter plot of many countries.

2. Specific example countries are identified verbally as they appear.

1. Each sphere represents one country positioned according to the selected three factors.

This visualizes the dataset as a cloud of points in a reduced 3D feature space.

1. Animation
   Observation

   The original axes GDP, Social, Life are shown beside a new set of axes labeled 1, 2, 3.

2. Audio
   Observation

   The narrator explains that PCA produces new factors called principal components.

1. The exact geometric rule for the rotation is not shown in this interval.

1. Original axis triad labeled GDP, Social, Life

2. New axis triad labeled 1, 2, 3

1. The coordinate frame changes from named original variables to numbered components.

2. The new axes are presented as a smarter combination of the old ones.

1. Both frames represent coordinate systems for the same underlying data cloud.

The animation conveys that PCA re-expresses the data in a new basis ordered by importance.

1. Animation
   Observation

   A pink point $x$, a yellow vector $u$, and a projected point $x'$ are shown with a dashed perpendicular segment.

2. Formula
   Observation

   The board displays $x' = (x^T u)u$ and $(x^T u)^2$.

1. Point $x$

2. Unit vector $u$

3. Projected point $x'$

4. Dashed perpendicular from $x$ to the line through $u$

1. The position of $x$ relative to $u$ varies to show different projection lengths.

2. When $x$ aligns with $u$, the projection is longest; when $x$ is perpendicular, it collapses to the origin.

1. The projection always lies on the line spanned by $u$.

2. The formula $x' = (x^T u)u$ remains the governing relation.

The animation makes the inner product visible as the signed length coefficient of the projection and its square as preserved information.

1. Formula
   Observation

   The board sequentially shows the constrained maximization, the covariance matrix definition, the Lagrangian, the gradient condition, and the eigenvalue equation.

2. Animation
   Observation

   Text labels such as "Lagrange Multipliers" and "Optimization" appear during the derivation.

1. Some intermediate algebra is compressed on screen.

1. Objective $\max \sum_i (x_i^T u)^2$

2. Constraint $u^T u = 1$

3. Matrix definition $C = \frac{1}{n}\sum_i x_i x_i^T$

4. Lagrangian $u^T C u - \lambda(u^T u - 1)$

5. Final equation $Cu = \lambda u$

1. The expression evolves from a sum of squared projections to a quadratic form, then to a Lagrangian, and finally to an eigenvalue equation.

1. All steps concern the same constrained optimization problem for choosing the best projection direction.

The visual sequence organizes the mathematical argument into a compact derivation chain leading from data projection to eigenvectors of the covariance matrix.

1. Diagram
   Observation

   A 3D coordinate system labeled GDP, Social, and Life contains country-flag spheres; a black board shows $C u = \lambda u$ and $C = \frac{1}{n}\sum_i x_i x_i^T$ labeled "(cov matrix)".

2. Animation
   Observation

   The equation on the board changes from $C u = \lambda u$ to $u^T C u = \lambda u^T u$ and then to $u^T C u = \lambda$.

1. 3D axes labeled GDP, Social, Life

2. Country-flag spheres

3. Blackboard with formulas

1. The displayed equation progresses from $C u = \lambda u$ to $u^T C u = \lambda u^T u$ and then to $u^T C u = \lambda$.