Gram-Schmidt Process Example - Linear Algebra

1. Formula
   Observation

   On-screen text shows "Unit vectors" and the equation \|$\mathbf{u}$\| = 1.

2. Audio
   Observation

   The narrator says unit vectors or normal vectors have length one under the dot product.

\|$\mathbf{u}$\|

Norm (length) of a vector $\mathbf{u}$.

Vectors in an inner product space; here illustrated with the standard dot product.

1. Formula
   Observation

   The procedure slide lists the input sequence $\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$, $\ldots$.

2. Audio
   Observation

   The narrator describes starting from a set of basis vectors.

$\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$, $\ldots$

Given sequence of basis vectors to be transformed.

Basis vectors in an inner product space.

1. Formula
   Observation

   The slide defines $\mathbf{w}_1 = \mathbf{v}_1$ and then gives $\mathbf{w}_n = \mathbf{v}_n - \sum_{i=1}^{n-1} \operatorname{proj}_{\mathbf{w}_i}(\mathbf{v}_n)$.

2. Audio
   Observation

   The narrator says subsequent vectors are found by subtracting orthogonal projections onto previous vectors.

$\mathbf{w}_n$

Intermediate orthogonalized vector before normalization.

Defined recursively for $n \ge 1$ from the given basis vectors.

1. Formula
   Observation

   The general step uses $\operatorname{proj}_{\mathbf{w}_i}(\mathbf{v}_n)$, and the example later shows $\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$.

2. Audio
   Observation

   The narrator explains orthogonal projection as how much one vector points in the direction of another.

1. The clip does not display the explicit algebraic formula for $\operatorname{proj}_{\mathbf{a}}(\mathbf{b})$.

$\operatorname{proj}_{\mathbf{a}}(\mathbf{b})$

Orthogonal projection of vector $\mathbf{b}$ onto vector $\mathbf{a}$.

Vectors in an inner product space; the example is under the standard dot product.

1. Formula
   Observation

   The slide states $\mathbf{u}_n = \frac{\mathbf{w}_n}{\|\mathbf{w}_n\|}$ and maps $\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$, $\ldots \rightarrow \mathbf{u}_1$, $\mathbf{u}_2$, $\mathbf{u}_3$, $\ldots$.

2. Audio
   Observation

   The narrator says the final step is to normalize all vectors by dividing by their norms.

$\mathbf{u}_n$

Normalized orthonormal output vector corresponding to $\mathbf{w}_n$.

Defined after orthogonalization, provided \|$\mathbf{w}_n$\| $\neq 0$.

1. Diagram
   Observation

   Green vector labeled $v_1$ in the opening geometric diagram.

2. Formula
   Observation

   $v_1 = [1, 1, 0]^T$ appears as the first input vector in the worked example.

$\mathbf{v}_1$

First given vector; also used in the opening diagram as the reference direction for projection.

$\mathbb{R}^3$

1. Diagram
   Observation

   The expression $\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$ is shown as a cyan vector perpendicular to $\mathbf{v}_1$.

2. Formula
   Observation

   $v_2 = [0, 1, -1]^T$ appears as the second input vector.

$\mathbf{v}_2$

Second given vector; in the opening diagram it is the vector being projected onto $v_1$.

$\mathbb{R}^3$

1. Formula
   Observation

   $v_3 = [1, 0, -1]^T$ appears as the third input vector.

2. Audio
   Observation

   The narrator says the Gram-Schmidt process will be applied to three vectors in $R^3$.

$\mathbf{v}_3$

Third given vector in the example.

$\mathbb{R}^3$

1. Diagram
   Observation

   Opening title reads 'Orthonormal Projections' and the cyan vector is labeled $v_2$ - proj_{$v_1$}($v_2$).

2. Formula
   Observation

   Formulas use $\operatorname{proj}_{\mathbf{w}_1}(\mathbf{v}_2)$, $\operatorname{proj}_{\mathbf{w}_1}(\mathbf{v}_3)$, and $\operatorname{proj}_{\mathbf{w}_2}(\mathbf{v}_3)$.

$\operatorname{proj}_{\mathbf{u}}(\mathbf{v})$

Orthogonal projection of vector v onto vector u.

Defined by the inner-product formula shown later in the clip.

1. Formula
   Observation

   Set $w_1 = v_1 = [1, 1, 0]^T$.

2. Audio
   Observation

   Narrator says to let $v_1$ be the initial vector and not touch it until the very end.

$\mathbf{w}_1$

First orthogonal vector produced by the Gram-Schmidt process; here equal to the initial vector $v_1$.

$\mathbb{R}^3$

1. Formula
   Observation

   $w_2 = v_2$ - proj_{$w_1$}($v_2$) and then $w_2 = [-1/2, 1/2, -1]^T$.

2. Audio
   Observation

   Narrator says to find the second vector by subtracting the orthogonal projection of $v_2$ onto $v_1$.

$\mathbf{w}_2$

Second orthogonal vector obtained by removing from $v_2$ its component along $w_1$.

$\mathbb{R}^3$

1. Formula
   Observation

   $w_3 = v_3$ - proj_{$w_1$}($v_3$) - proj_{$w_2$}($v_3$) and then $w_3 = [2/3, -2/3, -2/3]^T$.

2. Audio
   Observation

   Narrator says that for the last vector we need to subtract two orthogonal projections from both $w_1$ and $w_2$.

$\mathbf{w}_3$

Third orthogonal vector obtained by removing from $v_3$ its components along $w_1$ and $w_2$.

$\mathbb{R}^3$

1. Audio
   Observation

   The narrator defines an orthonormal basis as vectors that are mutually orthogonal and unit vectors.

2. Diagram
   Observation

   A 2D grid shows two perpendicular arrows with a right-angle marker and the labels "Orthogonal vectors" and "Unit vectors \|$\mathbf{u}$\| = 1".

An orthonormal basis is a set of vectors that are pairwise orthogonal and each has norm 1. In the clip, orthogonality is illustrated by a 90-degree angle under the dot product, and normalization is illustrated by requiring length one.

1. The vectors belong to an inner product space.

2. The clip illustrates the standard dot product case.

1. Audio
   Observation

   The narrator says the Gram-Schmidt process takes any set of basis vectors in an inner product space and turns them into an orthonormal basis.

2. Formula
   Observation

   The slide maps $\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$, $\ldots$ to $\mathbf{u}_1$, $\mathbf{u}_2$, $\mathbf{u}_3$, $\ldots$.

The Gram-Schmidt process converts a given basis into an orthonormal basis. The procedure first constructs orthogonal intermediate vectors $\mathbf{w}_n$ and then normalizes them to obtain $\mathbf{u}_n$.

1. Input is a set of basis vectors in an inner product space.

2. The clip presents this as preparation for QR factorization.

1. Orthonormal basis

1. Formula
   Observation

   Step 1 on the slide reads "Set initial vector $\mathbf{w}_1$:" followed by $\mathbf{w}_1 = \mathbf{v}_1$.

2. Audio
   Observation

   The narrator says the process begins by setting an initial vector, which stays the same until normalization.

The first orthogonal intermediate vector is chosen to be the first input basis vector itself. No projection subtraction is needed for this first vector.

1. Applied at the start of the recursive construction.

1. Purpose of the Gram-Schmidt process

1. Formula
   Observation

   Step 2 on the slide reads "Find next vectors by subtracting orthogonal projections:" and shows $\mathbf{w}_n = \mathbf{v}_n - \sum_{i=1}^{n-1} \operatorname{proj}_{\mathbf{w}_i}(\mathbf{v}_n)$.

2. Audio
   Observation

   The narrator says subsequent vectors are found by subtracting the orthogonal projection of the current vector onto all previous vectors already put into the orthogonal set.

1. The clip does not explicitly state that the input vectors must be linearly independent for every $\mathbf{w}_n$ to be nonzero.

For each new input vector $\mathbf{v}_n$, remove its components along all previously constructed orthogonal vectors $\mathbf{w}_1$, $\ldots$, $\mathbf{w}_{n-1}$. The remainder $\mathbf{w}_n$ is orthogonal to those previous vectors.

1. Used for $n \ge 2$.

2. The sum runs over all previously constructed vectors $\mathbf{w}_i$.

1. Gram-Schmidt Step 1: initial vector
2. Geometric meaning of orthogonal projection

1. Formula
   Observation

   Step 3 on the slide reads "Normalize all vectors:" followed by $\mathbf{u}_n = \frac{\mathbf{w}_n}{\|\mathbf{w}_n\|}$.

2. Audio
   Observation

   The narrator says to normalize all vectors by dividing by the norm of the vectors themselves.

After constructing the orthogonal vectors $\mathbf{w}_n$, divide each by its own norm to obtain unit vectors $\mathbf{u}_n$. This completes the conversion from a basis to an orthonormal basis.

1. Requires \|$\mathbf{w}_n$\| $\neq 0$.

2. Applied after the projection-subtraction step.

1. Gram-Schmidt Step 2: subtract orthogonal projections
2. Orthonormal basis

1. Audio
   Observation

   The narrator says orthogonal projection can be thought of as how much one vector points in the direction of another vector.

2. Diagram
   Observation

   The animation drops a perpendicular from the tip of $\mathbf{v}_2$ to the line through $\mathbf{v}_1$, marking a right angle.

1. The clip gives geometric intuition but does not display the coordinate formula for projection.

The orthogonal projection of one vector onto another measures the component of the first vector lying in the direction of the second. Geometrically, it is obtained by dropping a perpendicular from the tip of the vector being projected onto the line spanned by the other vector.

1. The clip illustrates this under the standard dot product.

1. Audio
   Observation

   Narrator says perpendicular vectors are independent in a way where they do not point in each other's direction.

2. Diagram
   Observation

   Title 'Orthonormal Projections'; green vector $v_1$ and cyan vector $v_2$ - proj_{$v_1$}($v_2$) meet at a right angle marked by a square symbol.

The clip opens with a visual interpretation of projection: subtracting the component of one vector along another leaves a remainder perpendicular to that direction. The right-angle marker makes the orthogonality explicit.

1. Two vectors are in an inner-product space.

2. The subtraction removes the component of $v_2$ in the direction of $v_1$.

1. Audio
   Observation

   Narrator says they will use the standard dot product as the inner product.

2. Formula
   Observation

   All three input vectors are displayed as column vectors in $R^3$.

Before computing projections, the video fixes the ambient space and inner product: the example is carried out in $R^3$ using the standard dot product.

1. Vectors lie in $\mathbb{R}^3$.

2. The standard Euclidean dot product is used.

1. Formula
   Observation

   Header '1. Set initial vector' and equation 'Set $w_1 = v_1 = [1, 1, 0]^T$'.

2. Audio
   Observation

   Narrator says to begin by letting $v_1$ be the initial vector and not touch it until the very end.

The algorithm starts by taking the first input vector unchanged as the first orthogonal vector.

1. There is at least one input vector.

2. No normalization has yet been applied in this stage.

1. Inner product choice for the example

1. Formula
   Observation

   Header '2. Find subsequent vectors' with $w_2 = v_2$ - proj_{$w_1$}($v_2$), then expanded as $w_2 = v_2$ - ($\langle v_2$,$w_1\rangle$ / \|$w_1$\|^2) $w_1$.

2. Formula
   Observation

   Computed values shown: $\langle v_2$,$w_1\rangle = 1$, \|$w_1$\|^$2 = 2$, and $w_2 = [-1/2, 1/2, -1]^T$.

3. Audio
   Observation

   Narrator says to find the second vector by subtracting the orthogonal projection of $v_2$ onto $v_1$.

To make the next vector orthogonal to the previous one, the component of $v_2$ along $w_1$ is removed using the projection formula.

1. $w_1$ is already known.

2. The inner product and induced norm are fixed.

3. $w_1$ is nonzero so division by \|$w_1$\|^2 is valid.

1. First step of Gram-Schmidt: keep the initial vector
2. Inner product choice for the example

1. Audio
   Observation

   Narrator says you can check that $w_2$ is orthogonal to $w_1$ by dotting them and making sure you get zero.

2. Formula
   Observation

   The displayed vectors are $w_1 = [1,1,0]^T$ and $w_2 = [-1/2,1/2,-1]^T$.

1. The clip states the check verbally but does not show the arithmetic line for $w_1 \cdot w_2 = 0$.

After constructing a new vector, the video recommends checking pairwise dot products to confirm orthogonality.

1. The chosen inner product is the standard dot product in this example.

2. The vectors under test are nonzero.

1. Inner product choice for the example

1. Formula
   Observation

   $w_3 = v_3$ - proj_{$w_1$}($v_3$) - proj_{$w_2$}($v_3$), then expanded as $w_3 = v_3$ - ($\langle v_3$,$w_1\rangle$ / \|$w_1$\|^2)$w_1$ - ($\langle v_3$,$w_2\rangle$ / \|$w_2$\|^2)$w_2$.

2. Formula
   Observation

   Simplified displayed form: $w_3 = v_3 - (1/2)w_1 - (1/3)w_2$, followed by $w_3 = [2/3, -2/3, -2/3]^T$.

3. Audio
   Observation

   Narrator says for the last vector we need to subtract two orthogonal projections from both $w_1$ and $w_2$.

For the third vector, the algorithm removes from $v_3$ both the component along $w_1$ and the component along $w_2$, producing a vector orthogonal to the span of the earlier ones.

1. $w_1$ and $w_2$ are already constructed.

2. $w_1$ and $w_2$ are mutually orthogonal in the intended Gram-Schmidt process.

3. Each denominator \|$w_j$\|^2 is nonzero.

1. Second Gram-Schmidt vector by projection removal
2. Inner product choice for the example

1. Audio
   Observation

   The narrator says that if we subtract away the projection from our vector, the new vector does not point in the direction of the other vector at all, and therefore the two vectors are orthogonal.

2. Diagram
   Observation

   The resulting cyan vector labeled $\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$ is shown perpendicular to $\mathbf{v}_1$ with a right-angle marker.

1. The clip states the conclusion visually and verbally but does not provide an algebraic proof using inner products.

If the orthogonal projection of $\mathbf{v}_2$ onto $\mathbf{v}_1$ is subtracted from $\mathbf{v}_2$, the resulting vector $\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$ is orthogonal to $\mathbf{v}_1$.

1. The example is under the standard dot product.

2. $\operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$ denotes the orthogonal projection of $\mathbf{v}_2$ onto $\mathbf{v}_1$.

For the two vectors shown in the example.

1. Audio
   Observation

   Narrator says two vectors that are perpendicular are sort of independent in a way where they don't point in each other's direction.

1. This is presented as intuition rather than a formal theorem statement.

Perpendicular vectors do not point in each other's direction, which the narrator describes as a form of independence.

1. The discussion is geometric and informal.

2. The setting is the opening projection diagram.

Informal universal-style statement about pairs of perpendicular vectors.

1. Audio
   Observation

   Narrator says you can check that $w_2$ is indeed orthogonal to $w_1$ by dotting them and making sure that you get zero.

2. Formula
   Observation

   $w_1 = [1,1,0]^T$ and $w_2 = [-1/2,1/2,-1]^T$ are displayed.

1. The verification is stated verbally; the explicit dot-product computation is not shown on screen.

The constructed vector $w_2$ is orthogonal to $w_1$.

1. $w_1 = v_1$.

2. $w_2$ is defined by $w_2 = v_2$ - ($\langle v_2$,$w_1\rangle$ / \|$w_1$\|^2)$w_1$.

3. The inner product is the standard dot product on $\mathbb{R}^3$.

For the specific vectors in this worked example.

1. Audio
   Observation

   Narrator says again you can check that all three of these vectors are mutually orthogonal by dotting them and making sure that you get zero.

2. Formula
   Observation

   $w_1$, $w_2$, $w_3$ are displayed together before normalization.

1. The clip asserts the check but does not display all pairwise dot-product computations.

The three constructed vectors $w_1$, $w_2$, $w_3$ are mutually orthogonal.

1. $w_1$, $w_2$, $w_3$ are produced by the Gram-Schmidt steps shown.

2. The inner product is the standard dot product on $\mathbb{R}^3$.

For the three specific vectors in this example.

1. Audio
   Observation

   Narrator says with that we have found an orthonormal basis for $R^3$.

2. Formula
   Observation

   Three normalized vectors $u_1$, $u_2$, $u_3$ are displayed.

1. The video does not separately prove spanning; it presents the result after completing the three-step process on three vectors in $R^3$.

The normalized vectors $u_1$, $u_2$, $u_3$ form an orthonormal basis for $\mathbb{R}^3$.

1. The original three vectors are in $\mathbb{R}^3$.

2. The Gram-Schmidt process produced three nonzero orthogonal vectors.

3. Each vector was normalized using the induced norm.

For the specific example computed in the clip.

1. Formula
   Observation

   The slide lists three numbered steps: $\mathbf{w}_1 = \mathbf{v}_1$, $\mathbf{w}_n = \mathbf{v}_n - \sum_{i=1}^{n-1} \operatorname{proj}_{\mathbf{w}_i}(\mathbf{v}_n)$, and $\mathbf{u}_n = \frac{\mathbf{w}_n}{\|\mathbf{w}_n\|}$.

2. Audio
   Observation

   The narrator explains the same three-step order: initialize, subtract projections, then normalize.

1. Expression

   $$\mathbf{w}_1 = \mathbf{v}_1$$
   Explanation

   Start with the first basis vector unchanged as the first orthogonal intermediate vector.

   Justification

   Directly shown as Step 1 on the slide and described by the narrator.

   Shown in the video
2. Expression

   $$\mathbf{w}_n = \mathbf{v}_n - \sum_{i=1}^{n-1} \operatorname{proj}_{\mathbf{w}_i}(\mathbf{v}_n)$$
   Explanation

   For each later vector, subtract its projections onto all previously constructed orthogonal vectors.

   Justification

   Shown as Step 2 on the slide and explained in the audio as removing components along previous vectors.

   Shown in the video
3. Expression

   $$\mathbf{u}_n = \frac{\mathbf{w}_n}{\|\mathbf{w}_n\|}$$
   Explanation

   Normalize each orthogonal intermediate vector to obtain a unit vector.

   Justification

   Shown as Step 3 on the slide and described by the narrator as dividing by the norm.

   Shown in the video

The procedure transforms $\mathbf{v}_1$, $\mathbf{v}_2$, $\mathbf{v}_3$, $\ldots$ into $\mathbf{u}_1$, $\mathbf{u}_2$, $\mathbf{u}_3$, $\ldots$, an orthonormal set.

1. Animation
   Observation

   The animation constructs the projection of $\mathbf{v}_2$ onto $\mathbf{v}_1$, then forms $\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$ and shows it perpendicular to $\mathbf{v}_1$.

2. Audio
   Observation

   The narrator says subtracting the projection leaves a vector that does not point in the direction of the other vector, so the two are orthogonal.

1. This is a geometric demonstration rather than a full algebraic proof.

1. Expression

   $$\mathbf{v}_1, \mathbf{v}_2$$
   Explanation

   Begin with two non-orthogonal vectors in the plane.

   Justification

   Shown on the "Orthonormal Projections" slide.

   Shown in the video
2. Expression

   $$\operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$$
   Explanation

   Drop a perpendicular from the tip of $\mathbf{v}_2$ to the line through $\mathbf{v}_1$ to identify the component of $\mathbf{v}_2$ in the $\mathbf{v}_1$ direction.

   Justification

   Explained by the narrator as how much one vector points in the direction of another.

   Shown in the video
3. Expression

   $$\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)$$
   Explanation

   Subtract that component from $\mathbf{v}_2$.

   Justification

   The animation labels the resulting vector with this expression.

   Shown in the video
4. Expression

   $$\mathbf{v}_1 \perp \left(\mathbf{v}_2 - \operatorname{proj}_{\mathbf{v}_1}(\mathbf{v}_2)\right)$$
   Explanation

   The remaining vector is perpendicular to $\mathbf{v}_1$, as indicated by the right-angle marker.

   Justification

   Stated by the narrator and shown visually.

   Shown in the video

Removing the projection of $\mathbf{v}_2$ onto $\mathbf{v}_1$ leaves a vector orthogonal to $\mathbf{v}_1$, illustrating the core mechanism used in Gram-Schmidt.

1. Formula
   Observation

   The board shows $w_2 = v_2$ - proj_{$w_1$}($v_2$), then the expanded projection formula, then intermediate values $\langle v_2$,$w_1\rangle = 1$ and \|$w_1$\|^$2 = 2$, and finally $w_2 = [-1/2,1/2,-1]^T$.

2. Audio
   Observation

   Narrator explains that the second vector is found by subtracting the orthogonal projection of $v_2$ onto $v_1$.

1. Expression

   $$\mathbf{w}_1 = \mathbf{v}_1 = \begin{bmatrix}1\\1\\0\end{bmatrix}$$
   Explanation

   Start with the first orthogonal vector equal to the first input vector.

   Justification

   Displayed formula under '1. Set initial vector'.

   Shown in the video
2. Expression

   $$\mathbf{w}_2 = \mathbf{v}_2 - \operatorname{proj}_{\mathbf{w}_1}(\mathbf{v}_2)$$
   Explanation

   Define the second vector by removing from $v_2$ its projection onto $w_1$.

   Justification

   Displayed formula under '2. Find subsequent vectors'.

   Shown in the video
3. Expression

   $$\mathbf{w}_2 = \mathbf{v}_2 - \frac{\langle \mathbf{v}_2, \mathbf{w}_1 \rangle}{\|\mathbf{w}_1\|^2}\mathbf{w}_1$$
   Explanation

   Rewrite the projection using the inner-product formula.

   Justification

   Displayed algebraic expansion on screen.

   Shown in the video
4. Expression

   $$\langle \mathbf{v}_2, \mathbf{w}_1 \rangle = \begin{bmatrix}1\\1\\0\end{bmatrix} \cdot \begin{bmatrix}0\\1\\-1\end{bmatrix} = 1$$
   Explanation

   Compute the numerator of the projection coefficient.

   Justification

   Explicit dot-product calculation shown on screen.

   Shown in the video
5. Expression

   $$\|\mathbf{w}_1\|^2 = \begin{bmatrix}1\\1\\0\end{bmatrix} \cdot \begin{bmatrix}1\\1\\0\end{bmatrix} = 2$$
   Explanation

   Compute the squared norm in the denominator.

   Justification

   Explicit norm-squared calculation shown on screen.

   Shown in the video
6. Expression

   $$\mathbf{w}_2 = \begin{bmatrix}0\\1\\-1\end{bmatrix} - \frac{1}{2}\begin{bmatrix}1\\1\\0\end{bmatrix} = \begin{bmatrix}-1/2\\1/2\\-1\end{bmatrix}$$
   Explanation

   Substitute the computed coefficient and simplify to the final vector.

   Justification

   Final displayed result for $w_2$.

   Shown in the video