Mathematical modeling / English

Least Squares Approximation

MIT OpenCourseWare · YouTube · 8:04

YouTube · MIT OpenCourseWareOriginal video

Selected moment · 4:33 · Chapter markers are time references, not playback progress.

IN THIS VIDEO

01Least squares approximation problem setup
02Quadratic through the origin has no constant term
03First step of the matrix formulation

FROM THE LEARNING NOTES

ct + dt^2 = y

Quadratic through the origin has no constant term

SELECTED MOMENT

Computing $A^T A$

SUGGESTED PREPARATION

Quadratic functions
Basic matrix notation
Concept of fitting a curve to data
Basic matrix-vector multiplication
Systems of linear equations
Notion of a subspace and column space
Projection onto a subspace
Matrix transpose
Matrix-vector and matrix-matrix multiplication
Inconsistent linear systems
Least-squares formulation via normal equations
Matrix multiplication
Gaussian elimination
Basic notion of least-squares approximation
matrix multiplication
normal equations
polynomial fitting

READ & KEEP

The explanation, unpacked.

Reviewed learning material · Video analysis · English

Read the full overview

This 120-second clip is an MIT Linear Algebra recitation segment on least squares approximation. After a title slide, the instructor presents a worked problem: find the quadratic equation through the origin that best fits the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . He first explains that the desired model has the form $ct+dt^2=y$ , because passing through the origin forces the constant term to be zero. He then begins the next step, setting up a matrix equation, and starts constructing the design matrix $A$ by placing the first coordinates $1,2,-1$ into the first column. The clip ends before the full matrix system and final least-squares solution are completed. This 120-second whiteboard clip sets up a least-squares approximation problem for a quadratic through the origin, $ct+dt^2=y$ , fitted to the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . The instructor builds the matrix equation $A\widehat{x}=b$ with $A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ , $\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ , and $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ , then explains row by row why this encodes substitution of the data points into the model. He states that the exact system cannot be solved because no such quadratic passes through all three points, and reformulates the task as solving $A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)}b$ . The clip ends before any numerical least-squares solution is computed. This 120-second whiteboard excerpt sets up a least-squares problem for the model ct + $dt^2 = y$ with $A = [[1, 1], [2, 4], [-1, 1]]$ , $\hat{x} = [[c], [d]]$ , and $b = [[1], [5], [-2]]$ . The board already marks $A\hat{x}=b$ as unsolvable, and the instructor states that the real task is to solve the normal equations $A^T A \hat{x} = A^T b$ . He then computes $A^T A = [[6, 8], [8, 10]]$ and $A^T b = [[13], [19]]$ , correcting a sign-placement mistake along the way, and ends by writing the concrete system [[6, 8], [8, 10]] $\hat{x} = [[13], [19]]$ . The clip does not solve for $\hat{x}$ itself. This 120-second whiteboard segment completes a least-squares example for a quadratic fit through the origin. Starting from the already computed normal equations $A^T A\widehat{x}=A^T b$ , the instructor reduces $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ by the row operation $R_2\leftarrow 3R_2-4R_1$ , obtains the triangular system $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ , and back substitutes to find $d=-5/2$ and $c=11/2$ . He then boxes the fitted equation $y=11/2 t-5/2 t^2$ , calls it the best-fit quadratic through the origin, and recaps the workflow: identify the scalar model, write A and b, form the projection equation, and compute. The clip ends by stressing that the data points need not lie on the least-squares curve; at $t=1$ the fit gives 3 rather than the observed 1. The video clip shows the final blackboard presentation of the least squares solution process, including the matrix equation and the fitted equation y equals eleven halves t minus five halves t squared, with the instructor concluding the explanation with 'Thanks'.

Use the learning inspector for key ideas and moments, or open the reading tabs for the complete notes.

Chapters

0:00Title slide: Least Squares Approximation 0:08Problem statement on the board 0:51Choosing the quadratic model through the origin 1:26Beginning the matrix equation setup 2:00Problem statement and matrix setup 2:26Checking the rows of

A\widehat{x}=b

against the data points 3:20Why the exact system is unsolvable 3:45Least squares as projection onto the column space 4:00Unsolvable system and normal equations 4:33Computing

A^T A

5:20Computing

A^T b

5:47Final substituted 2 x 2 system 6:00Normal equations on the board 6:05Elimination step 6:20Back substitution for d and c 6:40Final boxed quadratic fit 7:10Recap of the least-squares workflow 7:43Check that data points need not lie on the fit 8:00final display of the least squares result

Learning script

Generated from the video's visuals and explanation; not verbatim speech.

The clip opens on a course title slide for MIT 18.06SC Linear Algebra, with the topic named "Least Squares Approximation." This establishes that the segment is a worked example rather than a theorem lecture.

The scene changes to a chalkboard problem statement: find the quadratic equation through the origin that is a best fit for the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . The instructor reads the problem aloud and invites the viewer to pause and try it first.

Returning to the solution, he identifies the first step as determining the form of the equation to be fitted. He writes $ct + dt^2 = y$ , using $c$ and $d$ as the unknown coefficients and $t$ and $y$ as the input and output variables.

He then explains why there is no constant term: a general quadratic would include one, but requiring the curve to pass through the origin forces that constant to be zero. This justifies the restricted model $ct+dt^2=y$ .

Next, he says the problem should be converted into a matrix equation. He begins writing a matrix $A$ and explains that the first column will contain the first coordinates of the three data points.

As he fills in the matrix, the visible first column becomes $1$ , $2$ , and $-1$ , corresponding to the $t$ -values from the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . The clip ends before the second column and the full least-squares system are completed.

The clip begins with a completed fitting problem on the left side of the board: find the quadratic through the origin that best fits $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . On the right, the model $ct+dt^2=y$ is already written, and the instructor finishes assembling the matrix form. He states that the second column of $A$ is made from the squares of the first coordinates, giving entries $1$ , $4$ , and $1$ , so the design matrix becomes $A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ . He then defines the unknown vector as $\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ and the observation vector as $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ , taken from the second coordinates of the three points.

Next he justifies the construction by checking what $A\widehat{x}$ means row by row. Multiplying the first row gives $1\cdot c+1\cdot d$ , which is exactly what one obtains by substituting the point $(1,1)$ into the left-hand side of $ct+dt^2=y$ . The second row gives $2c+4d$ , matching substitution of $(2,5)$ because $t=2$ and $t^2=4$ . In the same way, the third row corresponds to $-c+d$ for the point $(-1,-2)$ . The vector $b$ supplies the right-hand sides $1$ , $5$ , and $-2$ , so the single matrix equation $A\widehat{x}=b$ is just the compact form of the three attempted point-fitting equations.

Having established the correspondence, the instructor points out the obstruction: this exact system cannot be solved. He writes on the board that one cannot solve $A\widehat{x}=b$ , and explains verbally that no quadratic through the origin passes through all three given points. Therefore the original goal must shift from an exact fit to a best approximation in the least-squares sense.

The final idea in the clip is the geometric reformulation of that best approximation. Since every vector of the form $A\widehat{x}$ lies in the column space of $A$ , the closest attainable right-hand side to $b$ is the projection of $b$ onto $\operatorname{Col}(A)$ . Thus the least-squares problem is rewritten as $A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)}b$ . The segment ends just as he begins to refer back to earlier projection material, before any further algebra or numerical solution is shown.

The clip begins with the board already showing the model $ct+dt^2=y$ , the matrices $A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ , $\hat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ , and $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ , together with the note that $A\hat{x}=b$ cannot be solved directly.

The instructor then states the replacement rule for this least-squares setup: instead of solving the inconsistent system, solve the normal equations $A^TA\hat{x}=A^Tb$ .

He starts the computation by writing out $A^TA=\begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ .

During this step he notices that a negative sign was placed incorrectly in the transpose matrix, erases the mistaken entry, and corrects it before continuing.

With the corrected factors in place, he evaluates the product and writes $A^TA=\begin{pmatrix}6&8\\8&10\end{pmatrix}$ .

Next he computes the transformed right-hand side as $A^Tb=\begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix}$ .

He evaluates this product to obtain $A^Tb=\begin{pmatrix}13\\19\end{pmatrix}$ .

Finally, he substitutes the two computed blocks back into the normal equations, producing the concrete system $\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ .

The excerpt ends at this stage; it does not continue to solve for $\hat{x}$ or state the numerical values of $c$ and $d$ .

The clip opens on the right side of the chalkboard with the least-squares setup already computed: $A^T A = \begin{pmatrix}6&8\\8&10\end{pmatrix}$ and $A^T b = \begin{pmatrix}13\\19\end{pmatrix}$ . The instructor points to the normal-equation system $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ and says this can be solved by elimination.

He performs one explicit row operation, replacing the second row by 3 times the second row minus 4 times the first row. This is chosen to cancel the lower-left 8 while keeping the system equivalent. The board then shows the reduced triangular system $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ .

With the system now triangular, he back substitutes. The second equation gives -2d=5, so $d=-5/2$ . Substituting that value into the first equation yields $c=11/2$ . These two coefficient values are written beneath the systems on the board.

The instructor then assembles the model. Since the fit is quadratic through the origin, the scalar form is ct+ $dt^2=y$ . Inserting $c=11/2$ and $d=-5/2$ gives the final equation $y=11/2 t-5/2 t^2$ , which he writes and boxes prominently.

He identifies the boxed result as the best-fit quadratic equation through the origin. The phrase signals that the model has no constant term, so the fitted curve passes through (0,0) by construction even though it is only an approximation to the data.

The camera pans left to the earlier setup board, where the general workflow is visible: ct+ $dt^2=y$ , $A=\begin{pmatrix}1&2\\2&4\\-1&1\end{pmatrix}$ , $\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ , and $b=\begin{pmatrix}5\\1\\-2\end{pmatrix}$ . Beside them are the contrasting statements 'Can't solve $A\widehat{x}=b$ .' and 'Really solve $A^T A\widehat{x}=A^T b$ .'

Using that left board, he summarizes the method in three steps: first determine the general scalar form of the fitting equation, next rewrite the problem in matrix form with A and b, and finally set up the projection or normal equation. After that, the remaining work is computation.

Returning to interpretation, he notes that the three data points do not have to lie on the fitted quadratic. As an example, at $t=1$ the fit gives $y=3$ rather than matching the observed value 1. He treats this mismatch as expected behavior for a least-squares approximation rather than an error.

At the end of the video, the instructor concludes the lecture on the least squares method. The blackboard clearly shows the entire solution process: first, matrix multiplication yields A transpose b as thirteen nineteen transpose; next, the normal equation six eight; zero negative two times x hat equals thirteen five transpose is written. After solving for the coefficients, the constant c equals eleven halves is labeled in the lower left corner. Finally, the fitted quadratic polynomial equation y equals eleven halves t minus five halves t squared is highlighted inside a box at the bottom of the blackboard. The instructor then says 'Thanks'.

Knowledge cards

Least squares approximation problem setup

This segment introduces a concrete least-squares fitting problem in linear algebra. The task is not to interpolate exactly, but to find the quadratic curve through the origin that best fits three given data points in the least-squares sense.

Quadratic through the origin has no constant term

The instructor chooses the model $ct+dt^2=y$ . The reason the constant term is absent is that the curve must pass through the origin; equivalently, substituting $t=0$ and $y=0$ forces the constant term to be zero.

ct + dt^2 = y

First step of the matrix formulation

After fixing the model, the next step is to build a matrix equation for the unknown coefficients. The instructor begins the design matrix $A$ by placing the input coordinates $t=1,2,-1$ into the first column, preparing for the full least-squares system.

A = \left( \begin{array}{c} 1 \\ 2 \\ -1 \end{array} \right)

Quadratic-through-origin fitting model

The problem asks for a best-fit curve of the form $ct+dt^2=y$ through the origin for the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ . Here $t$ denotes the first coordinate of a data point, $y$ the second coordinate, and $c,d$ are the unknown coefficients to be determined.

ct+dt^2=y

Design matrix for the three data points

To convert the fitting problem into linear algebra, the instructor builds a matrix whose rows are $(t,t^2)$ for each point. For the given data this yields first column $1,2,-1$ and second column $1,4,1$ .

A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}

Unknown vector and observation vector

The unknown coefficient vector is $\widehat{x}=(c,d)^T$ , and the right-hand-side vector is formed from the observed $y$ -coordinates of the three points.

\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix},\quad b=\begin{pmatrix}1\\5\\-2\end{pmatrix}

Row-by-row meaning of $A\widehat{x}=b$

Each row of the matrix equation reproduces one substituted data-point equation. The first row gives $c+d=1$ , the second gives $2c+4d=5$ , and the third gives $-c+d=-2$ . This shows that $A\widehat{x}=b$ is exactly the collection of attempts to make the quadratic pass through all three points.

\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}\begin{pmatrix}c\\d\end{pmatrix}=\begin{pmatrix}1\\5\\-2\end{pmatrix}

Why the exact system is unsolvable

The instructor explicitly states that there is no quadratic through the origin containing all three points, so the exact linear system has no solution. This is the reason the problem must be treated as a least-squares approximation rather than an exact interpolation.

Least squares as projection onto the column space

Because $A\widehat{x}$ can only produce vectors in the column space of $A$ , the best approximation to an inconsistent $b$ is obtained by replacing $b$ with its projection onto $\operatorname{Col}(A)$ . The clip therefore reformulates the task as solving the projected system.

A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)} b

Normal equations replace an unsolvable system

The board first marks $A\hat{x}=b$ with 'Can't solve' and then writes 'Really solve' followed by $A^TA\hat{x}=A^Tb$ . In this clip, the instructor presents the normal equations as the actual least-squares problem to solve once the direct system is recognized as inconsistent.

A^T A \hat{x} = A^T b

Concrete matrices in the example

The example uses $A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ , $\hat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ , and $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ , with the model $ct+dt^2=y$ written above them. These are the objects manipulated throughout the rest of the clip.

A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix},\quad \hat{x}=\begin{pmatrix}c\\d\end{pmatrix},\quad b=\begin{pmatrix}1\\5\\-2\end{pmatrix}

Computing $A^T A$

The instructor forms $A^TA$ by multiplying $\begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}$ and $\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ . After correcting a sign-placement error, he writes the result as $\begin{pmatrix}6&8\\8&10\end{pmatrix}$ .

A^T A = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix} = \begin{pmatrix}6&8\\8&10\end{pmatrix}

Computing $A^T b$

The right-hand side of the normal equations is obtained from $A^Tb=\begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix}$ . The instructor evaluates this product and writes the column vector $\begin{pmatrix}13\\19\end{pmatrix}$ .

A^T b = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix} = \begin{pmatrix}13\\19\end{pmatrix}

Final substituted least-squares system

After both products are computed, the lecture rewrites the normal equations as the explicit $2\times2$ linear system $\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ . The clip ends before solving this system for $\hat{x}$ .

\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}

Sign-placement pitfall when writing $A^T$

While setting up $A^TA$ , the instructor realizes he put a negative sign in the wrong position in the transpose matrix, erases it, and corrects the entry before continuing. This is an explicit on-board caution about careful transcription during matrix multiplication.

Normal equations shown on the board

The worked example begins from the already computed least-squares system $A^T A\widehat{x}=A^T b$ . On the board this is the concrete $2\times 2$ system $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ , which is the object actually solved in the clip.

A^T A\widehat{x}=A^T b

Elimination step used to triangularize the system

To solve the normal equations, the instructor applies one row operation: replace the second row by 3 times the second row minus 4 times the first row. This converts the system into upper-triangular form while updating the right-hand side from (13,19)^T to (13,5)^T.

R_2\leftarrow 3R_2-4R_1

Back substitution gives the coefficients

From the reduced system, the second equation directly gives $d=-5/2$ . Substituting that into the first equation gives $c=11/2$ . These are the two coefficients of the fitted quadratic model.

d=-\frac{5}{2},\quad c=\frac{11}{2}

Final fitted quadratic through the origin

Substituting the solved coefficients into the model ct+ $dt^2=y$ produces the boxed answer $y=11/2 t-5/2 t^2$ . The instructor explicitly describes this as the best-fit quadratic equation through the origin.

y=\frac{11}{2}t-\frac{5}{2}t^2

Workflow recap for least squares

The left board summarizes the broader method: start with the general scalar form, translate it into matrix form using A and b, then solve the normal equations rather than the inconsistent direct system $A\widehat{x}=b$ .

ct+dt^2=y,\quad A\widehat{x}\neq b\ \text{directly},\quad A^T A\widehat{x}=A^T b

Least-squares fit need not pass through the data

The closing remark stresses an important conceptual point: the fitted curve is an approximation, not an interpolant. In this example, plugging in $t=1$ gives $y=3$ , not the observed value 1, and that is acceptable for a least-squares fit.

y(1)=3\neq 1

least squares fitted equation

Solving the normal equations A transpose A x hat equals A transpose b yields the coefficients for the least squares fit. The video shows the computation and ultimately arrives at the fitted curve equation.

y = \frac{11}{2}t - \frac{5}{2}t^2

Detailed learning notes

Explore conditions, steps and evidence. Supplementary explanations are labeled separately from content shown in the video.

Symbols · 36

c

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the equation will look like "ct plus dt squared equals y".
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Symbol

Meaning

Unknown coefficient multiplying the linear term $t$ in the quadratic model.

Domain

Scalar unknown to be solved for in the least-squares problem.

d

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the equation will look like "ct plus dt squared equals y".
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Symbol

Meaning

Unknown coefficient multiplying the quadratic term $t^2$ in the model.

Domain

Scalar unknown to be solved for in the least-squares problem.

t

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the equation will look like "ct plus dt squared equals y".
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Symbol

Meaning

Independent variable (input coordinate) in the quadratic equation.

Domain

Real-valued input; in this example it takes the values $1$ , $2$ , and $-1$ from the given points.

y

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the equation will look like "ct plus dt squared equals y".
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Symbol

Meaning

Dependent variable (output coordinate) to be approximated by the quadratic model.

Domain

Real-valued output; in this example it takes the values $1$ , $5$ , and $-2$ from the given points.

A

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, "Next we need to set up a matrix equation. So we need an A, a matrix A."
Formula
Observation
He writes $A=$ followed by large parentheses and begins filling the first column.

Uncertainties

The full matrix is not completed within the provided clip.

Symbol

Meaning

Design matrix for the least-squares system $A\begin{bmatrix}c\\d\end{bmatrix}=y$ .

Domain

Matrix whose first column is being populated with the $t$ -coordinates of the data points.

ct+ $dt^2=y$

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows the model form $ct+dt^2=y$ at the top.

Symbol

ct+ $dt^2=y$

Meaning

Quadratic equation through the origin used as the fitting model, with unknown coefficients $c$ and $d$ .

Domain

$t$ is the first coordinate of a data point and $y$ is the second coordinate; $c,d$ are scalar coefficients to be determined.

A

Clear evidence

Shown in the video

Evidence

Formula
Observation
The instructor writes $A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$ on the board.
Audio
Observation
He says the second column is the squares of the first coordinates, giving $1$ , $4$ , and $1$ .

Symbol

Meaning

Design matrix whose first column is the $t$ -values of the data points and whose second column is their squares.

Domain

A $3\times 2$ real matrix.

$\widehat{x}$

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ .
Audio
Observation
He says, "my x hat, that's just c and d."

Symbol

$\widehat{x}$

Meaning

Unknown coefficient vector containing the quadratic-fit parameters.

Domain

A $2\times 1$ real vector.

b

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ .
Audio
Observation
He says this is going to be the second coordinates, $1$ , $5$ , and negative $2$ .

Symbol

Meaning

Right-hand-side vector formed from the $y$ -coordinates of the three given points.

Domain

A $3\times 1$ real vector.

(1,1), (2,5), (-1,-2)

Clear evidence

Shown in the video

Evidence

Formula
Observation
The left side of the board states the problem for the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .

Symbol

(1,1), (2,5), (-1,-2)

Meaning

The three data points to be fitted by a quadratic through the origin.

Domain

Points in the plane with first coordinate $t$ and second coordinate $y$ .

column space of A

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says solving the best approximation is the same as solving $A\widehat{x}$ equals the projection of $b$ onto the column space of $A$ .

Symbol

column space of A

Meaning

The span of the columns of $A$ , i.e. all vectors expressible as $A\widehat{x}$ for some $\widehat{x}$ .

Domain

A subspace of $\mathbb{R}^3$ .

A

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $A = [[1, 1], [2, 4], [-1, 1]]$ at the start and uses this matrix again in the computations of $A^T A$ and $A^T b$ .

Symbol

Meaning

The coefficient matrix for the overdetermined linear system being replaced by a least-squares problem.

Domain

A is a 3 x 2 real matrix.

Knowledge points · 18

Topic: Least Squares Approximation

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, "Today we're going to do a problem about least squares approximations."
Caption evidence
Observation
Title slide reads "Least Squares Approximation".

Definition

Explanation

The clip introduces a worked problem in least squares approximation, specifically fitting a quadratic curve to data points.

Formula

Conditions

The video presents this as a problem-solving session rather than a formal definition lecture.

Quadratic model through the origin

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, "our equation is going to look like ct plus dt squared equals y" and then explains that through the origin means the constant term is zero.
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Formula

Explanation

The fitted quadratic is written without a constant term because the curve must pass through the origin. The unknowns are the coefficients $c$ and $d$ .

Formula

ct + dt^2 = y

Conditions

The model is required to pass through the origin.
The general constant term is omitted because it must equal zero.

Least-squares fitting problem for a quadratic through the origin

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board text reads: find the quadratic equation through the origin that is a best fit for the points $(1,1)$ , $(2,5)$ , $(-1,-2)$ .
Formula
Observation
The model equation $ct+dt^2=y$ is written above the matrices.
Audio
Observation
The instructor explains that the second column is the squares of the first coordinates and that $b$ uses the second coordinates.

Definition

Explanation

The clip sets up a least-squares approximation problem: instead of requiring exact passage through the data points, one seeks coefficients $c,d$ in the model $ct+dt^2=y$ that give the best fit to the three given points. The instructor converts this into matrix form by placing the $t$ -values and their squares into the columns of $A$ , the unknowns into $\widehat{x}$ , and the observed $y$ -values into $b$ .

Formula

ct+dt^2=y,\quad A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix},\quad \widehat{x}=\begin{pmatrix}c\\d\end{pmatrix},\quad b=\begin{pmatrix}1\\5\\-2\end{pmatrix}

Conditions

The fitting model is quadratic through the origin.
There are three data points.
The goal is a best fit rather than an exact solution.

Why each row of $A\widehat{x}=b$ encodes one data point

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says the first coordinate of $A\widehat{x}$ is $1$ times $c$ plus $1$ times $d$ , matching the first point substituted into the left-hand side.
Audio
Observation
He says the second point gives $2$ times $c$ plus $4$ times $d$ , matching the second coordinate of $A\widehat{x}$ .
Audio
Observation
He says $b$ came from plugging the points into the right-hand side, so $1$ , $5$ , and $-2$ are the $y$ -coordinates.

Method

Explanation

The instructor verifies the setup row by row. For a point $(t,y)$ , substituting into $ct+dt^2=y$ gives the scalar equation $tc+d(t^2)=y$ . In matrix form, the row $(t,t^2)$ of $A$ multiplied by $(c,d)^T$ reproduces exactly that left-hand side, while the corresponding entry of $b$ is the observed $y$ . Thus the three rows encode the three attempted equations from the three data points.

Formula

\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}\begin{pmatrix}c\\d\end{pmatrix}=\begin{pmatrix}1\\5\\-2\end{pmatrix}

Conditions

Each data point contributes one row to $A$ and one entry to $b$ .
The first column of $A$ stores $t$ values and the second column stores $t^2$ values.

Prerequisites

Least-squares fitting problem for a quadratic through the origin

The exact system $A\widehat{x}=b$ is inconsistent here

Clear evidence

Shown in the video

Evidence

Formula
Observation
The instructor writes on the board: Can't solve $A\widehat{x}=b$ .
Audio
Observation
He says it would be great if we could solve $Ax$ hat equals $b$ , but we can't because there isn't a quadratic equation through the origin that contains these three points.

Method

Explanation

The clip explicitly states that the constructed linear system cannot be solved exactly because no quadratic through the origin passes through all three given points. This is the motivation for replacing an exact solution by a least-squares approximation.

Formula

A\widehat{x}=b\ \text{has no exact solution for this data}

Conditions

The data points are $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .
The model is restricted to quadratics through the origin.

Prerequisites

Why each row of $A\widehat{x}=b$ encodes one data point

Least-squares solution as projection onto the column space

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says finding the best approximation is the same as solving $A\widehat{x}$ equals the projection of $b$ onto the column space of $A$ .
Audio
Observation
He adds that we only have a chance of solving $A\widehat{x}$ equals something if it's in the column space of $A$ .

Uncertainties

The sentence continues beyond the clip, so the next derivation step is not present in this segment.

Definition

Explanation

Because $A\widehat{x}$ can only produce vectors in the column space of $A$ , the best possible approximation to $b$ is obtained by replacing $b$ with its projection onto that column space. Solving the projected equation gives the least-squares coefficients.

Formula

A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)} b

Conditions

$b$ need not lie in $\operatorname{Col}(A)$ .
The target vector on the right-hand side must be in $\operatorname{Col}(A)$ for the equation $A\widehat{x}=\cdot$ to be solvable.

Prerequisites

The exact system $A\widehat{x}=b$ is inconsistent here

Normal equations for least squares

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says this is the same as solving A transpose A x hat equals A transpose b.
Formula
Observation
He writes 'Really solve' and then $A^T A \hat{x} = A^T b$ on the board.

Formula

Explanation

For an inconsistent system $A\hat{x}=b$ , the video states that one should instead solve the normal equations $A^T A \hat{x} = A^T b$ . In this clip the instructor presents this as the actual problem to solve after noting that $A\hat{x}=b$ cannot be solved directly.

Formula

A^T A \hat{x} = A^T b

Conditions

The starting system is written as $A\hat{x}=b$ .
The video treats $A\hat{x}=b$ as unsolvable directly and replaces it by the normal equations.
No explicit proof of equivalence is given in this clip.

Prerequisites

Direct system $A\hat{x}=b$ is not solvable

Direct system $A\hat{x}=b$ is not solvable

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board already contains 'Can't solve' above $A\hat{x}=b$ when the clip begins.
Audio
Observation
The instructor contrasts the unsolvable system with what they are 'really going to solve.'

Uncertainties

The clip does not show the row-reduction or inconsistency check that proves $A\hat{x}=b$ has no solution.

Method

Explanation

The board explicitly labels the original system $A\hat{x}=b$ with 'Can't solve,' and the lecture then moves to the normal equations rather than attempting to solve that system directly.

Formula

A\hat{x} = b

Conditions

This item records only the video's statement that the displayed system is not directly solvable.
The reason for inconsistency is not demonstrated within this clip.

Computation of $A^T A$

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor asks, 'So what is A transpose A?'
Formula
Observation
He writes $A^T A = [[1, 2, -1], [1, 4, 1]] [[1, 1], [2, 4], [-1, 1]]$ and then gives the product [[6, 8], [8, 10]].

Method

Explanation

The lecture performs the matrix product $A^T A$ by multiplying the displayed 2 x 3 transpose matrix with the displayed 3 x 2 matrix A. The resulting 2 x 2 matrix is written on the board as [[6, 8], [8, 10]].

Formula

A^T A = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix} = \begin{pmatrix}6&8\\8&10\end{pmatrix}

Conditions

Use the specific matrices shown on the board.
The multiplication is presented as routine computation rather than derived from a theorem in this clip.

Prerequisites

A
$A^T$
Normal equations for least squares

Computation of $A^T b$

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor asks, 'And what is A transpose b?'
Formula
Observation
He writes $A^T b = [[1, 2, -1], [1, 4, 1]] [[1], [5], [-2]]$ and then gives the result [[13], [19]].

Method

Explanation

The lecture computes the transformed right-hand side by multiplying $A^T$ with b. The displayed product yields the column vector [[13], [19]].

Formula

A^T b = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix} = \begin{pmatrix}13\\19\end{pmatrix}

Conditions

Use the specific matrices and vector shown on the board.
The clip does not separately verify the arithmetic beyond asking students to check it.

Prerequisites

b
$A^T$
Normal equations for least squares

Substituted normal-equation system

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, 'what we're solving here is 6 8 8 10 times x hat equals 13 19.'
Formula
Observation
He writes $\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}$ .

Uncertainties

The clip ends before solving for $\hat{x}$ explicitly.

Formula

Explanation

After computing both sides of the normal equations, the lecture rewrites the problem as a concrete 2 x 2 linear system for $\hat{x}$ . This is the final displayed state of the example in the clip.

Formula

\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}

Conditions

This follows from substituting the computed $A^T A$ and $A^T b$ into $A^T A \hat{x} = A^T b$ .
The solution vector itself is not reached within the provided duration.

Prerequisites

Normal equations for least squares
Computation of $A^T A$
Computation of $A^T b$

Normal equations for least squares

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board displays $A^T A = \begin{pmatrix}6&8\\8&10\end{pmatrix}$ , $A^T b = \begin{pmatrix}13\\19\end{pmatrix}$ , and then $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ .
Audio
Observation
At 01:34.880-01:39.200 the instructor says, 'And lastly, you set up your projection equation.'

Formula

Explanation

This clip uses the normal equations as the computational form of the least-squares problem. The visible matrices show that the overdetermined fit is converted into a square $2\times 2$ system for the coefficient vector.

Formula

A^T A\widehat{x}=A^T b

Conditions

The fit is being solved in coefficient form $\widehat{x}$ .
The displayed instance has $A^T A = \begin{pmatrix}6&8\\8&10\end{pmatrix}$ and $A^T b = \begin{pmatrix}13\\19\end{pmatrix}$ .

Claims and conditions · 7

Through the origin implies zero constant term

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, "Now if it were just any quadratic equation then we would have a constant term, but through the origin just means that the constant term is zero."

Proposition

Statement

For a quadratic equation, requiring the graph to pass through the origin means the constant term is zero.

Hypotheses

The equation is quadratic.
The graph passes through the origin.

Quantifiers

For the quadratic model under discussion.

No exact quadratic through the origin fits all three points

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor states there isn't a quadratic equation through the origin that contains these three points.
Formula
Observation
He writes: Can't solve $A\widehat{x}=b$ .

Proposition

Statement

For the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ , there is no choice of coefficients $c,d$ such that $ct+dt^2=y$ holds for all three points simultaneously.

Hypotheses

The model is restricted to $ct+dt^2=y$ .
The data set consists of the three listed points.

Quantifiers

For all real $c,d$ , the three equations are not simultaneously satisfied.

Best approximation is obtained by projecting $b$ onto $\operatorname{Col}(A)$

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says that finding the best approximation is the same as solving $A\widehat{x}$ equals the projection of $b$ onto the column space of $A$ .

Uncertainties

The clip does not include the proof or the subsequent normal-equation derivation.

Proposition

Statement

When $A\widehat{x}=b$ is inconsistent, the least-squares approximation is found by solving $A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)}b$ .

Hypotheses

$A$ is the design matrix built from the data.
$b$ is the observation vector.
The goal is the best approximation in the least-squares sense.

Quantifiers

For the given inconsistent system, the optimal right-hand side is the projection of $b$ onto the column space of $A$ .

Least-squares problem is solved via normal equations

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor states that the problem is 'the same as solving A transpose A x hat equals A transpose b.'
Formula
Observation
The board changes from 'Can't solve $A\hat{x}=b$ ' to 'Really solve $A^T A \hat{x} = A^T b$ .'

Uncertainties

The clip asserts the replacement rule but does not prove it here.

Proposition

Statement

For the displayed inconsistent system $A\hat{x}=b$ , the lecture states that one should solve $A^T A \hat{x} = A^T b$ instead.

Hypotheses

$A\hat{x}=b$ is the starting system.
The system is treated as not directly solvable.
The instructor identifies the normal equations as the actual problem to solve.

Quantifiers

The claim is made for the specific displayed matrices A, $\hat{x}$ , and b in this example.

Explicit row operation used in elimination

Clear evidence

Shown in the video

Evidence

Audio
Observation
At 00:05.840-00:11.840 the instructor states, 'We replace the second row by three times the second row minus four times the first row.'
Formula
Observation
The written transition from $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ to $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ matches that operation.

Proposition

Statement

For the displayed normal-equation system, replacing the second row by 3 times the second row minus 4 times the first row yields the reduced system $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ .

Hypotheses

The starting system is $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ .

Quantifiers

Applies to the specific $2\times 2$ system shown on the board.

Solved coefficient values

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $d = -5/2$ and $c = 11/2$ .
Audio
Observation
At 00:26.800-00:39.920 the instructor says, 'D is negative 5 halves, and C equals ... I think you're going to get 11 over 2.'

Proposition

Statement

The solution of the reduced normal-equation system is $c=11/2$ and $d=-5/2$ .

Hypotheses

The reduced system is $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ .

Quantifiers

For the displayed example only.

Best-fit curve need not pass through the data points

Clear evidence

Shown in the video

Evidence

Audio
Observation
At 01:44.480-01:52.240 the instructor says, 'Good, it might also be worth noting that these three points certainly aren't on this quadratic equation.'
Audio
Observation
At 01:52.240-01:56.640 he says, 'For instance, if I plug in one here, I don't get one one, I get one three.'

Uncertainties

The exact coordinates of all three data points are not fully restated in this clip, although $b=(5,1,-2)^T$ is visible earlier.

Proposition

Statement

The fitted quadratic does not necessarily pass through the original data points; at $t=1$ the fit gives $y=3$ rather than matching the observed point with $y=1$ .

Hypotheses

The fitted equation is $y=11/2 t - 5/2 t^2$ .
One observed data value corresponds to $t=1$ , $y=1$ .

Quantifiers

Stated for the three points in this example, with $t=1$ given as an instance.

Derivations and proofs · 8

Setting up the quadratic model from the problem statement

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the first step is figuring out what the equation looks like, then states $ct+dt^2=y$ and explains the missing constant term.
Formula
Observation
He writes $ct + dt^2 = y$ on the board.

Intuitive argument

Steps

Expression
$ct + dt^2 = y$
Explanation
The instructor chooses a quadratic form with unknown coefficients $c$ and $d$ .
Justification
This matches the requested quadratic fit.
Shown in the video
Expression
Explanation
He notes that a general quadratic would include a constant term, but this one does not.
Justification
The problem requires the curve to pass through the origin, so the constant term is zero.
Shown in the video

Conclusion

The appropriate model for this problem is $ct + dt^2 = y$ .

Beginning the matrix equation setup

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, "Next we need to set up a matrix equation. So we need an A, a matrix A," then explains he will put the first coordinates into the first column.
Formula
Observation
He writes $A=$ and begins filling the first column with $1$ , $2$ , and $-1$ .

Uncertainties

The second column of $A$ is not shown before the clip ends.
The full matrix equation is not completed in this segment.

Intuitive argument

Steps

Expression
$A = ( \; )$
Explanation
The instructor starts writing a matrix $A$ for the least-squares system.
Justification
A matrix equation is needed to solve for the unknown coefficients.
Shown in the video
Expression
$\begin{pmatrix}1\\2\\-1\end{pmatrix}$
Explanation
He places the first coordinates of the three points into the first column of $A$ .
Justification
These are the values of $t$ corresponding to the data points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .
Shown in the video

Conclusion

The design matrix $A$ is being constructed column by column; the first column is the vector of $t$ -values.

Derivation of the matrix equation from pointwise substitution

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor verbally multiplies out the first and second rows of $A\widehat{x}$ and compares them with direct substitution into $ct+dt^2=y$ .
Formula
Observation
The board displays $A$ , $\widehat{x}$ , and $b$ while he points between the matrices and the original point list.

Intuitive argument

Steps

Expression
$ct+dt^2=y$
Explanation
Start from the quadratic-through-origin model for a generic point $(t,y)$ .
Justification
This is the stated fitting model on the board.
Shown in the video
Expression
$(1)c+(1)d=1$
Explanation
Substitute the first point $(1,1)$ into the model.
Justification
Direct substitution of $t=1$ , $y=1$ .
Derived from the video
Expression
$(2)c+(4)d=5$
Explanation
Substitute the second point $(2,5)$ into the model.
Justification
Direct substitution of $t=2$ , $y=5$ , using $t^2=4$ .
Derived from the video
Expression
$(-1)c+(1)d=-2$
Explanation
Substitute the third point $(-1,-2)$ into the model.
Justification
Direct substitution of $t=-1$ , $y=-2$ , using $t^2=1$ .
Derived from the video
Expression
$\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}\begin{pmatrix}c\\d\end{pmatrix}=\begin{pmatrix}1\\5\\-2\end{pmatrix}$
Explanation
Collect the three scalar equations into one matrix equation.
Justification
Each row of $A$ stores $(t,t^2)$ and each entry of $b$ stores the corresponding $y$ .
Shown in the video

Conclusion

The matrix equation $A\widehat{x}=b$ is exactly the compact form of the three attempted point-fitting equations.

From inconsistent exact fit to projected least-squares system

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says we cannot solve $A\widehat{x}=b$ exactly, then reformulates the task as solving $A\widehat{x}$ equal to the projection of $b$ onto the column space of $A$ .
Formula
Observation
The board text 'Can't solve $A\widehat{x}=b$ ' introduces the obstruction.

Uncertainties

The clip stops before any algebraic computation of the projection or normal equations.

Intuitive argument

Steps

Expression
$A\widehat{x}=b$
Explanation
Begin with the exact system encoding all three data points.
Justification
This is the matrix form derived from substitution.
Shown in the video
Expression
$\text{No exact solution exists}$
Explanation
Recognize that the system is inconsistent for the given data.
Justification
The instructor explicitly states there is no quadratic through the origin containing all three points.
Shown in the video
Expression
$A\widehat{x}\in \operatorname{Col}(A)$
Explanation
Note that any vector of the form $A\widehat{x}$ must lie in the column space of $A$ .
Justification
Definition of column space as the set of all matrix-vector products with $A$ .
Derived from the video
Expression
$A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)} b$
Explanation
Replace $b$ by its projection onto $\operatorname{Col}(A)$ to obtain the closest attainable right-hand side.
Justification
Stated directly by the instructor as the meaning of the best approximation.
Shown in the video

Conclusion

The least-squares problem is reformulated as a solvable system whose right-hand side lies in the column space of $A$ .

From normal equations to a concrete 2 x 2 system

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, 'now all we have to do is just a computation' and then computes $A^T A$ and $A^T b$ .
Formula
Observation
The board successively shows $A^T A$ = ..., $A^T b$ = ..., and finally the substituted system.

Uncertainties

The derivation stops at the substituted linear system; solving for $\hat{x}$ is not included in the clip.

Numerical verification

Steps

Expression
$A^T A \hat{x} = A^T b$
Explanation
Start from the normal equations stated on the board.
Justification
This is the formula the instructor writes under 'Really solve'.
Shown in the video
Expression
$A^T A = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$
Explanation
Insert the displayed transpose and original matrix into the left-hand side.
Justification
These are the matrices written on the board for $A^T$ and A.
Shown in the video
Expression
$A^T A = \begin{pmatrix}6&8\\8&10\end{pmatrix}$
Explanation
Evaluate the matrix product to obtain the 2 x 2 coefficient matrix.
Justification
The instructor writes this result and says students can check it.
Shown in the video
Expression
$A^T b = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix}$
Explanation
Insert the displayed transpose and right-hand-side vector into the right-hand side.
Justification
These are the matrices and vector written on the board.
Shown in the video
Expression
$A^T b = \begin{pmatrix}13\\19\end{pmatrix}$
Explanation
Evaluate the product to obtain the transformed right-hand side.
Justification
The instructor writes this result and says students can check it.
Shown in the video
Expression
$\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}$
Explanation
Substitute the computed blocks back into the normal equations.
Justification
This is the final displayed system at the end of the clip.
Shown in the video

Conclusion

The example reduces the abstract normal equations to the explicit linear system [[6, 8], [8, 10]] $\hat{x} = [[13], [19]]$ .

Derivation of the coefficients from the normal equations

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $A^T A$ , $A^T b$ , the normal-equation system, the reduced triangular system, and the final values $d=-5/2$ , $c=11/2$ .
Audio
Observation
The instructor narrates elimination, changing the b vector, and back solving.

Proof

Steps

Expression
$A^T A=\begin{pmatrix}6&8\\8&10\end{pmatrix},\quad A^T b=\begin{pmatrix}13\\19\end{pmatrix}$
Explanation
The least-squares problem is first converted into the normal-equation system.
Justification
Visible board computation of $A^T A$ and $A^T b$ .
Shown in the video
Expression
$\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$
Explanation
This is the explicit $2\times 2$ linear system to be solved for the coefficient vector.
Justification
Written directly on the board below the computed products.
Shown in the video
Expression
$R_2\leftarrow 3R_2-4R_1$
Explanation
The instructor applies one elimination step to remove the lower-left entry.
Justification
Spoken explicitly at 00:05.840-00:11.840 and consistent with the written result.
Shown in the video
Expression
$\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$
Explanation
The transformed system is upper triangular, with the right-hand side updated simultaneously.
Justification
Written on the board after the reduction arrow.
Shown in the video
Expression
$-2d=5\Rightarrow d=-\frac{5}{2}$
Explanation
The second row gives the lower coefficient directly by back substitution.
Justification
Visible final writing $d=-5/2$ and spoken statement 'd is negative 5 halves'.
Shown in the video
Expression
$6c+8\left(-\frac{5}{2}\right)=13\Rightarrow c=\frac{11}{2}$
Explanation
Substituting d into the first row yields the remaining coefficient.
Justification
The intermediate substitution is not fully written out, but the board shows $c=11/2$ and the audio says to plug it in.
Derived from the video

Conclusion

The normal-equation system solves to $\widehat{x}=\begin{pmatrix}11/2\\-5/2\end{pmatrix}$ , giving the fitted coefficients $c=11/2$ and $d=-5/2$ .

Assembly of the fitted quadratic from the solved coefficients

Clear evidence

Shown in the video

Evidence

Formula
Observation
The boxed equation $y=11/2 t - 5/2 t^2$ appears after the coefficient values are written.
Audio
Observation
The instructor asks for the final equation and then states it aloud.

Intuitive argument

Steps

Expression
$ct+dt^2=y$
Explanation
The model form is quadratic through the origin with coefficients c and d.
Justification
Visible on the left board during the recap and consistent with the final equation structure.
Shown in the video
Expression
$c=\frac{11}{2},\ d=-\frac{5}{2}$
Explanation
Insert the solved coefficient values from the normal equations.
Justification
Previously derived on the board.
Shown in the video
Expression
$y=\frac{11}{2}t-\frac{5}{2}t^2$
Explanation
This is the resulting best-fit quadratic equation.
Justification
Boxed on the board and spoken aloud.
Shown in the video

Conclusion

Substituting the solved coefficients into the model produces the final fitted equation $y=11/2 t - 5/2 t^2$ .

derivation from the matrix equation to the fitted equation

Clear evidence

Shown in the video

Evidence

Formula
Observation
The blackboard displays, from top to bottom, matrix multiplication, the normal equations, and finally a scalar equation

Uncertainties

the specific derivation steps are not verbally explained in the video, only the written results are shown

Proof

Steps

Expression
$\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix} \begin{pmatrix} 1 \\ 5 \\ -2 \end{pmatrix} = \begin{pmatrix} 13 \\ 19 \end{pmatrix}$
Explanation
compute the value of A transpose b
Justification
definition of matrix multiplication
Shown in the video
Expression
$\begin{pmatrix} 6 & 8 \\ 0 & -2 \end{pmatrix} \hat{x} = \begin{pmatrix} 13 \\ 5 \end{pmatrix}$
Explanation
set up the normal equations A transpose A x hat equals A transpose b
Justification
principle of the least squares method
Shown in the video
Expression
$y = \frac{11}{2}t - \frac{5}{2}t^2$
Explanation
substitute the computed coefficients into the polynomial to obtain the final fitted equation
Justification
polynomial representation
Shown in the video

Conclusion

the least squares fitted equation y equals eleven halves t minus five halves t squared has been obtained

Worked examples · 4

Best-fit quadratic through the origin for three points

Clear evidence

Shown in the video

Evidence

Caption evidence
Observation
Board text reads: "Find the quadratic equation through the origin that is a best fit for the points (1,1), (2,5), (-1,-2)."
Audio
Observation
The instructor reads the same problem aloud and begins solving it.

Uncertainties

The final least-squares solution is not reached within the provided clip.
Only the beginning of the matrix setup is shown.

Problem

Find the quadratic equation through the origin that is a best fit for the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .

Given

Data points: $(1,1)$ , $(2,5)$ , $(-1,-2)$ .
The quadratic must pass through the origin.
The fit is in the least-squares sense.

Goal

Set up the quadratic model and begin forming the matrix equation used to find the best-fit coefficients.

Steps

Expression
$ct + dt^2 = y$
Explanation
Choose a quadratic model with no constant term.
Justification
Passing through the origin forces the constant term to be zero.
Shown in the video
Expression
$A = \left( \begin{array}{c} 1 \\ 2 \\ -1 \end{array} \right)$
Explanation
Begin the design matrix by placing the $t$ -coordinates in the first column.
Justification
These are the inputs corresponding to the three data points.
Shown in the video

Answer

The clip only reaches the partial setup: the model is $ct+dt^2=y$ , and the first column of $A$ is $\begin{pmatrix}1\\2\\-1\end{pmatrix}$ .

Verification

No final numerical verification is shown in this segment.

Setting up a least-squares quadratic fit through the origin

Clear evidence

Shown in the video

Evidence

Formula
Observation
The left panel states the problem for the points $(1,1)$ , $(2,5)$ , $(-1,-2)$ .
Formula
Observation
The right panel shows $ct+dt^2=y$ , $A$ , $\widehat{x}$ , and $b$ .
Audio
Observation
The instructor explains how the columns of $A$ and the entries of $b$ are chosen from the data.

Uncertainties

The final numerical least-squares coefficients are not reached within this clip.

Problem

Find the quadratic equation through the origin that best fits the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .

Given

Model form: $ct+dt^2=y$ .
Data points: $(1,1)$ , $(2,5)$ , $(-1,-2)$ .
First column of $A$ uses the $t$ -coordinates.
Second column of $A$ uses the squared $t$ -coordinates.
Vector $b$ uses the $y$ -coordinates.

Goal

Translate the fitting problem into matrix form and identify why an exact solve is impossible.

Steps

Expression
$A=\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$
Explanation
Build the design matrix from $t$ and $t^2$ for the three points.
Justification
For $(1,1)$ , $(2,5)$ , $(-1,-2)$ , the first coordinates and their squares are $(1,1)$ , $(2,4)$ , $(-1,1)$ .
Shown in the video
Expression
$\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$
Explanation
Place the unknown coefficients into the solution vector.
Justification
The model has exactly two unknown parameters, $c$ and $d$ .
Shown in the video
Expression
$b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$
Explanation
Use the observed second coordinates as the right-hand side.
Justification
These are the $y$ -values of the three data points.
Shown in the video
Expression
$\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}\begin{pmatrix}c\\d\end{pmatrix}=\begin{pmatrix}1\\5\\-2\end{pmatrix}$
Explanation
Write the full exact-fit system.
Justification
Each row corresponds to substituting one point into $ct+dt^2=y$ .
Derived from the video
Expression
$\text{Can't solve }A\widehat{x}=b$
Explanation
Conclude that the exact system is inconsistent.
Justification
The instructor states no quadratic through the origin contains all three points.
Shown in the video
Expression
$A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)} b$
Explanation
Reformulate the task as a least-squares projection problem.
Justification
Only vectors in the column space of $A$ can be written as $A\widehat{x}$ , so the best approximation projects $b$ into that space.
Shown in the video

Answer

The clip obtains the matrix setup $A\widehat{x}=b$ and identifies the least-squares reformulation $A\widehat{x}=\operatorname{proj}_{\operatorname{Col}(A)}b$ ; it does not compute the final coefficients within the provided duration.

Verification

Row-by-row multiplication of $A\widehat{x}$ reproduces the three substituted equations from the model $ct+dt^2=y$ , confirming the setup.

Least-squares setup for ct + $dt^2 = y$

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows ct + $dt^2 = y$ , $A = [[1, 1], [2, 4], [-1, 1]]$ , $\hat{x} = [[c], [d]]$ , $b = [[1], [5], [-2]]$ , 'Can't solve $A\hat{x}=b$ ', and 'Really solve $A^T A \hat{x} = A^T b$ '.
Audio
Observation
The instructor says they are really going to solve A transpose A x hat equals A transpose b and then performs the computation.

Uncertainties

The clip does not show the final solution for c and d.
The mapping from the model ct + $dt^2 = y$ to the rows of A and b is implied by the setup but not verbally expanded in this excerpt.

Problem

Given the model ct + $dt^2 = y$ with $A = [[1, 1], [2, 4], [-1, 1]]$ , $\hat{x} = [[c], [d]]$ , and $b = [[1], [5], [-2]]$ , the direct system $A\hat{x}=b$ is labeled unsolvable; form and compute the corresponding normal equations.

Given

ct + $dt^2 = y$
$A = \begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix}$
$\hat{x} = \begin{pmatrix}c\\d\end{pmatrix}$
$b = \begin{pmatrix}1\\5\\-2\end{pmatrix}$
The board states 'Can't solve $A\hat{x}=b$ '.

Goal

Replace the unsolvable system by the normal equations and compute the concrete 2 x 2 system to be solved.

Steps

Expression
$A^T A \hat{x} = A^T b$
Explanation
Write the normal equations as the actual problem to solve.
Justification
The instructor states this is what they are 'really going to solve' and writes it on the board.
Shown in the video
Expression
$A^T A = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1&1\\2&4\\-1&1\end{pmatrix} = \begin{pmatrix}6&8\\8&10\end{pmatrix}$
Explanation
Compute the left-hand coefficient matrix of the normal equations.
Justification
This multiplication is performed explicitly on the board.
Shown in the video
Expression
$A^T b = \begin{pmatrix}1&2&-1\\1&4&1\end{pmatrix}\begin{pmatrix}1\\5\\-2\end{pmatrix} = \begin{pmatrix}13\\19\end{pmatrix}$
Explanation
Compute the right-hand side of the normal equations.
Justification
This multiplication is performed explicitly on the board.
Shown in the video
Expression
$\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}$
Explanation
Substitute the computed pieces back into the normal equations.
Justification
This is the final displayed system at the end of the clip.
Shown in the video

Answer

The clip ends with the concrete normal-equation system $\begin{pmatrix}6&8\\8&10\end{pmatrix}\hat{x} = \begin{pmatrix}13\\19\end{pmatrix}$ ; it does not display the solved vector $\hat{x}$ .

Verification

The instructor tells students to check the products 6, 8, 8, 10 and 13, 19, but no separate verification step is shown in the clip.

Worked least-squares fit of a quadratic through the origin

Clear evidence

Shown in the video

Evidence

Formula
Observation
The full worked example is visible across the board: $A^T A$ , $A^T b$ , the normal equations, the reduced system, $c=11/2$ , $d=-5/2$ , and the boxed final equation.
Audio
Observation
The instructor narrates elimination, back substitution, the final equation, and a recap of the method.

Uncertainties

The original data table is not fully rewritten in this clip, though $b=(5,1,-2)^T$ is visible and one point with $t=1$ , $y=1$ is discussed verbally near the end.

Problem

Solve the displayed least-squares problem for a quadratic fit through the origin using the normal equations.

Given

$A^T A=\begin{pmatrix}6&8\\8&10\end{pmatrix}$
$A^T b=\begin{pmatrix}13\\19\end{pmatrix}$
The normal-equation system is $\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$ .

Goal

Find the coefficients and write the final fitted quadratic equation.

Steps

Expression
$\begin{pmatrix}6&8\\8&10\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\19\end{pmatrix}$
Explanation
Start from the normal equations already computed on the board.
Justification
Visible formula at the beginning of the clip.
Shown in the video
Expression
$R_2\leftarrow 3R_2-4R_1$
Explanation
Apply elimination to create a zero in the lower-left position.
Justification
Spoken explicitly by the instructor.
Shown in the video
Expression
$\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$
Explanation
Record the reduced triangular system after updating both rows and the right-hand side.
Justification
Written on the board after the reduction arrow.
Shown in the video
Expression
$d=-\frac{5}{2}$
Explanation
Solve the second equation for the lower coefficient.
Justification
Written and spoken during back substitution.
Shown in the video
Expression
$c=\frac{11}{2}$
Explanation
Substitute d into the first equation and solve for c.
Justification
Written on the board; the intermediate algebra is mostly spoken rather than fully expanded.
Shown in the video
Expression
$y=\frac{11}{2}t-\frac{5}{2}t^2$
Explanation
Insert c and d into the quadratic model and box the result.
Justification
Final boxed equation on the board.
Shown in the video

Answer

$y=\frac{11}{2}t-\frac{5}{2}t^2$

Verification

The instructor checks conceptually that the fit is not required to pass through the data points, noting that at $t=1$ the fitted value is 3 rather than 1.

Visual events · 11

Course title slide

Clear evidence

Shown in the video

Evidence

Caption evidence
Observation
A title slide shows "18.06SC, Fall 2011", "Linear Algebra", "Ben Harris, Teaching Assistant", and "Least Squares Approximation".

Objects

MIT logo
course title text
topic title text

Changes

Static title card is displayed before the chalkboard scene.

Invariants

No mathematical derivation is shown yet.

Interpretation

The slide identifies the lecture topic as least squares approximation in a linear algebra course.

Problem statement on chalkboard

Clear evidence

Shown in the video

Evidence

Caption evidence
Observation
The left side of the chalkboard contains the handwritten problem statement with the three points.
Audio
Observation
The instructor reads the problem and pauses for the viewer to try it.

Objects

Handwritten prompt
three data points
instructor at board

Changes

The instructor stands beside the written problem and gestures toward it.

Invariants

The problem statement remains visible throughout this interval.

Interpretation

The visual establishes the exact fitting problem: a quadratic through the origin best-fitting three given points.

Writing the quadratic model

Clear evidence

Shown in the video

Evidence

Formula
Observation
The instructor writes $ct + dt^2 = y$ on the right side of the board.
Audio
Observation
He says the equation is going to look like "ct plus dt squared equals y".

Objects

Chalk
equation $ct+dt^2=y$

Changes

A new equation appears on the board after the instructor turns to write.

Invariants

The original problem statement remains on the left.

Interpretation

The visual records the chosen functional form for the least-squares fit.

Starting the design matrix

Clear evidence

Shown in the video

Evidence

Formula
Observation
The instructor writes $A=$ and large parentheses, then fills the first column with $1$ , $2$ , and $-1$ .
Audio
Observation
He says he will put the first coordinates of the three points in the first column.

Uncertainties

The second column is not yet written when the clip ends.

Objects

Matrix label $A$
parentheses
first column entries $1,2,-1$

Changes

The board gains the beginning of a matrix below the model equation.

Invariants

The model equation $ct+dt^2=y$ stays above the matrix.

Interpretation

The visual shows the transition from scalar equation to matrix formulation for least squares.

Board construction of the least-squares setup

Clear evidence

Shown in the video

Evidence

Animation
Observation
The instructor writes the second column of $A$ , then $\widehat{x}$ , then $b$ , and later writes 'Can't solve $A\widehat{x}=b$ ' below the matrices.
Diagram
Observation
The board is split into a left problem statement and a right working area with the model equation and matrices.

Objects

Left problem statement with the three points.
Top model equation $ct+dt^2=y$ .
Matrix $A$ .
Vector $\widehat{x}$ .
Vector $b$ .
Written note 'Can't solve $A\widehat{x}=b$ '.

Changes

At the beginning, only the first column of $A$ is visible.
The second column of $A$ is filled in with $1$ , $4$ , $1$ .
$\widehat{x}$ is written as $(c,d)^T$ .
$b$ is written as $(1,5,-2)^T$ .
A new line appears stating that $A\widehat{x}=b$ cannot be solved.
The instructor points among the point list, the model equation, and the matrix entries while explaining.

Invariants

The three data points remain fixed throughout the clip.
The model remains $ct+dt^2=y$ throughout the clip.
The dimensions stay $A\in\mathbb{R}^{3\times2}$ , $\widehat{x}\in\mathbb{R}^{2}$ , $b\in\mathbb{R}^{3}$ .

Interpretation

The visual progression shows the conversion of a curve-fitting question into a linear algebra system, followed by the recognition that the exact system is inconsistent and must be replaced by a projection-based least-squares formulation.

Shift from unsolvable system to normal equations

Clear evidence

Shown in the video

Evidence

Diagram
Observation
At the start the left half of the board contains ct+ $dt^2=y$ , A, $\hat{x}$ , b, and 'Can't solve $A\hat{x}=b$ '; the instructor then writes 'Really solve' and $A^T A \hat{x} = A^T b$ to the right.

Objects

Original model ct+ $dt^2=y$
Matrix A
Vector $\hat{x}$
Vector b
Text 'Can't solve'
Equation $A\hat{x}=b$
Text 'Really solve'
Equation $A^T A \hat{x} = A^T b$

Changes

The instructor adds the phrase 'Really solve'.
He writes the normal-equation formula to the right of the earlier unsolvable-system note.

Invariants

The original matrices A, $\hat{x}$ , and b remain visible on the left while the new formula is added.

Interpretation

The visual layout contrasts the failed direct solve with the least-squares replacement via normal equations.

Stepwise construction of the numeric normal-equation system

Clear evidence

Shown in the video

Evidence

Diagram
Observation
On the right side of the board the instructor writes $A^T A$ = ..., then $A^T b$ = ..., then the substituted system.
Animation
Observation
The writing proceeds sequentially from the product setup to the numeric results and finally to the combined equation.

Objects

$A^T$
A
b
$A^T A$
$A^T b$
Final system matrix and vector

Changes

First the product $A^T A$ is written out.
Then its numeric result [[6, 8], [8, 10]] is added.
Next the product $A^T b$ is written and evaluated to [[13], [19]].
Finally the two results are assembled into one displayed linear system.

Invariants

The same A, $A^T$ , and b entries are reused throughout the computation.

Interpretation

The board animation turns the symbolic normal equations into a concrete 2 x 2 system ready to solve.

Right-board computation view

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The camera stays on the right half of the chalkboard while the instructor writes the reduced system and the final boxed equation.

Objects

Chalkboard formulas for $A^T A$
Chalkboard formulas for $A^T b$
Normal-equation system
Reduced triangular system
Values $d=-5/2$ and $c=11/2$
Boxed final equation $y=11/2 t - 5/2 t^2$

Changes

A reduction arrow is added beside the original normal-equation system.
The triangular system $\begin{pmatrix}6&8\\0&-2\end{pmatrix}\widehat{x}=\begin{pmatrix}13\\5\end{pmatrix}$ is written to the right.
The coefficient values $d=-5/2$ and $c=11/2$ are added below.
The final quadratic equation is written and enclosed in a box.

Invariants

The previously computed matrices $A^T A$ and $A^T b$ remain visible above.
The example remains $a 2\times 2$ linear solve throughout this interval.

Interpretation

The visual sequence tracks the solution path from normal equations to elimination, back substitution, and the final fitted model.

Recap pan to the setup board

Clear evidence

Shown in the video

Evidence

Animation
Observation
The camera pans left to reveal the earlier setup on the board.
Diagram
Observation
The left board shows ct+ $dt^2=y$ , $A=\begin{pmatrix}1&2\\2&4\\-1&1\end{pmatrix}$ , $\widehat{x}=\begin{pmatrix}c\\d\end{pmatrix}$ , $b=\begin{pmatrix}5\\1\\-2\end{pmatrix}$ , 'Can't solve $A\widehat{x}=b$ .', and 'Really solve $A^T A\widehat{x}=A^T b$ .'

Objects

Scalar model ct+ $dt^2=y$
Matrix A
Vector $\widehat{x}$
Vector b
Text 'Can't solve $A\widehat{x}=b$ .'
Text 'Really solve $A^T A\widehat{x}=A^T b$ .'

Changes

The view shifts from the solved right side to the original formulation on the left.
The instructor gestures toward the general form, then the matrices, then the normal-equation statement.

Invariants

The same example is being discussed; only the board region shown changes.

Interpretation

This visual move supports the spoken summary of the least-squares workflow: start from the scalar model, encode it as A and b, then replace the unsolvable direct system with the normal equations.

Verbal check against data points

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The instructor stands beside the boxed equation and gestures toward it while discussing whether the data points lie on the curve.

Uncertainties

No graph of the parabola or plotted data points is shown in this clip; the check is verbal and gestural only.

Objects

Boxed equation $y=11/2 t - 5/2 t^2$
Instructor’s pointing hand

Changes

Attention shifts from solving to interpreting the result.
The instructor references $t=1$ and contrasts the observed value with the fitted value.

Invariants

The boxed equation remains the focal visual object.

Interpretation

The gesture accompanies the conceptual point that a least-squares fit minimizes overall discrepancy and need not interpolate the given points.

blackboard presentation

Clear evidence

Shown in the video

Evidence

Diagram
Observation
the blackboard is filled with linear algebra calculations, and the final result is enclosed in a box at the bottom

Objects

the instructor
the blackboard
mathematical formulas

Invariants

the content of the formulas on the blackboard remains unchanged

Interpretation

shows the final blackboard state of the least squares solution process

Misconceptions · 6

Forgetting that “through the origin” removes the constant term

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor explicitly contrasts a general quadratic with one through the origin and says the constant term is zero.

Misconception

One might write a full quadratic with a nonzero constant term even though the problem requires passage through the origin.

Clarification

Because the curve must pass through the origin, the constant term is zero, leaving only $ct+dt^2=y$ .

Assuming the matrix equation must be exactly solvable

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says it would be great if we could solve $Ax$ hat equals $b$ , but we can't because there isn't a quadratic equation through the origin that contains these three points.
Formula
Observation
He writes 'Can't solve $A\widehat{x}=b$ ' on the board.

Misconception

One may think that once a fitting problem is rewritten as $A\widehat{x}=b$ , an exact solution should exist.

Clarification

The clip explicitly shows that the constructed system can be inconsistent; here no quadratic through the origin passes through all three points, so one must seek a best approximation instead.

Thinking any right-hand side can be matched by $A\widehat{x}$

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says we only have a chance of solving $A\widehat{x}$ equals something if it's in the column space of $A$ .

Misconception

One may overlook that $A\widehat{x}$ can produce only vectors in the column space of $A$ .

Clarification

The instructor uses this restriction to justify replacing $b$ by its projection onto $\operatorname{Col}(A)$ when solving the least-squares problem.

Misplacing a sign in the transpose matrix

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says, 'I had the negative in the wrong place. Thank you. Thanks for that correction. I had the negative backwards there.'
Diagram
Observation
He erases part of the written $A^T A$ product and rewrites the corrected transpose matrix before continuing.

Misconception

When forming $A^T A$ by hand, a negative sign can be attached to the wrong entry of the transposed matrix.

Clarification

The video explicitly corrects this error on the board before proceeding with the multiplication.

Mistaking a least-squares fit for an interpolating curve

Clear evidence

Shown in the video

Evidence

Audio
Observation
At 01:44.480-01:52.240 the instructor says, 'Good, it might also be worth noting that these three points certainly aren't on this quadratic equation.'
Audio
Observation
At 01:52.240-01:56.640 he says, 'For instance, if I plug in one here, I don't get one one, I get one three.'
Audio
Observation
At 01:57.600-01:59.840 he adds, 'But that's okay.'

Misconception

One may expect the best-fit quadratic to pass through every data point.

Clarification

The video explicitly states that the three points are not on the fitted quadratic; at $t=1$ the fit gives 3 instead of the observed 1, and that is acceptable for a least-squares approximation.

Trying to solve the overdetermined system directly

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The left board contains the text 'Can't solve $A\widehat{x}=b$ .' next to 'Really solve $A^T A\widehat{x}=A^T b$ .'

Misconception

One may try to solve $A\widehat{x}=b$ exactly for an overdetermined least-squares problem.

Clarification

The board contrast indicates that the direct system is not solvable here, so the correct route is the normal equations $A^T A\widehat{x}=A^T b$ .

Concept relations · 16

Topic: Least Squares Approximation → Best-fit quadratic through the origin for three points

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor introduces least squares approximations and immediately presents a specific fitting problem.

Application

Explanation

The general topic of least squares approximation is applied to a concrete quadratic-fitting example.

Through the origin implies zero constant term → Quadratic model through the origin

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor uses the origin condition to justify omitting the constant term in the written model.

Proof dependency

Explanation

The claim about the zero constant term directly justifies the specific form $ct+dt^2=y$ .

Quadratic model through the origin → Beginning the matrix equation setup

Clear evidence

Shown in the video

Evidence

Audio
Observation
After establishing the model, the instructor says the next step is to set up a matrix equation and begins writing $A$ .

Uncertainties

The full matrix equation is not completed in this clip.

Application

Explanation

The chosen quadratic model determines how the design matrix $A$ is constructed.

Least-squares fitting problem for a quadratic through the origin → Why each row of $A\widehat{x}=b$ encodes one data point

Clear evidence

Shown in the video

Evidence

Audio
Observation
After writing the matrices, the instructor explains their meaning by multiplying rows and comparing with substitution into $ct+dt^2=y$ .

Application

Explanation

The general fitting problem is applied concretely by translating each data point into a row of the matrix equation.

Why each row of $A\widehat{x}=b$ encodes one data point → The exact system $A\widehat{x}=b$ is inconsistent here

Clear evidence

Shown in the video

Evidence

Audio
Observation
He moves from explaining the rows to stating that the exact system cannot be solved for these points.

Proof dependency

Explanation

The row-by-row interpretation shows that solving $A\widehat{x}=b$ would mean satisfying all three point equations exactly, which the clip then declares impossible.

The exact system $A\widehat{x}=b$ is inconsistent here → Least-squares solution as projection onto the column space

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says that because we cannot solve the exact system, the best approximation is obtained by projecting $b$ onto the column space of $A$ .

Generalizes

Explanation

The projection formulation generalizes the failed exact solve into the standard least-squares replacement when $b\notin\operatorname{Col}(A)$ .

Least-squares fitting problem for a quadratic through the origin → Setting up a least-squares quadratic fit through the origin

Clear evidence

Shown in the video

Evidence

Formula
Observation
The worked board content is precisely the stated example problem with the three given points.

Contains

Explanation

The example is a concrete instance of the least-squares fitting definition introduced at the start of the clip.

Direct system $A\hat{x}=b$ is not solvable → Normal equations for least squares

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board juxtaposes 'Can't solve $A\hat{x}=b$ ' with 'Really solve $A^T A \hat{x} = A^T b$ '.
Audio
Observation
The instructor says the least-squares task is the same as solving the normal equations.

Application

Explanation

Because the displayed direct system is marked unsolvable, the lecture applies the normal-equations method in its place.

Computation of $A^T A$ → Substituted normal-equation system

Clear evidence

Shown in the video

Evidence

Formula
Observation
The computed matrices $A^T A$ and $A^T b$ are substituted into the final displayed system.

Proof dependency

Explanation

The final concrete system depends on the previously computed left-hand matrix $A^T A$ .

Computation of $A^T b$ → Substituted normal-equation system

Clear evidence

Shown in the video

Evidence

Formula
Observation
The computed vector $A^T b$ appears as the right-hand side of the final system.

Proof dependency

Explanation

The final concrete system also depends on the computed right-hand side $A^T b$ .

Normal equations for least squares → Elimination step for the $2\times 2$ normal-equation system

Clear evidence

Shown in the video

Evidence

Formula
Observation
The reduced triangular system is written immediately to the right of the normal-equation system.

Application

Explanation

The elimination method is applied directly to the $2\times 2$ normal-equation system shown on the board.

Elimination step for the $2\times 2$ normal-equation system → Back substitution from the triangular system

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says they 'back solve' after changing the b vector.
Formula
Observation
The triangular system leads to $d=-5/2$ and then $c=11/2$ .

Proof dependency

Explanation

Back substitution depends on the prior elimination step producing an upper-triangular system.

Find an answer · 20

What topic is this clip introducing?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor states that the problem is about least squares approximations.

Knowledge points

Topic: Least Squares Approximation

Why does the quadratic equation have no constant term here?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor explains that through the origin means the constant term is zero.

Knowledge points

Through the origin implies zero constant term
Quadratic model through the origin

How does the instructor begin setting up the matrix equation for this least-squares problem?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says the next step is to set up a matrix equation and begins filling the first column of $A$ with the first coordinates.

Uncertainties

Only the first column is shown.

Knowledge points

Beginning the matrix equation setup

What data points are used in the worked example?

Clear evidence

Shown in the video

Evidence

Caption evidence
Observation
The board lists the points $(1,1)$ , $(2,5)$ , and $(-1,-2)$ .

Knowledge points

Best-fit quadratic through the origin for three points

Why does the second column of $A$ contain $1$ , $4$ , and $1$ in this quadratic fit?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor explains that the second column is the squares of the first coordinates.

Knowledge points

Least-squares fitting problem for a quadratic through the origin
Why each row of $A\widehat{x}=b$ encodes one data point

Where does the vector $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ come from?

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says $b$ came from plugging the points into the right-hand side, so the entries are the $y$ -coordinates.

Knowledge points

Least-squares fitting problem for a quadratic through the origin
Why each row of $A\widehat{x}=b$ encodes one data point

Why can't we just solve $A\widehat{x}=b$ exactly here?

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board explicitly says 'Can't solve $A\widehat{x}=b$ '.
Audio
Observation
He explains that no quadratic through the origin contains all three points.

Knowledge points

The exact system $A\widehat{x}=b$ is inconsistent here
No exact quadratic through the origin fits all three points

What does it mean to replace $b$ by its projection onto the column space of $A$ ?

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says the best approximation is the same as solving $A\widehat{x}$ equal to the projection of $b$ onto the column space of $A$ .

Knowledge points

Least-squares solution as projection onto the column space
Best approximation is obtained by projecting $b$ onto $\operatorname{Col}(A)$

Why must the right-hand side lie in the column space of $A$ for $A\widehat{x}$ to be solvable?

Clear evidence

Shown in the video

Evidence

Audio
Observation
He says we only have a chance of solving $A\widehat{x}$ equals something if it's in the column space of $A$ .

Knowledge points

Least-squares solution as projection onto the column space
Thinking any right-hand side can be matched by $A\widehat{x}$

Why does the lecture replace $A\hat{x}=b$ with $A^T A \hat{x} = A^T b$ ?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The instructor says this is the same as solving A transpose A x hat equals A transpose b.
Formula
Observation
The board writes 'Really solve $A^T A \hat{x} = A^T b$ '.

Knowledge points

Direct system $A\hat{x}=b$ is not solvable
Normal equations for least squares

How is $A^T A$ computed for the displayed matrix A?

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows the full multiplication for $A^T A$ and the result [[6, 8], [8, 10]].

Knowledge points

Computation of $A^T A$

What is $A^T b$ in this least-squares example?

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $A^T b = [[1, 2, -1], [1, 4, 1]] [[1], [5], [-2]] = [[13], [19]]$ .

Knowledge points

Computation of $A^T b$

Coverage and review notes

Covered · Title slide identifying the course and topic.

Covered · Instructor reads the least-squares quadratic fitting problem and pauses for the viewer.

Covered · Instructor chooses the model $ct+dt^2=y$ and explains why the constant term is zero.

Covered · Instructor begins the matrix setup by writing $A$ and filling its first column with the $t$ -coordinates.

Covered · The board problem statement and the construction of $A$ , $\widehat{x}$ , and $b$ are shown and explained.

Covered · The instructor verifies the setup by matching rows of $A\widehat{x}=b$ to direct substitution of the three points into $ct+dt^2=y$ .

Covered · The exact system is declared unsolvable because no quadratic through the origin fits all three points.

Covered · The least-squares problem is reformulated as projecting $b$ onto the column space of $A$ ; the clip ends before further computation.

Covered · The clip opens with the model and matrices on the board, states that $A\hat{x}=b$ cannot be solved directly, and writes the normal equations as the actual problem.

Covered · Brief transition in which the instructor finishes writing the normal-equation statement and prepares to compute.

Covered · The instructor computes $A^T A$ , notices and corrects a misplaced negative sign, and writes the resulting matrix [[6, 8], [8, 10]].

Covered · Short pause between finishing $A^T A$ and beginning $A^T b$ .

Covered · The instructor computes $A^T b$ and writes the resulting vector [[13], [19]].

Covered · The computed pieces are substituted back to produce the final displayed system [[6, 8], [8, 10]] $\hat{x} = [[13], [19]]$ .

Covered · Board shows $A^T A$ , $A^T b$ , the normal equations, and the elimination step to a triangular system.

Covered · The instructor back substitutes to obtain $d=-5/2$ and $c=11/2$ .

Covered · The coefficients are inserted into the model and the final boxed equation is written.

Covered · The camera pans left and the instructor reviews the general workflow from scalar form to matrices to normal equations.

Covered · The instructor notes that the fitted quadratic does not pass through the data points and gives the $t=1$ check.

Covered · The video shows the instructor finishing with closing remarks while the blackboard displays the complete least squares calculation process and final result.

Explore the knowledge in this video

Reviewed subject paths

Least squares fitting

Least Squares Approximation

The explanation, unpacked.

Chapters

Learning script

Knowledge cards

Least squares approximation problem setup

Quadratic through the origin has no constant term

First step of the matrix formulation

Quadratic-through-origin fitting model

Design matrix for the three data points

Unknown vector and observation vector

Row-by-row meaning of Ax^=bA\widehat{x}=bAx=b

Why the exact system is unsolvable

Least squares as projection onto the column space

Normal equations replace an unsolvable system

Concrete matrices in the example

Computing ATAA^T AATA

Computing ATbA^T bATb

Final substituted least-squares system

Sign-placement pitfall when writing ATA^TAT

Normal equations shown on the board

Elimination step used to triangularize the system

Back substitution gives the coefficients

Final fitted quadratic through the origin

Workflow recap for least squares

Least-squares fit need not pass through the data

least squares fitted equation

Detailed learning notes

c

d

t

y

A

ct+dt2=ydt^2=ydt2=y

A

x^\widehat{x}x

b

(1,1), (2,5), (-1,-2)

column space of A

A

Topic: Least Squares Approximation

Quadratic model through the origin

Least-squares fitting problem for a quadratic through the origin

Why each row of Ax^=bA\widehat{x}=bAx=b encodes one data point

The exact system Ax^=bA\widehat{x}=bAx=b is inconsistent here

Least-squares solution as projection onto the column space

Normal equations for least squares

Direct system Ax^=bA\hat{x}=bAx^=b is not solvable

Computation of ATAA^T AATA

Computation of ATbA^T bATb

Substituted normal-equation system

Normal equations for least squares

Through the origin implies zero constant term

No exact quadratic through the origin fits all three points

Best approximation is obtained by projecting bbb onto Col⁡(A)\operatorname{Col}(A)Col(A)

Least-squares problem is solved via normal equations

Explicit row operation used in elimination

Solved coefficient values

Best-fit curve need not pass through the data points

Setting up the quadratic model from the problem statement

Beginning the matrix equation setup

Derivation of the matrix equation from pointwise substitution

From inconsistent exact fit to projected least-squares system

From normal equations to a concrete 2 x 2 system

Derivation of the coefficients from the normal equations

Assembly of the fitted quadratic from the solved coefficients

derivation from the matrix equation to the fitted equation

Best-fit quadratic through the origin for three points

Setting up a least-squares quadratic fit through the origin

Least-squares setup for ct + dt2=ydt^2 = ydt2=y

Worked least-squares fit of a quadratic through the origin

Course title slide

Problem statement on chalkboard

Writing the quadratic model

Starting the design matrix

Board construction of the least-squares setup

Shift from unsolvable system to normal equations

Stepwise construction of the numeric normal-equation system

Right-board computation view

Recap pan to the setup board

Row-by-row meaning of $A\widehat{x}=b$

Computing $A^T A$

Computing $A^T b$

Sign-placement pitfall when writing $A^T$

ct+ $dt^2=y$

$\widehat{x}$

Why each row of $A\widehat{x}=b$ encodes one data point

The exact system $A\widehat{x}=b$ is inconsistent here

Direct system $A\hat{x}=b$ is not solvable

Computation of $A^T A$

Computation of $A^T b$

Best approximation is obtained by projecting $b$ onto $\operatorname{Col}(A)$

Least-squares setup for ct + $dt^2 = y$

Thinking any right-hand side can be matched by $A\widehat{x}$

Least-squares fitting problem for a quadratic through the origin → Why each row of $A\widehat{x}=b$ encodes one data point

Why each row of $A\widehat{x}=b$ encodes one data point → The exact system $A\widehat{x}=b$ is inconsistent here

The exact system $A\widehat{x}=b$ is inconsistent here → Least-squares solution as projection onto the column space

Direct system $A\hat{x}=b$ is not solvable → Normal equations for least squares

Computation of $A^T A$ → Substituted normal-equation system

Computation of $A^T b$ → Substituted normal-equation system

Normal equations for least squares → Elimination step for the $2\times 2$ normal-equation system

Elimination step for the $2\times 2$ normal-equation system → Back substitution from the triangular system

Why does the second column of $A$ contain $1$ , $4$ , and $1$ in this quadratic fit?

Where does the vector $b=\begin{pmatrix}1\\5\\-2\end{pmatrix}$ come from?

Why can't we just solve $A\widehat{x}=b$ exactly here?

What does it mean to replace $b$ by its projection onto the column space of $A$ ?

Why must the right-hand side lie in the column space of $A$ for $A\widehat{x}$ to be solvable?

Why does the lecture replace $A\hat{x}=b$ with $A^T A \hat{x} = A^T b$ ?

How is $A^T A$ computed for the displayed matrix A?

What is $A^T b$ in this least-squares example?