Introduction To Ordinary Least Squares With Examples

1. Diagram
   Observation

   The vertical axis is labeled "y [tonnes]" with the subtitle "(Potato yield)".

y

Observed response variable in the example, measured as potato yield in tonnes.

tonnes

1. Diagram
   Observation

   The horizontal axis is labeled "$X_1$ [kg]" with the subtitle "(Nitrogen Fertilizer)".

$X_1$

Explanatory variable in the example, measured as nitrogen fertilizer amount in kilograms.

kg

1. Diagram
   Observation

   A label "$y_i$" appears on the y-axis aligned with a blue data point.

2. Audio
   Observation

   The speaker says, "Now for each pair of points we have two values, y i and y i hat."

$y_i$

Actual observed value for the i-th data point; in the example, the actual potato yield obtained when a specific amount of nitrogen fertilizer was used.

tonnes

1. Diagram
   Observation

   A label "$\hat{y}_i$" appears on the y-axis aligned with the middle regression line at the same x-coordinate as the selected point.

2. Audio
   Observation

   The speaker says, "Y i hat on the other hand is what this linear regression that we're considering what it predicts the yield to be or would have been."

$\hat{y}_i$

Predicted or fitted value for the i-th data point from the chosen linear regression line.

tonnes

1. Diagram
   Observation

   The right-hand plot labels the vertical axis as y [tonnes] (Potato yield).

y

Observed response variable in the example, measured in tonnes of potato yield.

Real-valued response values.

1. Diagram
   Observation

   The right-hand plot labels the horizontal axis as $X_1$ [kg] (Nitrogen Fertilizer).

$X_1$

Predictor variable in the example, measured in kilograms of nitrogen fertilizer.

Real-valued predictor values.

1. Diagram
   Observation

   Blue data points on the scatter plot are associated with the label $y_i$.

2. Audio
   Observation

   The speaker says, "there's our $y_i$ and $y_i$ hat".

$y_i$

Actual observed value for the i-th data point.

Real numbers.

1. Diagram
   Observation

   Grey points on the fitted line are associated with the label $\hat{y}_i$.

2. Audio
   Observation

   The speaker refers to "the predicted yield" and then names it "$y_i$ hat".

$\hat{y}_i$

Predicted value for the i-th data point from the regression line.

Real numbers.

1. Formula
   Observation

   On-screen text defines residual: $\varepsilon_i = y_i - \hat{y}_i$.

2. Audio
   Observation

   The speaker says, "the difference between them is called the residual".

$\varepsilon_i$

Residual for the i-th observation, equal to the vertical difference between observed and predicted values.

Real numbers.

1. Formula
   Observation

   The equation $\hat{y} = b_0 + b_1 X_1$ appears on screen.

2. Audio
   Observation

   The speaker says the best equation has parameters "$b_0$ and $b_1$".

$b_0$

Intercept parameter of the fitted simple linear regression line.

Real numbers.

1. Formula
   Observation

   The equation $\hat{y} = b_0 + b_1 X_1$ appears on screen.

2. Audio
   Observation

   The speaker says the best equation has parameters "$b_0$ and $b_1$".

$b_1$

Slope parameter of the fitted simple linear regression line.

Real numbers.

1. Formula
   Observation

   On-screen text states: SUM($y_i - \hat{y}_i$)^2 is minimized.

2. Audio
   Observation

   The speaker says, "the sum of the squares of the residuals is minimized".

1. The displayed formula uses SUM(...) without explicit summation limits or index notation.

SUM($y_i - \hat{y}_i$)^2

Aggregate squared-residual objective used to choose the best regression line.

Nonnegative real-valued objective over all data points.

1. Audio
   Observation

   The speaker says, "So in order to answer those questions we need to look at a method called the ordinary least squares method."

2. Diagram
   Observation

   Text "Ordinary Least Squares:" appears on the slide.

The video introduces ordinary least squares as the method used to decide which candidate regression line should count as the best one for a given scatter of data points.

1. The discussion is about choosing among multiple possible straight-line fits to a set of data points.

1. Audio
   Observation

   The speaker says, "And the way it works in a visual sense is we need to take our data points and project them vertically onto our linear regression line."

2. Animation
   Observation

   Dashed vertical segments are drawn from each blue data point down or up to the middle solid regression line.

In this introductory segment, ordinary least squares is presented visually by projecting each observed point vertically onto a candidate regression line. The clip explicitly demonstrates this projection for one middle line only, while stating that the same idea would apply to every candidate line being considered.

1. The explanation is visual rather than algebraic in this clip.

2. Only one candidate line is actually shown with projections, although the speaker says the procedure applies to all candidate lines.

1. Ordinary Least Squares as the method under discussion

1. Audio
   Observation

   The speaker says, "Now for each pair of points we have two values, y i and y i hat."

2. Diagram
   Observation

   Labels "$y_i$" and "$\hat{y}_i$" appear on the y-axis, aligned respectively with a data point and the middle regression line.

For each observation, the video distinguishes the actual response value $y_i$ from the model's predicted value $\hat{y}_i$. Here $y_i$ is the observed potato yield for a given fertilizer amount, while $\hat{y}_i$ is the yield predicted by the chosen regression line at that same fertilizer amount.

1. Applies to each paired observation in the regression example.

2. The comparison is made at the same explanatory-variable value $X_1$.

1. $y_i$
2. $\hat{y}_i$

1. Diagram
   Observation

   Slide title reads "Simple Linear Regression" and subtitle reads "Ordinary Least Squares:".

The clip introduces ordinary least squares as the method used to find the best line in a simple linear regression setting. The visual example is a scatter plot with a fitted line and several alternative candidate lines.

1. One predictor variable.

2. Linear model form for the fitted line.

1. Formula
   Observation

   residual: $\varepsilon_i = y_i - \hat{y}_i$

2. Audio
   Observation

   The speaker says, "the difference between them is called the residual".

3. Diagram
   Observation

   A blue point labeled $y_i$ and a grey point labeled $\hat{y}_i$ are connected by a dashed vertical segment.

For each observation, the residual is the vertical difference between the actual observed value and the value predicted by the regression line. In the example, this is the gap between the blue data point and the corresponding grey point on the fitted line.

1. Defined for each data point i.

2. Uses an observed value $y_i$ and a predicted value $\hat{y}_i$ from a chosen line.

1. Topic framing: ordinary least squares within simple linear regression

1. Formula
   Observation

   $\hat{y} = b_0 + b_1 X_1$

2. Audio
   Observation

   The speaker says, "here's our equation" and later refers to parameters $b_0$ and $b_1$.

The fitted prediction is written as a linear function of one predictor. Here $b_0$ is the intercept and $b_1$ is the slope multiplying $X_1$.

1. Single predictor $X_1$.

2. Parameters $b_0$ and $b_1$ define one candidate regression line.

1. Topic framing: ordinary least squares within simple linear regression

1. Formula
   Observation

   SUM($y_i - \hat{y}_i$)^2 is minimized

2. Audio
   Observation

   The speaker says, "the sum of the squares of the residuals is minimized" and "that's why it's called the ordinary least squares method".

1. The slide writes SUM(...) rather than a fully indexed summation notation.

Ordinary least squares chooses the regression line whose squared residuals, summed across all data points, are as small as possible. The speaker explicitly links this minimization rule to the name of the method.

1. Applied after defining residuals for each point.

2. Used to compare candidate regression lines.

3. Minimization is over the parameters that determine the line.

1. Residual definition
2. Simple linear regression equation

1. Audio
   Observation

   The speaker says the line is "never going to go perfectly through every single data point" and calls that "simply impossible".

In the presented regression setting, the fitted line is not expected to pass exactly through every observed data point; differences between actual and predicted values are normal.

1. There are multiple observed data points.

2. A single straight regression line is being used to summarize them.

Universal claim about the data points in the shown setting.

1. Audio
   Observation

   The speaker says, "we want to find the best line" and ties it to how small the differences are.

2. Formula
   Observation

   SUM($y_i - \hat{y}_i$)^2 is minimized

3. Audio
   Observation

   The speaker says, "for whichever regression line this value is going to be the smallest, that'll be the best regression line".

Among candidate regression lines, the one that minimizes the sum of squared residuals is identified as the best regression line for modeling the problem.

1. Residuals are defined as $y_i - \hat{y}_i$.

2. Candidate lines are compared using the same dataset.

Over the candidate regression lines considered in the clip.

1. Audio
   Observation

   The speaker asks how to know which sloped line is the best one and notes that multiple sloped lines can be drawn through the data points.

2. Animation
   Observation

   Several candidate straight lines are overlaid on the scatter plot, including a darker middle line and lighter alternatives above and below.

1. Expression

   $$$$
   Explanation

   The clip begins with a scatter plot of observed data points relating nitrogen fertilizer to potato yield.

   Justification

   Directly shown on the graph and stated in the narration.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Multiple different sloped lines are drawn through the same cloud of points.

   Justification

   Visible animation adds several candidate lines over the scatter plot.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Because more than one line can pass through the data, the video raises the question of which line is the best linear regression and even how to define "best."

   Justification

   Explicitly stated by the speaker.

   Shown in the video

The existence of many candidate regression lines motivates the need for a formal selection method, introduced next as ordinary least squares.

1. Audio
   Observation

   The speaker introduces ordinary least squares and describes projecting data points vertically onto the regression line.

2. Animation
   Observation

   Vertical dashed segments connect each blue point to the middle solid line.

1. The clip does not yet show the squared-error formula or the minimization step.

2. The speaker says the procedure would be done for every candidate line, but only one line is demonstrated visually.

1. Expression

   $$$$
   Explanation

   Introduce ordinary least squares as the method for answering which candidate line is best.

   Justification

   Stated directly in the audio and shown as slide text.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Select one candidate regression line for illustration, namely the middle line.

   Justification

   The speaker says that for simplicity the tutorial will use the middle line.

   Shown in the video
3. Expression

   $$$$
   Explanation

   For each data point, draw a vertical segment from the point to the chosen line.

   Justification

   Explicitly described as projecting the data points vertically onto the regression line and shown by dashed segments.

   Shown in the video
4. Expression

   $$$$
   Explanation

   Repeat the same vertical projection idea for any other candidate line if one wanted to compare them.

   Justification

   Stated verbally, though not fully animated in this clip.

   Shown in the video

Within this excerpt, ordinary least squares is introduced as a method based on comparing observed points to their vertical projections on a regression line.

1. Audio
   Observation

   The speaker defines $y_i$ and $\hat{y}_i$ using a numerical example: 15 kg of nitrogen fertilizer yields 2 tonnes actually, while the regression predicts 1.5 tonnes.

2. Diagram
   Observation

   Horizontal guide lines mark the levels of $y_i$ and $\hat{y}_i$ on the y-axis.

1. The exact x-coordinate of the highlighted point is not numerically printed on the axis; the value 15 kg comes from the spoken example.

2. The clip stops before explicitly naming the difference $y_i-\hat{y}_i$ as a residual.

1. Expression

   $$$$
   Explanation

   Pick one observation in the scatter plot and identify its actual response value as $y_i$.

   Justification

   The speaker says $y_i$ is the actual amount of potatoes yielded when a specific amount of nitrogen fertilizer was used.

   Shown in the video
2. Expression

   $$15\ \text{kg} \rightarrow 2\ \text{tonnes}$$
   Explanation

   In the spoken example, when 15 kg of nitrogen fertilizer were used, the farm actually yielded 2 tonnes of potatoes.

   Justification

   Directly stated in the audio.

   Shown in the video
3. Expression

   $$$$
   Explanation

   At the same fertilizer amount, read off the value on the chosen regression line and call it $\hat{y}_i$.

   Justification

   The speaker defines $\hat{y}_i$ as what the linear regression predicts the yield to be.

   Shown in the video
4. Expression

   $$15\ \text{kg} \rightarrow 1.5\ \text{tonnes}$$
   Explanation

   For the same 15 kg input, the regression line predicts 1.5 tonnes of potatoes.

   Justification

   Directly stated in the audio.

   Shown in the video
5. Expression

   $$$$
   Explanation

   Therefore the actual and predicted values differ for this observation.

   Justification

   The speaker concludes, "As you can see there's a slight difference between the actual value" before the clip ends.

   Shown in the video

The example shows concretely that $y_i$ is the observed outcome and $\hat{y}_i$ is the model prediction at the same explanatory-variable value, so they need not coincide.

1. Audio
   Observation

   Continuous spoken explanation from imperfect fit to residual definition to minimization criterion.

2. Diagram
   Observation

   Scatter plot with multiple parallel candidate lines and vertical residual segments.

3. Formula
   Observation

   residual: $\varepsilon_i = y_i - \hat{y}_i$ and SUM($y_i - \hat{y}_i$)^2 is minimized.

1. This is an intuitive derivation of the method, not a formal proof of optimality.

1. Expression

   $$$$
   Explanation

   Start from the observation that a single line will not pass exactly through every data point, so actual and predicted values differ.

   Justification

   Stated directly by the speaker at the beginning of the clip.

   Shown in the video
2. Expression

   $$\varepsilon_i = y_i - \hat{y}_i$$
   Explanation

   Name the difference for each point as the residual.

   Justification

   Defined on screen and in speech after introducing $y_i$ and $\hat{y}_i$.

   Shown in the video
3. Expression

   $$(y_i - \hat{y}_i)^2$$
   Explanation

   Square each residual so that the comparison uses squared differences rather than raw signed gaps.

   Justification

   The speaker says, "we need to square them for every single data point".

   Shown in the video
4. Expression

   $$SUM(y_i - \hat{y}_i)^2$$
   Explanation

   Add the squared residuals across all data points to obtain one aggregate measure for a candidate line.

   Justification

   The speaker says, "then we need to add up the sum".

   Shown in the video
5. Expression

   $$$$
   Explanation

   Choose the line for which this aggregate value is smallest; that line is called the best regression line.

   Justification

   The speaker states that whichever regression line makes this value smallest is the best regression line.

   Shown in the video

Ordinary least squares selects the regression line that minimizes the sum of squared residuals across the data points.

1. Audio
   Observation

   The speaker uses the example: 15 kilograms of nitrogen fertilizer produced 2 tonnes of potatoes, while the regression predicts 1.5 tonnes.

2. Diagram
   Observation

   The plot axes are labeled potato yield in tonnes and nitrogen fertilizer in kg, and labels $y_i$ and $\hat{y}_i$ are shown for one point.

1. The numerical values 15 kg, 2 tonnes, and 1.5 tonnes are spoken rather than printed on screen.

2. The clip does not compute an explicit residual value or square it.

Using the displayed simple linear regression setup, explain what the actual value $y_i$ and the predicted value $\hat{y}_i$ mean for one observation.

1. The response variable is potato yield in tonnes.

2. The explanatory variable is nitrogen fertilizer in kg.

3. One spoken example uses 15 kg of nitrogen fertilizer.

4. The actual observed yield in that example is 2 tonnes.

5. The chosen regression line predicts 1.5 tonnes for the same 15 kg input.

Identify and contrast $y_i$ and $\hat{y}_i$ in the regression context.

1. Expression

   $$y_i = 2\ \text{tonnes}$$
   Explanation

   The actual observed potato yield for the chosen fertilizer amount is identified as $y_i$.

   Justification

   Stated directly by the speaker as the actual amount yielded from the farm.

   Shown in the video
2. Expression

   $$\hat{y}_i = 1.5\ \text{tonnes}$$
   Explanation

   The value predicted by the regression line at the same fertilizer amount is identified as $\hat{y}_i$.

   Justification

   Stated directly by the speaker as what the linear regression predicts the yield would have been.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Compare the two values at the same $X_1$ to see that the model prediction differs from the observed outcome.

   Justification

   The speaker explicitly notes a slight difference between the actual value and the regression prediction.

   Shown in the video

For the example observation, $y_i=2$ tonnes is the actual yield and $\hat{y}_i=1.5$ tonnes is the regression-predicted yield at the same fertilizer level.

The conclusion matches both the spoken numerical example and the on-screen placement of $y_i$ above $\hat{y}_i$ on the y-axis for the highlighted point.

1. Diagram
   Observation

   Right-hand plot uses axes y [tonnes] (Potato yield) and $X_1$ [kg] (Nitrogen Fertilizer).

2. Audio
   Observation

   The speaker refers to "actual yield and the predicted yield".

1. No numerical coordinates or fitted parameter values are provided in the clip.

Use a scatter plot of potato yield against nitrogen fertilizer to explain how ordinary least squares chooses a regression line.

1. Response variable: y [tonnes] (Potato yield).

2. Predictor variable: $X_1$ [kg] (Nitrogen Fertilizer).

3. Multiple observed data points are plotted.

4. Several candidate straight lines are drawn through the cloud of points.

Identify the best-fitting line by comparing residuals and their squared sum.

1. Expression

   $$$$
   Explanation

   Read each blue point as an actual observed yield $y_i$ and the corresponding grey point on a candidate line as the predicted yield $\hat{y}_i$.

   Justification

   Directly shown by the labeled plot and explained in speech.

   Shown in the video
2. Expression

   $$\varepsilon_i = y_i - \hat{y}_i$$
   Explanation

   Measure the vertical gap between each observed point and the line as the residual.

   Justification

   Formula appears on screen and is named in the audio.

   Shown in the video
3. Expression

   $$SUM(y_i - \hat{y}_i)^2$$
   Explanation

   Aggregate the squared vertical gaps over all points to score a candidate line.

   Justification

   The slide states that this quantity is minimized and the speaker describes squaring and adding the residuals.

   Shown in the video
4. Expression

   $$$$
   Explanation

   Select the line with the smallest aggregate value as the best regression line for the example.

   Justification

   Explicitly stated by the speaker near the end of the explanation.

   Shown in the video

The best line in the example is the one that minimizes the sum of squared residuals between observed potato yields and predicted yields from the line.

No numerical verification is shown; the clip verifies the idea visually through multiple candidate lines and verbally through the least-squares rule.

1. Animation
   Observation

   Opening title cards show course branding and then the topic title "Ordinary Least Squares".

2. Caption evidence
   Observation

   Readable text includes "This tutorial comes from the MACHINE LEARNING IN PYTHON COURSE", "Enjoy MACHINE LEARNING!", and "Ordinary Least Squares".

1. Blue geometric background

2. Course title text

3. Topic title "Ordinary Least Squares"

1. Course branding appears first.

2. The topic title "Ordinary Least Squares" replaces the earlier course text.

1. No mathematical diagram is shown yet.

The clip announces that the subject is ordinary least squares before moving into the regression example.

1. Diagram
   Observation

   A scatter plot titled "Simple Linear Regression" appears with axes labeled "$y$ [tonnes] (Potato yield)" and "$X_1$ [kg] (Nitrogen Fertilizer)".

2. Animation
   Observation

   Several sloped lines are added over the same set of blue data points.

1. The exact number of candidate lines is not verbally fixed; visually there are multiple lines, with one darker middle line and lighter surrounding lines.

1. Blue data points

2. Vertical axis for potato yield

3. Horizontal axis for nitrogen fertilizer

4. Multiple sloped straight lines

1. First only the scatter plot is visible.

2. Then one or more candidate lines are overlaid, ending with several alternative fits around a darker middle line.

1. The data points themselves do not move.

2. The axis labels remain unchanged.

The animation illustrates that many different straight lines can be drawn through the same data, motivating a criterion for choosing the best fit.

1. Animation
   Observation

   Dashed vertical segments are drawn from each blue data point to the middle solid regression line.

2. Audio
   Observation

   The speaker says the data points are projected vertically onto the linear regression line.

1. The clip does not show the same construction for every candidate line, only for the middle one.

1. Blue data points

2. Middle solid regression line

3. Dashed vertical connector segments

1. Dashed segments appear one by one or collectively from each point to the chosen line.

1. The regression line used for projection stays the middle line throughout this demonstration.

2. The horizontal coordinate of each point is unchanged during projection.

This visualizes the core ordinary-least-squares idea used in the clip: compare each observation with its fitted value by moving vertically to the regression line.

1. Diagram
   Observation

   Labels "$y_i$" and "$\hat{y}_i$" appear on the y-axis, with horizontal guide lines indicating their levels.

2. Audio
   Observation

   The speaker explains that each pair of points has two values, $y_i$ and $\hat{y}_i$.

1. The exact highlighted data point is not named by index on screen.

1. Selected blue data point

2. Middle regression line

3. Label "$y_i$"

4. Label "$\hat{y}_i$"

5. Horizontal guide lines

1. The y-axis gains two new labels aligned with the selected observation and the regression line.

1. The selected point remains at the same $(X_1,y)$ location.

2. The regression line remains the same middle line.

The graphic turns the verbal distinction between observed and predicted values into a visible vertical separation on the plot.

1. Diagram
   Observation

   A static scatter plot shows blue data points, a darker central fitted line, and several lighter parallel lines above and below it.

2. Diagram
   Observation

   Vertical dashed segments connect blue points to grey points on the central line.

1. The exact number of lighter candidate lines is approximately five visible bands/lines around the main line.

1. Blue observed data points.

2. Grey predicted points on the central line.

3. Dark central regression line.

4. Lighter parallel candidate lines.

5. Dashed vertical residual segments.

6. Axes labeled y [tonnes] (Potato yield) and $X_1$ [kg] (Nitrogen Fertilizer).

1. Left-side formulas appear progressively while the right-hand plot remains essentially unchanged.

2. The visual emphasis shifts from the overall scatter plot to the residual gap for one point and then to the minimization statement.

1. The data points stay fixed.

2. The axes and example context remain the same throughout the teaching slide.

The picture contrasts many possible straight-line fits to the same data and uses vertical gaps to represent residuals whose squared sum is to be minimized.

1. Diagram
   Observation

   The teaching slide is replaced by a dark blue outro card with subscription and course text.

1. Dark blue background.

2. Text: Click here to subscribe.

3. Text: Thanks for watching!

4. Text: Access the full course using the link in the description.

5. SuperDataScience logo.

1. Mathematical content disappears and promotional end-card text appears.

1. No further mathematical explanation is introduced after the transition.

This interval contains no additional mathematical content beyond the end of the lesson.

1. Audio
   Observation

   The speaker asks, "how do we know which of the sloped lines is the best one?" and notes that multiple sloped lines can be drawn through the data points.

2. Animation
   Observation

   Several different lines are shown over the same scatter plot.

One might think a set of data points automatically has a single natural regression line.

The video explicitly shows multiple candidate sloped lines through the same points and uses that ambiguity to motivate ordinary least squares as a selection method.

1. Audio
   Observation

   The speaker introduces ordinary least squares as the method needed to answer which line is best.

2. Animation
   Observation

   Vertical projections are drawn from the data points to a chosen line.

1. The clip does not yet state the full minimization rule in formula form.

One might think ordinary least squares means simply choosing a visually plausible line.

In this excerpt, ordinary least squares is presented as a systematic comparison based on projecting observations vertically onto the regression line, not as an arbitrary eyeball choice.

1. Audio
   Observation

   The speaker separately defines $y_i$ as the actual amount and $\hat{y}_i$ as what the regression predicts.

2. Diagram
   Observation

   Two distinct y-axis labels, "$y_i$" and "$\hat{y}_i$", are shown at different heights.

One might confuse the actual data value with the value predicted by the model.

The video distinguishes $y_i$, the observed outcome, from $\hat{y}_i$, the regression prediction, and gives a numerical example where they differ (2 tonnes versus 1.5 tonnes).

1. Audio
   Observation

   The speaker says it is "never going to be perfectly" and "simply impossible" for the line to go through every single data point.

One might expect the best regression line to match every observed data point exactly.

The video explicitly states that a line will generally not pass perfectly through every data point; residuals are normal and the method instead minimizes their squared sum.

1. Audio
   Observation

   The speaker emphasizes squaring the residuals before adding them.

2. Formula
   Observation

   SUM($y_i - \hat{y}_i$)^2 is minimized.

1. The video does not state this as a named misconception; this entry summarizes an implicit contrast between raw signed differences and squared differences.

One might think the best line should simply minimize the raw differences $y_i - \hat{y}_i$.

The clip specifies that ordinary least squares minimizes the sum of squared residuals, not the unsquared signed differences.

1. Audio
   Observation

   The speaker first asks how to know which sloped line is best, then says ordinary least squares is the method needed to answer those questions.

Ordinary least squares is introduced as the method that addresses the problem of choosing among multiple candidate regression lines.

1. Audio
   Observation

   The speaker explains ordinary least squares by saying we project data points vertically onto the regression line.

2. Animation
   Observation

   Dashed vertical segments connect points to the middle line.

The vertical-projection construction is part of how this clip explains the ordinary least squares method.

1. Audio
   Observation

   The speaker says $\hat{y}_i$ is what the linear regression predicts the yield to be.

2. Diagram
   Observation

   $\hat{y}_i$ is placed on the y-axis at the height of the regression line for the chosen point.

The meaning of $\hat{y}_i$ in this clip depends on the vertical projection of a data point onto the chosen regression line.

1. Audio
   Observation

   The speaker contrasts $y_i$ as the actual amount with $\hat{y}_i$ as the predicted amount.

2. Diagram
   Observation

   Both labels are shown simultaneously at different heights.

The clip explicitly contrasts the observed value $y_i$ with the fitted or predicted value $\hat{y}_i$.

1. Formula
   Observation

   residual: $\varepsilon_i = y_i - \hat{y}_i$ appears before SUM($y_i - \hat{y}_i$)^2 is minimized.

2. Audio
   Observation

   The speaker first defines the residual and then explains squaring and summing those residuals.

The least-squares objective depends on first defining what a residual is for each data point.

1. Formula
   Observation

   $\hat{y} = b_0 + b_1 X_1$ appears immediately before the minimization statement.

2. Audio
   Observation

   The speaker introduces the equation and then says the best equation is the one with parameters $b_0$ and $b_1$ such that the squared-residual sum is minimized.

Choosing $b_0$ and $b_1$ by least squares presupposes a linear regression equation for $\hat{y}$.

1. Diagram
   Observation

   The slide title is Simple Linear Regression and the subtitle is Ordinary Least Squares:

The clip presents ordinary least squares as the method contained within the broader topic of simple linear regression.

1. Audio
   Observation

   The speaker asks which of the sloped lines is the best one and notes that multiple lines can be drawn through the data points.

2. Animation
   Observation

   Several candidate lines appear over the scatter plot.

1. Motivation for needing a criterion for the best regression line
2. Ordinary Least Squares as the method under discussion

1. Audio
   Observation

   The speaker says ordinary least squares works by projecting data points vertically onto the regression line.

2. Animation
   Observation

   Dashed vertical segments connect points to the middle line.

1. Ordinary Least Squares as the method under discussion
2. Visual rule of ordinary least squares in this clip
3. Visual derivation of the ordinary least squares idea in this excerpt

1. Audio
   Observation

   The speaker defines $y_i$ as the actual amount of potatoes yielded for a specific fertilizer amount.

2. Diagram
   Observation

   The label "$y_i$" is aligned with a data point on the y-axis.

1. $y_i$
2. Distinction between actual value and predicted value
3. Potato-yield versus nitrogen-fertilizer example

1. Audio
   Observation

   The speaker defines $\hat{y}_i$ as what the linear regression predicts the yield to be.

2. Diagram
   Observation

   The label "$\hat{y}_i$" is aligned with the middle regression line on the y-axis.

1. $\hat{y}_i$
2. Distinction between actual value and predicted value
3. Potato-yield versus nitrogen-fertilizer example

1. Audio
   Observation

   The speaker gives the example of 15 kg nitrogen fertilizer, 2 tonnes actual yield, and 1.5 tonnes predicted yield.

1. The numbers are spoken, not printed on screen.

1. Potato-yield versus nitrogen-fertilizer example
2. Worked interpretation of the symbols $y_i$ and $\hat{y}_i$

1. Audio
   Observation

   The speaker says the procedure would be done for every candidate line but for simplicity the tutorial uses the middle line.

2. Animation
   Observation

   Only the middle line receives the dashed vertical projections.

1. Visual rule of ordinary least squares in this clip
2. Visual derivation of the ordinary least squares idea in this excerpt

1. Formula
   Observation

   residual: $\varepsilon_i = y_i - \hat{y}_i$

2. Audio
   Observation

   The speaker says the difference between actual and predicted yield is called the residual.

1. Residual definition

1. Formula
   Observation

   SUM($y_i - \hat{y}_i$)^2 is minimized

2. Audio
   Observation

   The speaker says, "And that's why it's called the ordinary least squares method."

1. Ordinary least squares objective

1. Audio
   Observation

   The speaker says the regression line with the smallest value is the best regression line.

2. Formula
   Observation

   SUM($y_i - \hat{y}_i$)^2 is minimized

1. Ordinary least squares objective
2. Best regression line under ordinary least squares

1. Audio
   Observation

   The speaker says the line is never going to go perfectly through every single data point.

1. A regression line generally will not pass through every data point
2. Expecting the regression line to pass through every point