What is the least squares cost function?

Conditions

• Used for regression problems where the goal is to predict a continuous value.
• Assumes a linear relationship between the features and the target variable.

Reasoning, step by step

1. Identify the predicted value for a data point: $\theta_1 x_1 + \theta_2 x_2$.
2. Identify the actual target value: $y$.
3. Calculate the error (difference) for the data point: $(\theta_1 x_1 + \theta_2 x_2 - y)$.
4. Square the error: $(\theta_1 x_1 + \theta_2 x_2 - y)^2$.
5. Sum the squared errors over all $m$ training examples and divide by $2m$.
6. The cost function is $J(\theta_1, \theta_2) = \frac{1}{2m} \sum (\theta_1 x_1 + \theta_2 x_2 - y)^2$.

Example

The video displays the formula on-screen as "$J(\theta_1, \theta_2) = \frac{1}{2m} \sum (\theta_1 x_1 + \theta_2 x_2 - y)^2$" and refers to it as the least square cost function.

Common misconceptions

• Forgetting the factor of $\frac{1}{2m}$, which averages the squared errors and simplifies the derivative calculation.
• Assuming the cost function measures the absolute difference rather than the squared difference.

Watch the explanation

Explore next

• What is the parameter update rule for gradient descent?
• Why does gradient descent oscillate when features are not scaled?
• What is the formula for standardization?