Deriving the least squares estimators of the slope and intercept (simple linear regression)

The least-squares objective function sums the squared residuals over all observations. A residual is defined as the observed y value minus the line’s predicted value.

Conditions: The model is simple linear regression.; Residuals are defined as observed y minus predicted y.

By setting the partial derivative of the least-squares objective with respect to the intercept to zero, we obtain the relationship: intercept = mean(y) − slope · mean(x). This implies that the fitted line always passes through the point of sample means.

Conditions: The model includes an intercept term.; The derivative is taken with respect to the intercept parameter.

The least-squares slope is calculated as the sum of the products of centered x and y values, divided by the sum of squared centered x values. Mathematically, this is $\Sigma[(x_i - \bar{x})(y_i - \bar{y})] / \Sigma(x_i - \bar{x})^2$.

Conditions: The denominator $\Sigma(x_i - \bar{x})^2$ is positive.; The model is simple linear regression with an intercept.

The unique slope formula is unavailable if all x values are identical. In this case, the denominator of the slope formula, which is the sum of squared deviations of x from its mean, becomes zero.

Conditions: All observed x values are identical.; The model attempts to fit a line with a unique slope.