Objective
Given a small dataset, compare a regression line with the mean-only baseline and explain how minimizing the sum of squared residuals determines the OLS slope and intercept.
Prediction first
How much can a slope beat the mean?
The horizontal baseline predicts the average score, 4, for every week. Its SSE is 10. Before revealing a fitted line, predict how much of that squared error the best slope removes.
Choose a range, then reveal the fitted line.
Look for this: the line pivots through the average point while the loss marker traces a bowl. At slope 0.90, SSE reaches 1.90—the bottom of the bowl—versus 10.00 for predicting the mean every time.
From baseline to objective
Without using x, the best constant prediction under squared error is the mean outcome, ȳ. It is the baseline OLS must beat. For any candidate line ŷᵢ = b₀ + b₁xᵢ, observation i leaves a residual:
eᵢ = yᵢ − ŷᵢ
Positive and negative residuals would cancel if we simply added them. Squaring keeps every miss positive and makes a large miss matter more. Ordinary least squares chooses the coefficients that minimize:
SSE(b₀, b₁) = Σᵢ [yᵢ − (b₀ + b₁xᵢ)]²
Why the line crosses the average point
At the minimum, nudging the intercept up or down cannot improve SSE. That condition makes the residuals sum to zero, so their average is zero:
ȳ = b₀ + b₁x̄ → b₀ = ȳ − b₁x̄
Put x̄ into the line and its prediction is ȳ. That is why the interaction can hold the average point fixed while you change the slope. Centering both variables then gives the slope:
b₁ = Σᵢ(xᵢ − x̄)(yᵢ − ȳ) / Σᵢ(xᵢ − x̄)²
The numerator measures whether x and y move together; the denominator scales that movement by how much x varies. If every x is identical, the denominator is zero and a slope cannot be learned.
Worked example
For the five points in the interaction, x̄ = 3 and ȳ = 4. Predicting 4 every week gives residuals −2, −1, 1, 0, and 2, so the baseline SSE is 10.
Slopeb₁ = 9 / 10 = 0.90
Interceptb₀ = 4 − 0.90(3) = 1.30
Fitted ruleŷ = 1.30 + 0.90x
Squared errorSSE = 1.90
The fitted line removes 8.10 of the baseline's 10 squared-error units in this sample. That is an in-sample comparison, not evidence that practice caused the score change or that future predictions will be equally accurate.
Common interview trap
“The residuals sum to zero, so the model has zero error.”
False. With an intercept, positive and negative residuals balance by construction. OLS minimizes the sum of their squares, which is 1.90 here—not zero. Always name the objective precisely.
Where least squares can fail you
The optimizer only answers, “Which candidate line has the smallest training SSE?” It does not prove that a line is appropriate, that the relationship is causal, or that the model will generalize. Squared loss is also sensitive to unusual observations because one large miss receives disproportionate weight. In an interview, separate the mechanical fit from the checks you would perform next: validation, residual patterns, influential points, and domain plausibility.
Three quick checks
Why is the mean-only prediction the right squared-error baseline?
For constant prediction c, the derivative of Σ(yᵢ − c)² is zero at c = ȳ. Any other constant has larger squared error.
Why must an OLS line with an intercept pass through (x̄, ȳ)?
At the optimum, residuals sum to zero. Rearranging Σ[yᵢ − (b₀ + b₁xᵢ)] = 0 gives b₀ = ȳ − b₁x̄, so the fitted value at x̄ is ȳ.
If training SSE is tiny, what can you safely conclude?
Only that the line fits this sample closely under squared loss. You still need out-of-sample validation and diagnostic checks; tiny SSE alone does not establish causality, correct functional form, or reliable inference.
Try this in an interview
You are given five observations and asked to fit a simple regression by hand. Explain the mean-only baseline, derive the slope and intercept, compare SSEs, and finish with one conclusion the fit supports and two conclusions it does not. Aim for a five-minute answer.