What a regression line is doing

Start with the picture

Each dot is one observation: an input x paired with an outcome y. A line turns a new input into a predicted outcome, written ŷ = b₀ + b₁x. The hat matters: ŷ is what the rule predicts, not what was actually observed.

Use the plot to make a prediction before reading the answer. Move the slope and intercept, watch the vertical gaps change, and try to lower the score. Then add the unusual point. The larger outlined dot is intentionally awkward: it shows that a line can be pulled by one observation even when the other points follow a different pattern.

Read one observation all the way through

Suppose the line is ŷ = 2 + 3x and an observation has x = 4 and y = 10. The prediction is 2 + 3(4) = 14. The residual is observed minus predicted:

e = y − ŷ = 10 − 14 = −4

The negative sign says the observation fell below the line. In plain language, the model overpredicted by four outcome units. If the observed outcome had been 18, the residual would be positive 4 and the model would have underpredicted. A residual is not “the model’s accuracy” in general; it is the signed miss for one row under one fitted rule.

This sign convention is worth saying out loud in an interview. “Residual equals actual minus predicted” prevents a common ambiguity, especially when a prompt asks whether the model is systematically too high or too low. A collection of mostly negative residuals suggests overprediction in that region; a collection of mostly positive residuals suggests underprediction.

What changes when you move the line

Changing the intercept shifts every prediction by the same amount. If b₀ increases by one, every fitted value increases by one, so every residual decreases by one. Changing the slope rotates the line around its intercept: the effect is small near x = 0 and larger at large positive or negative values of x.

The slope is measured in outcome units per input unit. If x is days to first value and y is seven-day retention measured in percentage points, a slope of −2 means the model predicts two fewer percentage points of retention for each additional day, over the range represented in the data. It does not automatically mean that removing one day will cause retention to rise by two points.

The intercept is the predicted outcome when x = 0. Sometimes that is meaningful; sometimes it is just where the line crosses the axis. If the data cover users who take between two and ten days to reach value, an intercept at zero days may be outside the observed range and should not be given a business interpretation without more context.

Worked example: calculate the line and one miss

Here is a small dataset of five practice sessions. Let x be sessions completed and y be a rubric score:

The centered products add to 9 and the squared deviations of x add to 10, so the slope is b₁ = 9 / 10 = 0.90. The intercept is b₀ = ȳ − b₁x̄ = 3.6 − 0.9(3) = 0.90. The fitted rule is therefore:

ŷ = 0.90 + 0.90x

For the fourth session, the model predicts 0.90 + 0.90(4) = 4.50. The observed score is 5, so the residual is 5 − 4.50 = 0.50. The model is below that observation by half a score point. Across all five rows, this line has a sum of squared residuals of 1.10. A mean-only baseline that predicts 3.6 every time has a sum of squared residuals of 9.20, so the line describes this sample much more closely under squared error.

That last comparison is useful because “good fit” needs a reference point. A line that reduces error relative to the mean is doing something. It still might fail on new users, hide a subgroup pattern, or describe an association that is not useful for a decision. The calculation establishes what happened in this sample; it does not finish the analysis.

Applied example: a product decision

Imagine a product team notices that users who take longer to reach their first successful workflow are less likely to return in seven days. They fit a line to summarize the relationship. Before discussing coefficients, define the rows and units: one row per user, x as hours to first successful workflow, and y as a seven-day return indicator or rate. If the outcome is a user-level 0/1 indicator, a line can still be a descriptive first pass, but predictions should be checked for sensible bounds and a probability model may be a better next step.

Suppose the slope is −0.03 when y is a probability. The safe interpretation is: within the observed range and conditional on this fitted specification, one additional hour is associated with a three-percentage-point lower predicted return probability. A strong analyst would then ask whether the pattern is concentrated in a particular acquisition channel, plan type, device, or user intent. They would evaluate held-out users and inspect residuals or calibration before using the line to prioritize work.

The unsafe interpretation is: “If we make onboarding one hour faster, retention will rise by three points.” That is a causal claim. Users who reach value quickly may already be more motivated, have simpler jobs, or arrive from a different channel. A randomized onboarding change, a credible quasi-experiment, or a clearly defended adjustment strategy is needed for an intervention claim.

Where a line can mislead you

Least squares always returns the best line for its chosen squared-error objective, even when a line is a poor description of the data. In an interview, name the failure mode and the next check you would run.

• Curvature: if outcomes rise and then level off, a straight line leaves a pattern in the residuals. Plot residuals against x and compare a nonlinear or transformed specification.
• Influential observations: a point far out in x can rotate the line substantially. Refit with and without the point, verify the row, and report how much the conclusion changes.
• Extrapolation: a line fitted from two to ten days does not become trustworthy at 100 days because the equation still produces a number. Mark the observed range and ask whether the new case is comparable.
• Changing spread: if residuals fan out as x grows, the average line may be useful while uncertainty and error size vary across the range. Consider transformations, weighted methods, or prediction intervals in a later analysis.
• Subgroups and leakage: an overall line can average together groups with different slopes, and a feature measured after the outcome can make the fit look better than it would be at prediction time.

Nearest alternatives

The closest comparison is the mean-only baseline: predict the same ȳ for every row. A regression line earns its keep by using x to reduce error relative to that baseline. If the outcome is strongly curved, a nonlinear model may be a better description. If a few large misses dominate the squared loss, an absolute-error or robust approach may better match the decision. These alternatives change what “close” means; they are not magic repairs.

Also keep the prediction-versus-explanation distinction visible. A line can be a useful predictor without being a good explanation of why the outcome changes. Conversely, a relationship that is scientifically meaningful may not give accurate individual predictions. The right standard depends on the interview prompt: ask whether the goal is forecasting, ranking, diagnosis, or deciding what intervention to try.

Six interview drills

1. Translate a regression plot for a nontechnical partner.

   Strong answer: each dot is an observed pair, the line gives a predicted outcome for an input, and each vertical gap is observed minus predicted. Explain the units before mentioning how the line was fitted.

   Probe: what changes if the intercept increases? Signal: the candidate can move between a picture, an equation, and plain language.

2. For ŷ = 2 + 3x and point (4, 10), what is the residual?

   Strong answer: prediction is 14 and residual is 10 − 14 = −4; the model overpredicts by four outcome units. Probe: what would a positive residual mean? Signal: the sign convention is explicit rather than guessed from the picture.

3. Interpret a slope of −2 in a model of delivery time versus number of items.

   Strong answer: the model predicts two fewer time units for each additional item, within the observed range and conditional on the model. First check whether the units and direction make domain sense; a negative delivery-time slope might reveal a definition or selection problem.

   Probe: is that causal? Signal: separates association from an intervention claim.

4. Why do we square residuals instead of adding them?

   Strong answer: signed errors would cancel, while squaring makes all contributions nonnegative and emphasizes large misses. That is a choice with consequences, not a law of nature.

   Probe: what if one bad row dominates? Signal: mentions robust or absolute loss and data-quality checks.

5. The line fits the sample closely but predictions fail for a new segment. What do you inspect?

   Strong answer: verify the target and feature timing, compare train and segment ranges, inspect residuals and calibration by slice, look for curvature and influential points, and validate on comparable held-out data.

   Probe: what would you plot first? Signal: proposes a diagnostic tied to a failure hypothesis.

6. A product lead asks whether a steeper line proves that changing x will change y. How do you respond?

   Strong answer: the slope summarizes an association in the observed data. To claim that changing x changes y, define the intervention and address confounding with an experiment or credible causal design.

   Probe: what decision can the descriptive line still support? Signal: preserves useful prediction while refusing an unsupported causal leap.

Timed synthesis

Three quick checks

1. What does the sign of a residual tell you?

   With residual defined as observed minus predicted, a positive value means the observation is above the line and a negative value means it is below the line.

2. Why is a mean-only baseline worth keeping?

   It is the simplest prediction that ignores x. If a fitted line cannot beat it on an appropriate evaluation set, the extra complexity has not earned its place.

3. When should you distrust the line before interpreting its slope?

   When the relationship is curved, the spread changes, an influential point controls the fit, the new case is outside the observed range, or the feature is measured after the target.

Keep this in mind

A line is a prediction rule with a particular loss function. The slope has units, a residual has a sign, and a close fit is only a starting point. In the next lesson, we will make the “smallest squared error” claim precise and see why the mean and centered data appear in the OLS solution.