Holding other variables constant

Learning objective

Given a multiple-regression equation or output table, interpret a coefficient in units and in context, encode a categorical predictor, calculate an interaction or transformed prediction, and state what the model does—and does not—justify. A strong interview answer names the outcome, row unit, reference group, comparison point, observed support, and whether the claim is predictive, descriptive, or causal.

Why add a second predictor?

In simple regression, a line turns one input into a prediction. With predictors x₁, …, xₚ, the additive multiple-regression rule is:

ŷ = b₀ + b₁x₁ + b₂x₂ + ··· + bₚxₚ

“Linear” describes the coefficients, not necessarily the raw columns. A model can include x², log(x), or a product x₁x₂ and still be linear in the unknown coefficients. The fitted rule is chosen to minimize the same squared-error objective as before:

SSE(b) = Σᵢ [yᵢ − (b₀ + b₁xᵢ₁ + b₂xᵢ₂ + ···)]²

Adding a column cannot increase training SSE: the larger model can always set the new coefficient to zero and recover the smaller model. That is a statement about optimization on the training sample, not a guarantee of better held-out predictions, more stable coefficients, or a better decision. Complexity must earn its place through validation and a credible interpretation.

For an additive model, b₁ is the change in predicted outcome for a one-unit increase in x₁, comparing profiles with the same included values of x₂, …, xₚ. It is often called a partial or conditional slope. If onboarding sessions and prior intent move together, the multiple coefficient uses the part of session variation that remains after accounting for intent. That can differ from the simple slope because shared variation is being allocated differently; it is not automatically “bias correction.”

Mechanism: residualize before you compare

The geometry becomes useful in an interview. To understand b₁, first predict x₁ from the other predictors and keep its residual: the session variation not predictable from intent. Separately, predict y from the other predictors and keep that residual: the engagement variation not predictable from intent. The slope between those two residuals is the multiple-regression coefficient for sessions. This is the intuition behind the partial-slope result often called the Frisch–Waugh–Lovell theorem.

In matrix notation, ŷ = Xb, and ordinary least squares chooses b to minimize ‖y − Xb‖². With an intercept, unique unregularized coefficients require the design matrix to have full column rank. Exact duplicate columns make infinitely many coefficient vectors produce the same predictions. Near duplicates can leave predictions fairly stable while making individual coefficients large, noisy, or sign-sensitive.

This is why “all else equal” should be followed by a support check. If every Premium user with five sessions also has high intent, the dataset may not contain the comparison needed to isolate a five-session effect at low intent. A fitted number outside that overlap is extrapolation disguised as adjustment.

Worked calculation: two predictors by hand

Suppose one row is a user. Let x₁ be onboarding sessions, x₂ be a prior-intent score, and y be later engagement units. The five observations are:

The means are x̄₁=3, x̄₂=1, and ȳ=20. From the centered columns, calculate the cross-products:

The centered normal equations are therefore:

10b₁ + 2b₂ = 26    and    2b₁ + 2b₂ = 10

Subtract the second equation from the first: 8b₁=16, so b₁=2. Substitute into the second equation: 2(2)+2b₂=10, giving b₂=3. Recover the intercept from the means:

b₀ = ȳ − b₁x̄₁ − b₂x̄₂ = 20 − 2(3) − 3(1) = 11

Our fitted equation is ŷ=11+2x₁+3x₂. For user B, the prediction is 11+2(2)+3(1)=18. The residual is observed minus predicted: 16−18=−2. The five fitted values are 13, 18, 23, 22, and 24; the residuals are 1, −2, 1, 0, and 0, so SSE=1²+(−2)²+1²=6 engagement-units-squared. The mean-only baseline has SST=88, so the training improvement is 82 squared units.

Compare that with the simple regression of y on x₁ alone: its slope is 26/10=2.6. The multiple slope of 2.0 means the session comparison is made after accounting for the intent column. It does not prove that adding a session causes two engagement units, and five rows are not enough to validate a product policy.

Categorical predictors: name the reference

Suppose plan has three nominal levels: Basic, Plus, and Premium. With an intercept, use two indicators, for example I(Plus) and I(Premium), with Basic as the reference. The additive model might be:

ŷ = b₀ + b₁sessions + b₂intent + b₃I(Plus) + b₄I(Premium)

At sessions and intent equal to zero, b₀ is the predicted Basic outcome. The Plus coefficient compares Plus with Basic at the same numeric predictor values; Premium does the same. These are comparisons of model predictions, not universal group averages. If zero sessions is not a meaningful profile, center sessions—say, sessions minus seven—so the intercept and plan coefficients describe a typical week instead.

Do not code Basic, Plus, Premium as 1, 2, 3 unless order and spacing are substantively justified. That encoding would force the model to treat Premium minus Plus as the same step as Plus minus Basic. Including all three indicators with an intercept creates exact dependence because the indicators sum to one. Use K−1 indicators, a deliberate no-intercept parameterization, or another contrast system tied to the question.

Interactions: when the slope depends on the group

An additive model assumes the session slope is shared. If the response to sessions may differ by plan, include an interaction:

ŷ = b₀ + b₁x + b₂g + b₃(xg),   g ∈ {0,1}

For the reference group g=0, the line is ŷ=b₀+b₁x, so its slope is b₁. For g=1, the line is ŷ=(b₀+b₂)+(b₁+b₃)x, so its slope is b₁+b₃. The group gap at a chosen x is b₂+b₃x.

This makes the main effect conditional. b₂ is the plan gap at x=0, not a global “Premium effect.” If the interaction coefficient is −0.7, a Basic slope of 2.0 becomes 1.3 for Premium. At five sessions, the gap is not just b₂; it is b₂−0.7(5). Centering sessions at a meaningful value changes the point where b₂ is evaluated without changing fitted predictions when the terms are transformed consistently.

Transformations change the units

Transformations can express curvature, reduce skew, or make a multiplicative relationship easier to model. They also change the sentence you are allowed to say:

A logged outcome is also a different target. The mean of log(1+Y) is not simply the log of the mean of Y. If the business needs expected watch hours in the original units, plan how to retransformed predictions and validate that quantity directly.

Worked product scenario: forecast versus intervention

A streaming team wants to forecast 30-day watch hours and prioritize onboarding resources by plan. One row is a new account. Features available at signup or in a pre-specified early window are trial days, device count, plan, and acquisition channel. They model the skewed target as:

log(1 + watchHours₃₀) = b₀ + b₁trialDays + b₂devices + b₃Premium + b₄Paid + b₅(trialDays × Premium) + ε

Basic and Organic are the references. If trial days is centered at seven, the Premium coefficient is the plan gap for a typical one-week trial. Suppose b₁=.06, b₃=.10, and b₅=−.02. The trial-day slope is .06 for Basic and .06−.02=.04 for Premium. At a particular trial length, the plan gap is b₃+b₅x, not b₃ alone. The negative interaction says the modeled association is weaker for Premium; it does not say Premium customers watch less, and it definitely does not say shortening or extending a trial will cause a change.

Before the team changes trial policy, I would do four things. First, verify timing: no feature may be measured after watch behavior or after the proposed intervention. Second, check support and missingness by plan and channel; “same device count and trial length” must describe real comparable accounts. Third, compare additive and interaction specifications on a time-based holdout, inspecting error by plan. Fourth, define the intervention and randomize trial length or onboarding treatment if the question is whether a policy changes watch time. The regression is useful for targeting or forecasting; the experiment supplies evidence for the policy effect.

Assumptions and failure modes

At the point where you interpret a coefficient, ask whether the conditional mean is plausible, the columns are measured correctly, the rows are comparable, and the target is available at prediction time. Exact rank is needed for a unique unregularized coefficient vector. Independent rows and stable variance matter for classical uncertainty statements. Normal predictors are not required for OLS to compute a point estimate; normal errors are mainly a small-sample inference assumption, not a prerequisite for fitting the line.

• Near multicollinearity: correlated predictors can make coefficients unstable and signs sensitive to small data changes. Check condition numbers, correlation structure, coefficient stability, and whether the decision needs individual coefficients. Ridge can stabilize prediction; it does not repair confounding or preserve the same coefficient meaning.
• Omitted or post-treatment variables: adding a control can change a coefficient because it changes the comparison, not because the newest coefficient is automatically more truthful. Audit feature timing and the causal role of each control; do not adjust casually for a mediator or collider.
• Bad specification: curvature, unequal spread, group-specific slopes, and missing interactions can leave systematic residual patterns. Compare plausible transformations and interactions, then evaluate held-out performance rather than collecting terms until training SSE looks impressive.
• Support and extrapolation: if the data do not contain overlap, “holding constant” requires a counterfactual comparison the sample cannot show. Mark the range and report which slice is unsupported.
• Leakage and shift: a feature recorded after signup or after the outcome can make a forecast look excellent and fail at serving time. Use point-in-time feature definitions and a deployment-like temporal split.
• Causal overreach: a conditional association is not an intervention effect. Causal adjustment requires a defined treatment and outcome, pre-treatment confounders, overlap, and defensible design assumptions; randomization or a credible identification strategy is stronger evidence.

Nearest alternatives: choose by the question

The interview trap

Three checkpoint questions

1. What does a coefficient mean in an additive multiple-regression model?

   It is the change in predicted outcome per one-unit increase in that predictor, comparing otherwise equal rows on the included predictors and staying within observed support. It is not automatically a causal effect.

2. Why do three nominal plan levels need two indicators when an intercept is present?

   One level is the reference, and the other two indicators describe differences from it. All three indicators plus an intercept are exactly dependent because they sum to one.

3. In ŷ=b₀+b₁x+b₂g+b₃xg, what are the two slopes and the group gap?

   For g=0, the slope is b₁. For g=1, it is b₁+b₃. At a chosen x, the group gap is b₂+b₃x.

Interview practice bank

1. 1. Define a multiple-regression coefficient without saying “all else equal” and stopping there.

   Strong-answer outline: define the outcome and units; say it is a one-unit change in predicted conditional mean with included comparison variables fixed; restrict to observed support; distinguish association from causality. Likely follow-up: what if the predictors are strongly correlated? Signal: precise conditional language and support awareness.

2. 2. Fit the five-row, two-predictor example by hand.

   Strong-answer outline: state the means; compute the five centered cross-products; solve 10b₁+2b₂=26 and 2b₁+2b₂=10; recover b₀=11; calculate one fitted value, residual, and SSE. Likely follow-up: why does the session slope differ from 2.6? Signal: auditable arithmetic and understanding of shared variation.

3. 3. How would you encode a three-level nominal plan variable?

   Strong-answer outline: choose a reference, create two indicators with an intercept, and interpret each indicator as a comparison at the same numeric covariates. Explain why 1/2/3 coding imposes unjustified order and spacing. Likely follow-up: what changes if you relevel the reference? Signal: understands parameterization versus fitted predictions.

4. 4. Interpret an interaction with coefficients 2.0 for sessions, 1.5 for Premium, and −0.7 for sessions × Premium.

   Strong-answer outline: Basic slope is 2.0; Premium slope is 1.3; the Premium-minus-Basic gap at sessions x is 1.5−0.7x; the 1.5 main effect is only at x=0. Likely follow-up: would you center sessions? Signal: can reason from group-specific lines instead of reading one coefficient globally.

5. 5. A log-log model has coefficient .4. What sentence would you use?

   Strong-answer outline: approximately a 0.4% change in modeled outcome for a 1% change in predictor, conditional on the other terms and within support; call it an elasticity, not a raw-unit slope. Likely follow-up: how is this different from a log-outcome, level-predictor model? Signal: tracks units and transformations.

6. 6. A coefficient flips sign after adding a feature. Diagnose before you explain.

   Strong-answer outline: check joins, row grain, missingness, coding, units, leakage, feature timing, correlation/conditioning, support, influence, and whether the new variable changes the estimand. Then compare stability and held-out performance. Likely follow-up: does collinearity bias coefficients? Signal: separates instability from data or specification bias.

7. 7. When would you choose additive OLS, an interaction, or ridge?

   Strong-answer outline: additive OLS for a credible common slope and interpretability; interaction for substantively or predictively supported slope differences; ridge for correlated features when prediction stability matters. Name validation and do not claim ridge fixes confounding. Likely follow-up: can a ridge coefficient be read as the same partial slope? Signal: ties model choice to the decision and data geometry.

8. 8. Can this regression tell a product lead whether changing trial length increases retention?

   Strong-answer outline: first separate forecasting from intervention. For prediction, audit point-in-time features, temporal validation, and support. For causality, define treatment, outcome, population, and assumptions; avoid post-treatment controls; randomize or defend an identification strategy. Likely follow-up: what decision can the model support today? Signal: preserves useful modeling while refusing causal overreach.

Timed synthesis: eight minutes

Interview-ready recap

• Coefficient: conditional change in predicted outcome per unit, with named controls fixed and within support.
• Design matrix: OLS minimizes ‖y−Xb‖²; full rank gives unique unregularized coefficients.
• Category: use K−1 indicators with a reference when an intercept is present.
• Interaction: group slopes differ; compute the gap at the x-value you care about.
• Transformation: log and polynomial terms change units and often the target scale.
• Failure: collinearity, leakage, misspecification, weak overlap, and causal overreach can invalidate a confident sentence.
• Interview order: name the outcome and row, state the model and reference, calculate the comparison, check support/validation, then bound the decision.

Explain without notes: draw two parallel lines for an additive model, then tilt one for an interaction. In sixty seconds, state the reference group, both slopes, and why a fitted association is not an intervention effect.

Sources and boundaries

Technical claims are grounded in Penn State STAT 501 on multiple linear regression, categorical predictors, transformations, and multicollinearity and pitfalls (primary current academic sources). The scikit-learn linear-model guide and statsmodels formula guide are official documentation for design-matrix and feature-expansion details. Hernán and Robins, Causal Inference: What If supports the boundary between conditional modeling and causal claims. Employer materials calibrate interview emphasis, not universal interview prevalence. Detailed inference, residual diagnostics, and formal causal adjustment belong to adjacent lessons.