Sampling distributions and the CLT

Learning objective

Given a target population, an estimator, and a sampling plan, explain what the estimator’s sampling distribution represents; calculate its center and standard error; use the Central Limit Theorem (CLT) for a defensible approximate probability or precision statement; and diagnose when normal approximation or 1/√n precision is not justified.

1. Three distributions that interviewers expect you to separate

Imagine a population of eligible new users and their 30-day revenue. There are three different objects hiding in ordinary language:

• The population distribution describes individual revenue values. It may be zero-inflated and strongly right-skewed.
• The sample distribution is the histogram of the particular users you happened to observe. It is one realized dataset.
• The sampling distribution is the distribution of a statistic—say, X̄—over repeated samples drawn by the same design. It describes how the estimate would vary if the data collection were repeated.

The third object is the key. A parameter such as μ is fixed under the population model. The statistic X̄ is random before sampling because a different sample would usually give a different value. The standard error is the standard deviation of that sampling distribution. It is not the standard deviation of individual observations, and it is not automatically a measure of bias.

That distinction gives you a clean answer when an interviewer asks, “What does a standard error of 5 mean?” Say: “If I repeatedly drew samples of this size using this design, the resulting estimates would have a standard deviation of about 5 under the model.” Add the sampling unit and assumptions before turning that statement into a decision.

2. Why averaging reduces random variation

Let X₁, …, Xₙ be independent, identically distributed observations with mean μ and variance σ². The sample mean is:

X̄ = (1/n) Σ_i=1ⁿ X_i

Linearity of expectation does not require independence:

E[X̄] = (1/n) ΣE[X_i] = (1/n)(nμ) = μ

Variance is where the design enters. Under independence, all cross-covariance terms are zero:

Var(X̄) = (1/n²) ΣVar(X_i) = (1/n²)(nσ²) = σ²/n

Taking a square root gives the standard error:

SE(X̄) = σ/√n

The formula says two useful things at once. First, the population spread σ has not shrunk; individual users are still variable. Second, positive and negative deviations partly cancel in an average, so the estimate’s spread shrinks. The rate is deliberately slow: quadrupling n halves the standard error. To halve it again, you need four times as many independent sampling units.

When σ is unknown, replace it with the sample standard deviation s and use s/√n as an estimated standard error. This plug-in estimate inherits the design assumptions. It does not turn a convenience sample into a random sample or repeated events into independent users.

3. What the CLT adds—and what it does not

The variance calculation tells you the spread of the mean under the model. The CLT adds a shape statement: after centering and scaling, the sample mean approaches a standard normal distribution as n grows under suitable conditions:

Z = (X̄ − μ)/(σ/√n) → N(0, 1)

“Approaches” matters. The theorem concerns a sequence of sampling distributions as the sample size increases. It is not a guarantee attached to an arbitrary dataset, and it does not contain a universal n = 30 switch.

There are two especially useful contrasts:

• If the population itself is normal, X̄ is exactly normal for every n. The CLT is not needed to prove that special case.
• If the population is strongly right-skewed, a small-sample mean can remain right-skewed. As n grows, the shape often becomes more normal-like, but the required sample size depends on skew, tail behavior, dependence, and how accurate the tail probability needs to be.

The CLT is a statement about random sampling variation. It does not make a self-selected sample representative, fix informative missingness or measurement error, remove interference, or transform a descriptive mean into a causal effect. More independent rows can make a wrong estimate more precise around the wrong target.

4. Worked calculation: a mean of processing times

Question. A transaction-processing population has mean μ = 50 ms and standard deviation σ = 12 ms. A simple random sample of n = 36 independent transactions is summarized by its mean. What is the approximate probability that the sample mean is between 47 ms and 54 ms?

Step 1 — identify the statistic and its center. The statistic is the mean of 36 transaction times. Under simple random sampling, E[X̄] = μ = 50 ms.

Step 2 — calculate variance and standard error.

Step 3 — standardize both boundaries. Subtract the center and divide by the standard error:

P(47 ≤ X̄ ≤ 54) = P((47−50)/2 ≤ Z ≤ (54−50)/2) = P(−1.5 ≤ Z ≤ 2)

Using standard normal probabilities, Φ(2) ≈ 0.9772 and Φ(−1.5) ≈ 0.0668. Therefore:

P(47 ≤ X̄ ≤ 54) ≈ 0.9772 − 0.0668 = 0.9104

Interpretation. Under the stated population and sampling design, about 91% of repeated independent samples of 36 transactions would produce a mean in that interval. This is a repeated-sampling probability about a future statistic; it is not a 91% probability statement about a fixed population mean. If the 36 log events came from only six users, the iid calculation would be suspect because user-level shocks create dependence.

5. Proportions and finite populations: same logic, different boundaries

A proportion is an average of zero/one indicators. If Iᵢ = 1 when user i converts and Iᵢ = 0 otherwise, then p̂ = (1/n)ΣIᵢ. With independent users and conversion probability p:

SE(p̂) = √[p(1−p)/n]

In practice, use p̂ as a plug-in value or a planning value. For 1,000 independent users and a planning conversion rate of 8%:

SE(p̂) = √[0.08 × 0.92 / 1000] = √0.0000736 ≈ 0.00858

That is about 0.86 percentage points. Check the expected successes and failures before relying on a normal approximation. Three conversions out of 1,000 is not a comfortable setting for a symmetric normal tail; an exact binomial method or a method designed for rare events may be more defensible.

Sampling without replacement from a finite population also changes the variance. If a population has size N and you sample a substantial fraction without replacement, the finite-population correction gives:

Var(X̄) = (1 − n/N)S²/n

The negative dependence created by sampling without replacement can make the estimate more precise than the independent-draw formula suggests. The lesson is not “always apply a correction”; it is “derive the uncertainty from the data-collection design.”

6. Worked product scenario: 30-day revenue per eligible user

Question. A mobile product team is deciding whether an onboarding change is worth further investment. The target estimand is mean 30-day revenue per eligible randomized new user, including users who spend $0. Instrumentation has been checked. A simple random sample of 400 eligible users has mean revenue $4.80 and sample standard deviation $16.00. Revenue is zero-inflated and right-skewed. What can the team say?

Step 1 — name the unit and target. One row must be one eligible new user, not one purchase, session, or payment event. Including zeros makes the estimand mean revenue per eligible user. Excluding non-payers would answer a different question: mean revenue conditional on paying.

Step 2 — check the design before the formula. Confirm that the 400 users represent the target population, no user is counted twice, follow-up is handled consistently, and the 30-day window is defined. A skewed raw histogram does not automatically disqualify a normal approximation for the mean at n = 400, but tail heaviness and dependence still deserve a check.

Step 3 — estimate random sampling variability.

SÊ(X̄) = s/√n = 16/√400 = 16/20 = $0.80 per user

Step 4 — translate precision into business units. A rough 95%-style normal margin is 1.96 × $0.80 = $1.568. Centered on the observed mean, that is approximately $4.80 ± $1.57, or $3.23 to $6.37. Formal interval construction and interval interpretation belong to the next inference lesson; here, use the range as a planning approximation.

Step 5 — make a bounded decision. If the investment requires confidently planning on at least $5 per eligible user, this sample is not precise enough for that claim. Using provisional σ = $16 and a desired 95%-style half-width of $1, a planning calculation gives n ≈ (1.96 × 16/1)² = 983.45, so plan for about 984 independent users and revisit the variance estimate.

Step 6 — stress-test the tempting conclusion. A narrow standard error would not fix sampling only users who opened the app. Nor would 4,000 event rows imply 4,000 independent users if each user contributed ten rows. If treatment changes retention and missing 30-day revenue is informative, missingness is part of the estimand and analysis design. The CLT can justify an approximation conditional on the design; it cannot validate the design itself.

7. Dependence, selection, and the failure of “more rows”

If observations are dependent, write the variance rather than silently dropping the covariance terms:

Var(X̄) = (1/n²)[ΣVar(Xᵢ) + 2Σ_i<jCov(Xᵢ, Xⱼ)]

Positive within-cluster covariance makes the variance larger than σ²/n. Ten events from one user are not ten independent users; the nominal row count overstates the information. In an experiment, clusters may be households, stores, classrooms, or geographic regions. In a time series, nearby observations can share trends and shocks. Use a cluster-level analysis, a design-aware standard error, or a model that represents the dependence.

Selection creates a different problem. Suppose a team samples only highly engaged users. The sample mean may be very stable, yet it estimates the mean for highly engaged users rather than the mean for all eligible users. Random sampling variability can be small while systematic error is large. Ask “precise around what target?” before celebrating a small standard error.

A tiny population can also defeat a casual CLT slogan. For the finite population [0, 0, 0, 0, 10], drawing two elements without replacement produces ten equally likely unordered samples: six means of 0 and four means of 5. The sampling distribution is discrete and visibly non-normal. Its mean is (6×0 + 4×5)/10 = 2. With finite-population variance S² = 10, the exact variance is (1−2/5)(10/2) = 3, so its standard deviation is √3 ≈ 1.732. Saying “the sample is about 30” would not even be meaningful here.

8. Nearest alternatives and the decision boundary

The nearest alternative is usually bootstrap or simulation. Use the analytic CLT approximation when the statistic is a mean or proportion, the sampling design supports a tractable variance, and you need a fast calculation that can be explained aloud. Use simulation when the statistic is nonlinear, the sample is small or highly skewed, or you want to inspect how shape changes with n. Use the correct resampling unit: users or clusters, not rows.

Bootstrap and simulation inherit the sample’s representativeness. They can approximate a sampling distribution more flexibly, but they do not cure convenience sampling, tiny samples, extreme tails, or a missing dependence structure. For a small rare count, an exact binomial calculation may be better than a symmetric normal approximation. For a finite population, use the sampling design and its correction. The decision boundary is the estimand, statistic, and data-generating process—not a preference for “parametric” or “nonparametric.”

Common interview trap

Interview practice bank

1. 1. What is a sampling distribution?

   Strong answer: It is the distribution of a statistic over repeated samples drawn with the same design. It is not the histogram of individual observations in one sample. For a mean, it describes sampling variability around the population mean. Follow-up: Can one realized sample mean equal the population mean? Signal: identifies what is random and keeps parameter, sample, and statistic separate.

2. 2. Derive why the standard error of a mean is σ/√n.

   Strong answer: Write X̄=(1/n)ΣXᵢ, use linearity for E[X̄]=μ, then use independence to remove covariance terms and get Var(X̄)=σ²/n. Take the square root. Replace σ with s when it is unknown. Follow-up: What changes under correlation? Signal: explains the mechanism instead of quoting a formula.

3. 3. Your sample mean is 120 and its estimated standard error is 5. What does 5 mean?

   Strong answer: Across repeated samples of the same size and design, the mean would have a standard deviation of about 5 under the model. It is not the spread of individual observations and not a guarantee that this estimate is within 5 of the parameter. Follow-up: Does a small SE prove the estimate is unbiased? Signal: separates random variation, bias, and individual-level spread.

4. 4. Does n = 30 make any sample mean approximately normal?

   Strong answer: No. Required sample size depends on population skew, heavy tails, discreteness, dependence, and the tail accuracy required. A normal population is exact at any n; a severe right tail may need much more or a simulation. Follow-up: What would you inspect for a revenue metric? Signal: treats the CLT as asymptotic and design-dependent.

5. 5. A conversion rate is 8% from 1,000 independent users. What is its approximate SE?

   Strong answer: Treat conversion as a Bernoulli indicator: √(.08×.92/1000) ≈ .00858, or 0.86 percentage points. Check expected successes and failures; with only a few conversions, prefer an exact or boundary-aware method. Follow-up: What if users are clustered by household? Signal: computes with units and states the approximation boundary.

6. 6. A dashboard reports a tiny SE from 100,000 events generated by 1,000 users. Do you trust it?

   Strong answer: Not before checking the sampling unit, deduplication, within-user dependence, target population, missingness, and exposure. The event-level formula may understate uncertainty; a user- or cluster-level analysis may be needed. Follow-up: How would positive correlation change the variance? Signal: diagnoses the data-generating process before calculating.

7. 7. When would you use the CLT versus bootstrap for a median?

   Strong answer: The simple σ/√n CLT story is for means and other statistics with a suitable variance derivation. A median has a different sampling distribution, so bootstrap or a specialized asymptotic method may be useful when the sample represents the target and the resampling unit is correct. Follow-up: Would bootstrap fix convenience sampling? Signal: chooses by estimand and design, not by a method slogan.

8. 8. A product manager asks whether average wait time is below two minutes from a skewed sample. How do you proceed?

   Strong answer: Define the target population, time window, sampling unit, and threshold decision; check representativeness, duplicate customers, and dependence; compute x̄ and s/√n only after those checks; then report an uncertainty range in minutes. If the approximation is weak, simulate or use a method aligned to the design. Follow-up: What if the sample contains only weekday commuters? Signal: starts with the decision and data-generating process.

Timed synthesis · five minutes

Three quick checks

1. Why does the mean get less variable as n increases?

   Independent variances add, while dividing a sum by n contributes 1/n². The result is σ²/n, so the standard error is σ/√n.

2. Does a skewed raw metric rule out a normal approximation for its mean?

   No. Individual observations can remain skewed while sufficiently large-sample means become approximately normal. The needed n depends on skew, tails, dependence, and desired accuracy.

3. What does a small standard error fail to prove?

   It does not prove representativeness, independence, absence of selection or measurement bias, correct missingness handling, causality, or that any individual observation is close to the mean.

Interview-ready recap

• Sampling distribution: distribution of a statistic across repeated samples drawn by the same design.
• Mean: E[X̄] = μ and SE(X̄) = σ/√n under iid sampling.
• CLT: the standardized sampling distribution of many suitable means becomes approximately normal; the raw data do not become normal.
• Scaling: quadruple independent n to halve the standard error.
• Design first: identify the target population, sampling unit, dependence, exposure, missingness, and selection.
• Proportions: SE(p̂) = √[p(1−p)/n]; rare counts need extra care.
• Failure mode: more dependent or selected rows can yield a precise estimate of the wrong target.
• Decision: use CLT, exact methods, or simulation based on the statistic and design—not a magic cutoff.

Technical framing follows Penn State STAT 414 on the Central Limit Theorem, Penn State STAT 500 on sampling distributions, and the NIST/SEMATECH normal-distribution reference. Standard-error terminology and sampling-design boundaries are cross-checked with the NIST glossary and NIST sampling guidance. The interview emphasis on assumptions, reasoning, and decision communication is calibrated with the Chan Zuckerberg Initiative 2025 guide, one employer example rather than a universal loop.