A regression is an explanation with assumptions

Learning goals

• Estimate a line by least squares and interpret the slope, intercept and residuals.
• Evaluate fit, inference, forecasts and market beta without asserting causation.

Draft · approximately 60 minutes, excluding further revision.

Fit the line and inspect what it misses

For yᵢ=a+bxᵢ+εᵢ, ordinary least squares minimizes squared vertical residuals. The slope is Σ(xᵢ−x̄)(yᵢ−ȳ)/Σ(xᵢ−x̄)² and intercept is ȳ−bx̄. The slope measures the model’s change in predicted y for one unit of x; the intercept may have no useful economic interpretation if x=0 is outside the data. Units and transformations are part of the model.

For usual finite-sample t inference, examine a linear conditional mean, independent errors, zero conditional error mean, constant error variance and appropriate distributional conditions. Residual curvature suggests a missed functional form; a fan shape suggests changing variance; patterns over time suggest serial dependence. A high R² does not diagnose these violations or prove causality.

Separate fit, significance and prediction uncertainty

With an intercept, SST = SSR + SSE and R²=1−SSE/SST. Residual standard error is √[SSE/(n−2)]. In simple regression, the slope standard error is this residual error divided by √Sxx; t=(b−b₀)/SE(b). The one-regressor ANOVA F statistic is SSR/[SSE/(n−2)], equal to t² when testing zero slope under the same assumptions.

A new-observation prediction interval is ŷ₀ ± t*sₑ√[1+1/n+(x₀−x̄)²/Sxx]. The mean-response interval omits the first 1 and is narrower. Extrapolation can be risky even when the formula yields a narrow interval. In log-level models b approximates a proportional y change per unit x; in log-log models it is elasticity. For market-model returns, beta is Cov(Ri,Rm)/Var(Rm); a regression intercept is not automatically a forecast of sustainable skill. CAPM’s required return rf+β(E[Rm]−rf) uses an expected premium, not a realized sample return promised to recur.

Read a regression in the units in which it was fitted

Suppose excess return y and market excess return x are both recorded in percentage points. A fitted line is y=0.20+1.10 x. At x=2, predicted y=2.40 percentage points; observed y=1.80 gives residual−0.60 percentage point. Beta 1.10 is a slope, not a forecast of 10% return. An intercept 0.20 in monthly data is monthly and should not be labelled annual alpha without a stated conversion.

With a supplied slope standard error 0.25, testing zero slope gives t=1.10/0.25=4.4. A causal interpretation still needs identification beyond association. Inspect residuals against fitted values and time: a fan shape suggests changing variance; a pattern over time suggests dependence. An observation far outside the training x range calls for an extrapolation warning, even if the in-sample R² is high.

Formula and assumptions

b = Σ(x−x̄)(y−ȳ)/Σ(x−x̄)²; a = ȳ−bx̄; R² = 1−SSE/SST; β = Cov(Ri,Rm)/Var(Rm)

A five-point line with visible residuals

1. Use x = 0,1,2,3,4 and y = 1,2,2,4,6. Means are x̄=2 and ȳ=3. Sxx=10, cross-deviation sum=12, so b=1.2 and a=0.6.
2. Predictions are 0.6,1.8,3.0,4.2,5.4. SSE=1.6 and SST=16, so R²=0.90 and sₑ=√(1.6/3)=0.7303.
3. At x₀=3, ŷ=4.2 and prediction SE=√[(1.6/3)×(1+1/5+1/10)]=0.8327. Using supplied t*=3.182 gives an interval approximately [1.5505,6.8495]. These five points illustrate arithmetic, not sufficient evidence for a tradable model.

Calculation lab

Errors to catch

• Reading a high R² as proof of causality or out-of-sample skill.
• Using a mean-response interval as a new-observation prediction interval.

Original knowledge check

Answer every question before checking. Review explanations after submitting; retry starts a new attempt.

Sources and coverage status

These original lessons are drafts. Public outlines are used to check scope; the official learning-outcome text and proprietary question banks are not reproduced here. Follow the source for exam-year requirements and updates.

• Reference · www.cfainstitute.org
• Reference · itl.nist.gov