Fit is not forecast reliability

Learning goals

• Calculate the unadjusted R².
• Explain why adjusted R² applies to this case.
• Identify the case-specific assumption or failure mode before taking action.

Draft · approximately 50 minutes, excluding further revision.

Shared case and decision context

A two-regressor model with an intercept uses 50 observations. SST=100 and SSE=40. A proposed third regressor slightly raises R²; no holdout test has been run.

Economic or professional mechanism

R² compares unexplained variation with total variation in the estimation sample. Adjusted R² penalizes added parameters through residual degrees of freedom. A joint F-test asks whether a set of slopes is zero under its null; separate t-tests do not answer exactly the same question when regressors are correlated.

Apply, challenge and communicate

Here residual degrees of freedom are 47. R² can rise mechanically when an extra regressor is fitted, while adjusted R² may fall. Compare economically defensible models using consistent samples and forecast loss on a truly later sample rather than selecting solely by in-sample fit.

Compute a conditional prediction

A fitted model y=1+2x1−.5x2 gives11 when x1=6 and x2=4. The prediction is conditional on the regressor scenario; it is not an observed realization. A prediction interval for a new observation includes innovation uncertainty as well as uncertainty about the fitted conditional mean. Extrapolation beyond the estimation range can make even a narrow reported interval unreliable.

Mean response and a new observation are different uncertainties

At the same regressor values and confidence level, an interval for a new observation is wider than an interval for the conditional mean because it includes the new observation’s irreducible error as well as coefficient-estimation uncertainty. Neither is the confidence interval for a single coefficient. A model can estimate the conditional average precisely while individual investment outcomes remain dispersed.

For an illustrative large-sample normal approximation, the fitted mean is 5, its estimation standard error is 0.4, and independent new-observation residual standard deviation is 2.0. Using a specified multiplier 1.96, the mean interval is 5±0.784; the new-observation interval is 5±1.96√(0.4²+2²), or approximately [1.002,8.998]. In a real small-sample regression use the appropriate t multiplier and leverage-specific forecast variance; extrapolation or regime change can invalidate either interval.

Formula and assumptions

R²=1−SSE/SST; adjusted R²=1−(SSE/(n−k−1))/(SST/(n−1))

Case analysis with explicit assumptions

1. A two-regressor model with an intercept uses 50 observations. SST=100 and SSE=40. A proposed third regressor slightly raises R²; no holdout test has been run.
2. Method: R²=1−SSE/SST; adjusted R²=1−(SSE/(n−k−1))/(SST/(n−1))
3. Case result: 0.60. Here residual degrees of freedom are 47. R² can rise mechanically when an extra regressor is fitted, while adjusted R² may fall. Compare economically defensible models using consistent samples and forecast loss on a truly later sample rather than selecting solely by in-sample fit.

Errors to catch

• Do not replace the stated assumptions with an unstated shortcut. Reconcile units, timing and the claim being valued or duty being assessed.
• Guaranteed holdout improvement is not a defensible conclusion for these case facts.

Original knowledge check

Answer every question before checking. Review explanations after submitting; retry starts a new attempt.

Sources and coverage status

These original lessons are drafts. Public outlines are used to check scope; the official learning-outcome text and proprietary question banks are not reproduced here. Follow the source for exam-year requirements and updates.

• Reference · www.cfainstitute.org
• Reference · www.itl.nist.gov
• Reference · itl.nist.gov