Skip to content

Rewrite MLE as an elementary lecture on maximum likelihood - #849

Merged
jstac merged 5 commits into
mainfrom
mle-sequel
Sep 29, 2026
Merged

jstac merged 5 commits into
mainfrom
mle-sequel

Conversation

@jstac

@jstac jstac commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Rewrites mle as an elementary introduction to maximum likelihood. The wealth tax application moves to a separate lecture, wealth_tax, in a follow-up PR.

Contents

  1. Overview: what MLE is, its role in statistics and science, and some history (Bernoulli, Gauss, Fisher; citing Fisher 1922, Aldrich 1997, Stigler 2007).
  2. The likelihood function: a slow first example with loan defaults (Bernoulli θ, linking to bayes_intro). It covers:
    • the likelihood as a product;
    • a plot and a grid search;
    • the log likelihood and a full calculus derivation;
    • the distinction between likelihood and probability;
    • likelihood curves for increasing n.
  3. More examples:
    • Poisson, on the EPL goals data.
    • Normal, on women's heights, with a contour plot of ℓ(μ, σ). Includes the divisor n versus n − 1, and the fact that OLS is MLE under normal errors.
    • Pareto with known minimum.
  4. Consistency: simulated estimates of the Pareto tail index concentrating on the truth as n grows.
  5. Maximum likelihood and the method of moments: for the Pareto, the method of moments is invalid for α ≤ 1 (the formula still returns a value above 1) and unstable when the variance is infinite. MLE handles both cases.
  6. Numerical maximum likelihood:
    • Gamma fitted to Ames house prices with scipy.optimize.minimize, checked against gamma.fit.
    • Why we work with logs (underflow).
    • A note on misspecification.
  7. Heavy-tailed returns: Student's t fitted to Amazon returns by MLE (ν ≈ 3.6). This makes the existing forward reference in fitting_distributions accurate.
  8. Exercises: exponential MLE, Poisson consistency, uniform on [0, θ], bias of σ̂.

Also:

  • Fixes bugs in the old lecture: the lognormal density, and the sign of ∂ℓ/∂μ.
  • Adds bibliography entries for Fisher (1922), Aldrich (1997) and Stigler (2007).

Builds locally with no warnings.

🤖 Generated with Claude Code

- Compare the lognormal MLE with the method-of-moments fit, using the KS statistic
- Replace the by-eye tail comparison with log-scale Q-Q plots and KS
- Show why the method of moments fails for a Pareto tail with b <= 1
- Add a Student's t MLE fit to Amazon returns, as promised in fitting_distributions
- Fix the lognormal density, the sign of the derivative with respect to mu, and
  use \partial and aligned
- Add figure captions, axis labels and lw=2 per the style guide

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@netlify

netlify Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for taupe-gaufre-c4e660 ready!

Name Link
🔨 Latest commit b9715a8
🔍 Latest deploy log https://app.netlify.com/projects/taupe-gaufre-c4e660/deploys/6abc28ac109b6900086b2c05
😎 Deploy Preview https://deploy-preview-849--taupe-gaufre-c4e660.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

@github-actions
github-actions Bot temporarily deployed to pull request September 28, 2026 21:48 Inactive
@github-actions
github-actions Bot temporarily deployed to pull request September 28, 2026 21:50 Inactive
jstac and others added 2 commits September 29, 2026 10:46
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Build up the method through Bernoulli (loan defaults), Poisson, normal and
Pareto examples; add consistency and asymptotic normality simulations, a
comparison with the method of moments for Pareto tails, numerical MLE for
the gamma distribution, and keep the Student's t fit to stock returns.
The wealth tax application moves to a separate lecture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pull request September 29, 2026 00:57 Inactive
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pull request September 29, 2026 01:06 Inactive
@github-actions
github-actions Bot temporarily deployed to pull request September 29, 2026 01:07 Inactive
- Go slowly through the Bernoulli example: likelihood as a product, plot,
  then the log likelihood derivation; use a seed where the estimate differs
  from the truth
- Move the underflow point to the numerical MLE section, using house prices
- Identify the normal MLEs as sample mean and standard deviation
- Drop the asymptotic normality discussion
- Reframe the method of moments comparison: invalid for alpha <= 1,
  unstable for heavy tails

Co-Authored-By: John Stachurski <john.stachurski@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jstac jstac changed the title MLE lecture: make it a sequel to Fitting Distributions Rewrite MLE as an elementary lecture on maximum likelihood Sep 29, 2026
@github-actions
github-actions Bot temporarily deployed to pull request September 29, 2026 21:15 Inactive
@jstac
jstac merged commit 901462c into main Sep 29, 2026
8 checks passed
@jstac
jstac deleted the mle-sequel branch September 29, 2026 21:17
mmcky added a commit to QuantEcon/data-lectures that referenced this pull request Sep 30, 2026
QuantEcon/lecture-python-intro#849 (merged 2026-09-29 21:17 UTC)
rewrote mle.md to download monthly AMZN prices with yfinance and refit
by maximum likelihood the Student's t that fitting_distributions fits
by moments. The read has no annotation, so the strict audit fails on
main with `missing_api_annotations: lecture-python-intro:mle:yfinance`,
and the next push to main or the Monday 05:17 UTC run goes red, opens
an audit-drift issue and skips the Pages deploy.

The entry follows fitting_distributions', which reads the same series
the same way: pedagogy incidental. Checked 2026-09-30 with
build_audit.py all --strict over the eight lecture repos at main: exit
1 on main alone, exit 0 with this entry.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
pull request — b9715a8f Deployed Sep 29, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant