Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 28 additions & 27 deletions lectures/Bayesian Machine Learning.jl
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
### A Pluto.jl notebook ###
# v0.20.13
# v0.20.15

#> [frontmatter]
#> image = "https://github.com/bmlip/course/blob/v2/assets/figures/scientific-inquiry-loop-w-BML-eqs.png?raw=true"
Expand Down Expand Up @@ -31,13 +31,10 @@ using Distributions, StatsPlots, SpecialFunctions
using Plots, LaTeXStrings, Plots.PlotMeasures

# ╔═╡ caba8eee-dfea-45bc-a8a7-1dd20a1fa994
using PlutoUI, PlutoTeachingTools
using BmlipTeachingTools

# ╔═╡ 6a23b828-d294-11ef-371a-05d061144a43
md"""
# Bayesian Machine Learning

"""
title("Bayesian Machine Learning")

# ╔═╡ 6be2e966-4048-44d0-a37e-95060e3fe30b
PlutoUI.TableOfContents()
Expand Down Expand Up @@ -67,7 +64,7 @@ md"""

# ╔═╡ 6a24376c-d294-11ef-348a-e9027bd0ec29
md"""
$(section_outline("Challenge:", "Predicting a Coin Toss"))
$(challenge_statement("Predicting a Coin Toss"))

##### Problem

Expand Down Expand Up @@ -258,7 +255,7 @@ Then, solve the desired inference problem for the posterior over the model ``m_k
"""

# ╔═╡ 74fa1925-0d9f-47f6-a6bd-b822948a4fbc
details("Proof this yourself, and click for solution",
details("Prove this yourself, and click for solution",
md"""
```math
\begin{align}
Expand Down Expand Up @@ -348,7 +345,7 @@ As an aside, in the (statistics and machine learning) literature, performance co
"""

# ╔═╡ 99db44c9-185c-4f39-ae5e-1a4cd751d980
details("Proof this yourself, and click for solution",
details("Prove this yourself, and click for solution",
md"""
```math
\begin{align*}
Expand Down Expand Up @@ -400,7 +397,7 @@ p(x|D) = \int \underbrace{p(x|\theta)}_{\substack{\text{data } \\ \text{generati
"""

# ╔═╡ f6ee5570-9b92-42b6-baf3-3eed5352a060
details("Proof this yourself, and click for solution",
details("Prove this yourself, and click for solution",
md"""
```math
\begin{align*}
Expand Down Expand Up @@ -518,7 +515,7 @@ md"""
"""

# ╔═╡ 6a9ad1c4-dfb2-4987-9ddc-da6131605083
details("Click for proof",
hide_proof(
md"""
```math
\begin{flalign}
Expand Down Expand Up @@ -739,7 +736,7 @@ where ``B(\alpha,\beta) \triangleq \frac{\Gamma(\alpha) \Gamma(\beta)}{\Gamma(\a
"""

# ╔═╡ b426df32-5629-4773-b862-101cfbd82d42
details("Proof this, and click for solution",
details("Prove this, and click for solution",
md"""
```math
\begin{flalign*}
Expand Down Expand Up @@ -873,7 +870,7 @@ md"""
"""

# ╔═╡ 90f691ad-046c-4595-99b0-19a1d6cb599e
details("Proof this yourself, and click for solution",
details("Prove this yourself, and click for solution",
md"""
```math
\begin{align*}
Expand Down Expand Up @@ -902,7 +899,7 @@ For large ``N``, the gain goes to ``1`` and ``\left. p(x_\bullet=1|D)\right|_{N\
"""

# ╔═╡ 6a2abb16-d294-11ef-0243-d376e8a39bb0
section_outline("Code Example:", "Bayesian Evolution for the Coin Toss")
code_example("Bayesian Evolution for the Coin Toss")

# ╔═╡ 6a2acb7e-d294-11ef-185c-9d49ce79c31b
md"""
Expand Down Expand Up @@ -1249,15 +1246,15 @@ $(section_outline("Excercises","",header_level=1))
\,\mathrm{d}{\theta}
```

Proof that the Bayes estimate minimizes the mean-squared error, i.e., proof that
Prove that the Bayes estimate minimizes the mean-squared error, i.e., Prove that

```math
\hat \theta_{bayes} = \arg\min_{\hat \theta} \int_\theta (\hat \theta -\theta)^2 p \left( \theta |D \right) \,\mathrm{d}{\theta}
```
"""

# ╔═╡ 7dd9a456-9dca-47c8-98c5-51f87f28e6a4
details("Click for solution",
hide_solution(
md"""
To minimize the expected mean-squared error we will look for ``\hat{\theta}`` that makes the gradient of the integral with respect to ``\hat{\theta}`` vanish.

Expand Down Expand Up @@ -1296,7 +1293,7 @@ and a given data set ``D=\{x_1, x_2,\ldots,x_N\}``.
"""

# ╔═╡ 664d4183-edb6-4818-a44b-bf4c0a22a33c
details("Click for solution",
hide_solution(
md"""
- (a) The likelihood is given by ``p(D|\mu) = \mu^n\cdot (1-\mu)^{(N-n)}``. It follows that

Expand Down Expand Up @@ -1387,7 +1384,7 @@ Now assume that the model priors are given by
"""

# ╔═╡ de08c2a1-c5e3-4add-8b22-2c633247da48
details("Click for solutions",
hide_solution(
md"""
- (a) Work out the probability ``p(x=1|m_1)``.

Expand Down Expand Up @@ -1581,20 +1578,18 @@ wideq(x) = PlutoUI.ExperimentalLayout.Div([x]; style="min-width: max-content;")
# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
BmlipTeachingTools = "656a7065-6f73-6c65-7465-6e646e617262"
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
PlutoTeachingTools = "661c6b06-c737-4d37-b85c-46df65de6f69"
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
SpecialFunctions = "276daf66-3868-5448-9aa4-cd146d93841b"
StatsPlots = "f3b207a7-027a-5e70-b257-86293d7955fd"

[compat]
BmlipTeachingTools = "~1.1.0"
Distributions = "~0.25.119"
LaTeXStrings = "~1.4.0"
Plots = "~1.40.13"
PlutoTeachingTools = "~0.4.5"
PlutoUI = "~0.7.69"
SpecialFunctions = "~2.5.1"
StatsPlots = "~0.15.7"
"""
Expand All @@ -1603,9 +1598,9 @@ StatsPlots = "~0.15.7"
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.11.4"
julia_version = "1.11.6"
manifest_format = "2.0"
project_hash = "abd58c9f6700767f652d81b5f75ddf0c01e33a11"
project_hash = "c32aa4de72bf9cb3845a24e6c53ef8abfcfaabe9"

[[deps.AbstractFFTs]]
deps = ["LinearAlgebra"]
Expand Down Expand Up @@ -1676,6 +1671,12 @@ git-tree-sha1 = "0691e34b3bb8be9307330f88d1a3c3f25466c24d"
uuid = "d1d4a3ce-64b1-5f1a-9ba4-7e7e69966f35"
version = "0.1.9"

[[deps.BmlipTeachingTools]]
deps = ["HypertextLiteral", "InteractiveUtils", "Markdown", "PlutoTeachingTools", "PlutoUI", "Reexport"]
git-tree-sha1 = "17747c9318a7e81cd8ca4ee3d414d96e7d8bba3e"
uuid = "656a7065-6f73-6c65-7465-6e646e617262"
version = "1.1.0"

[[deps.Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl"]
git-tree-sha1 = "1b96ea4a01afe0ea4090c5c8039690672dd13f2e"
Expand Down Expand Up @@ -2317,7 +2318,7 @@ version = "0.3.27+1"
[[deps.OpenLibm_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "05823500-19ac-5b8b-9628-191a04bc5112"
version = "0.8.1+4"
version = "0.8.5+0"

[[deps.OpenSSL]]
deps = ["BitFlags", "Dates", "MozillaCACerts_jll", "OpenSSL_jll", "Sockets"]
Expand Down Expand Up @@ -2426,9 +2427,9 @@ version = "0.4.5"

[[deps.PlutoUI]]
deps = ["AbstractPlutoDingetjes", "Base64", "ColorTypes", "Dates", "Downloads", "FixedPointNumbers", "Hyperscript", "HypertextLiteral", "IOCapture", "InteractiveUtils", "JSON", "Logging", "MIMEs", "Markdown", "Random", "Reexport", "URIs", "UUIDs"]
git-tree-sha1 = "2d7662f95eafd3b6c346acdbfc11a762a2256375"
git-tree-sha1 = "fcfec547342405c7a8529ea896f98c0ffcc4931d"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.69"
version = "0.7.70"

[[deps.PrecompileTools]]
deps = ["Preferences"]
Expand Down
Loading
Loading