01_Simple_Linear_Model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #01\n",
    "# Simple Linear Model\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This tutorial demonstrates the basic workflow of using TensorFlow with a simple linear model. After loading the so-called MNIST data-set with images of hand-written digits, we define and optimize a simple mathematical model in TensorFlow. The results are then plotted and discussed.\n",
    "\n",
    "You should be familiar with basic linear algebra, Python and the Jupyter Notebook editor. It also helps if you have a basic understanding of Machine Learning and classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-gpu/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "from sklearn.metrics import confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This was developed using Python 3.6 (Anaconda) and TensorFlow version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.9.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set is about 12 MB and will be downloaded automatically if it is not located in the given path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mnist import MNIST\n",
    "data = MNIST(data_dir=\"data/MNIST/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST data-set has now been loaded and consists of 70.000 images and class-numbers for the images. The data-set is split into 3 mutually exclusive sub-sets. We will only use the training and test-sets in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n",
      "- Test-set:\t\t10000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(data.num_train))\n",
    "print(\"- Validation-set:\\t{}\".format(data.num_val))\n",
    "print(\"- Test-set:\\t\\t{}\".format(data.num_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copy some of the data-dimensions for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = data.img_size_flat\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = data.img_shape\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = data.num_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One-Hot Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output-data is loaded as both integer class-numbers and so-called One-Hot encoded arrays. This means the class-numbers have been converted from a single integer to a vector whose length equals the number of possible classes. All elements of the vector are zero except for the $i$'th element which is 1 and means the class is $i$. For example, the One-Hot encoded labels for the first 5 images in the test-set are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],\n",
       "       [0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],\n",
       "       [0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],\n",
       "       [1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0., 1., 0., 0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.y_test[0:5, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need the classes as integers for various comparisons and performance measures. These can be found from the One-Hot encoded arrays by taking the index of the highest element using the `np.argmax()` function. But this has already been done for us when the data-set was loaded, so we can see the class-number for the first five images in the test-set. Compare these to the One-Hot encoded arrays above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.y_test_cls[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function for plotting images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function used to plot 9 images in a 3x3 grid, and writing the true and predicted classes below each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot a few images to see if data is correct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4EGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6qeqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfeeAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMydOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABOOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgwILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4jnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cCJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVAMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBjw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDuvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUmiYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6SH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK09/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+veeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRekpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXUUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8E4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+ewD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bstNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2QNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32exx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrWfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23foXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJxU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDmlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2OOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pmR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8VeQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLyewIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQXN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBnaPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfMrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/VMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoMRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAANGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++fQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwIwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXrgS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmADRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohSSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYiIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVqe+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCLFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9NbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2eOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+M/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4f+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBYvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6USzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDKUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQdNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VHHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9VIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdffHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7ZjtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFBlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23KDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltuSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9XwvgMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpDERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGICBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90CX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CRzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1zblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAmoW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38waAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlRGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZSktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogAqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcYlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxTTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0aAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp33XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9jooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9FgjuEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD84he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneABjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBDZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjkkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBEDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcffzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2pfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfEY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUXAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQWu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0bAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPHjBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDwusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkDBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwagVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+eOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQRoR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5EgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5fgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKOZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4BoLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5nh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0bN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcAsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjsM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/mnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2ydszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhRebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cyaOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543szecs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiwug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoELgOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/SuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5lRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd2aa71fb38>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.x_test[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.y_test_cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow Graph\n",
    "\n",
    "The entire purpose of TensorFlow is to have a so-called computational graph that can be executed much more efficiently than if the same calculations were to be performed directly in Python. TensorFlow can be more efficient than NumPy because TensorFlow knows the entire computation graph that must be executed, while NumPy only knows the computation of a single mathematical operation at a time.\n",
    "\n",
    "TensorFlow can also automatically calculate the gradients that are needed to optimize the variables of the graph so as to make the model perform better. This is because the graph is a combination of simple mathematical expressions so the gradient of the entire graph can be calculated using the chain-rule for derivatives.\n",
    "\n",
    "TensorFlow can also take advantage of multi-core CPUs as well as GPUs - and Google has even built special chips just for TensorFlow which are called TPUs (Tensor Processing Units) that are even faster than GPUs.\n",
    "\n",
    "A TensorFlow graph consists of the following parts which will be detailed below:\n",
    "\n",
    "* Placeholder variables used to feed input into the graph.\n",
    "* Model variables that are going to be optimized so as to make the model perform better.\n",
    "* The model which is essentially just a mathematical function that calculates some output given the input in the placeholder variables and the model variables.\n",
    "* A cost measure that can be used to guide the optimization of the variables.\n",
    "* An optimization method which updates the variables of the model.\n",
    "\n",
    "In addition, the TensorFlow graph may also contain various debugging statements e.g. for logging data to be displayed using TensorBoard, which is not covered in this tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Placeholder variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Placeholder variables serve as the input to the graph that we may change each time we execute the graph. We call this feeding the placeholder variables and it is demonstrated further below.\n",
    "\n",
    "First we define the placeholder variable for the input images. This allows us to change the images that are input to the TensorFlow graph. This is a so-called tensor, which just means that it is a multi-dimensional vector or matrix. The data-type is set to `float32` and the shape is set to `[None, img_size_flat]`, where `None` means that the tensor may hold an arbitrary number of images with each image being a vector of length `img_size_flat`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, [None, img_size_flat])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we have the placeholder variable for the true labels associated with the images that were input in the placeholder variable `x`. The shape of this placeholder variable is `[None, num_classes]` which means it may hold an arbitrary number of labels and each label is a vector of length `num_classes` which is 10 in this case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true = tf.placeholder(tf.float32, [None, num_classes])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally we have the placeholder variable for the true class of each image in the placeholder variable `x`. These are integers and the dimensionality of this placeholder variable is set to `[None]` which means the placeholder variable is a one-dimensional vector of arbitrary length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_true_cls = tf.placeholder(tf.int64, [None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variables to be optimized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from the placeholder variables that were defined above and which serve as feeding input data into the model, there are also some model variables that must be changed by TensorFlow so as to make the model perform better on the training data.\n",
    "\n",
    "The first variable that must be optimized is called `weights` and is defined here as a TensorFlow variable that must be initialized with zeros and whose shape is `[img_size_flat, num_classes]`, so it is a 2-dimensional tensor (or matrix) with `img_size_flat` rows and `num_classes` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights = tf.Variable(tf.zeros([img_size_flat, num_classes]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second variable that must be optimized is called `biases` and is defined as a 1-dimensional tensor (or vector) of length `num_classes`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "biases = tf.Variable(tf.zeros([num_classes]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This simple mathematical model multiplies the images in the placeholder variable `x` with the `weights` and then adds the `biases`.\n",
    "\n",
    "The result is a matrix of shape `[num_images, num_classes]` because `x` has shape `[num_images, img_size_flat]` and `weights` has shape `[img_size_flat, num_classes]`, so the multiplication of those two matrices is a matrix with shape `[num_images, num_classes]` and then the `biases` vector is added to each row of that matrix.\n",
    "\n",
    "Note that the name `logits` is typical TensorFlow terminology, but other people may call the variable something else."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "logits = tf.matmul(x, weights) + biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now `logits` is a matrix with `num_images` rows and `num_classes` columns, where the element of the $i$'th row and $j$'th column is an estimate of how likely the $i$'th input image is to be of the $j$'th class.\n",
    "\n",
    "However, these estimates are a bit rough and difficult to interpret because the numbers may be very small or large, so we want to normalize them so that each row of the `logits` matrix sums to one, and each element is limited between zero and one. This is calculated using the so-called softmax function and the result is stored in `y_pred`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = tf.nn.softmax(logits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predicted class can be calculated from the `y_pred` matrix by taking the index of the largest element in each row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_cls = tf.argmax(y_pred, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cost-function to be optimized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make the model better at classifying the input images, we must somehow change the variables for `weights` and `biases`. To do this we first need to know how well the model currently performs by comparing the predicted output of the model `y_pred` to the desired output `y_true`.\n",
    "\n",
    "The cross-entropy is a performance measure used in classification. The cross-entropy is a continuous function that is always positive and if the predicted output of the model exactly matches the desired output then the cross-entropy equals zero. The goal of optimization is therefore to minimize the cross-entropy so it gets as close to zero as possible by changing the `weights` and `biases` of the model.\n",
    "\n",
    "TensorFlow has a built-in function for calculating the cross-entropy. Note that it uses the values of the `logits` because it also calculates the softmax internally."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(logits=logits,\n",
    "                                                           labels=y_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have now calculated the cross-entropy for each of the image classifications so we have a measure of how well the model performs on each image individually. But in order to use the cross-entropy to guide the optimization of the model's variables we need a single scalar value, so we simply take the average of the cross-entropy for all the image classifications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "cost = tf.reduce_mean(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimization method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have a cost measure that must be minimized, we can then create an optimizer. In this case it is the basic form of Gradient Descent where the step-size is set to 0.5.\n",
    "\n",
    "Note that optimization is not performed at this point. In fact, nothing is calculated at all, we just add the optimizer-object to the TensorFlow graph for later execution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.5).minimize(cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Performance measures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need a few more performance measures to display the progress to the user.\n",
    "\n",
    "This is a vector of booleans whether the predicted class equals the true class of each image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "correct_prediction = tf.equal(y_pred_cls, y_true_cls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This calculates the classification accuracy by first type-casting the vector of booleans to floats, so that False becomes 0 and True becomes 1, and then calculating the average of these numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TensorFlow Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create TensorFlow session\n",
    "\n",
    "Once the TensorFlow graph has been created, we have to create a TensorFlow session which is used to execute the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize variables\n",
    "\n",
    "The variables for `weights` and `biases` must be initialized before we start optimizing them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.run(tf.global_variables_initializer())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function to perform optimization iterations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 55.000 images in the training-set. It takes a long time to calculate the gradient of the model using all these images. We therefore use Stochastic Gradient Descent which only uses a small batch of images in each iteration of the optimizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "batch_size = 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for performing a number of optimization iterations so as to gradually improve the `weights` and `biases` of the model. In each iteration, a new batch of data is selected from the training-set and then TensorFlow executes the optimizer using those training samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def optimize(num_iterations):\n",
    "    for i in range(num_iterations):\n",
    "        # Get a batch of training examples.\n",
    "        # x_batch now holds a batch of images and\n",
    "        # y_true_batch are the true labels for those images.\n",
    "        x_batch, y_true_batch, _ = data.random_batch(batch_size=batch_size)\n",
    "        \n",
    "        # Put the batch into a dict with the proper names\n",
    "        # for placeholder variables in the TensorFlow graph.\n",
    "        # Note that the placeholder for y_true_cls is not set\n",
    "        # because it is not used during training.\n",
    "        feed_dict_train = {x: x_batch,\n",
    "                           y_true: y_true_batch}\n",
    "\n",
    "        # Run the optimizer using this batch of training data.\n",
    "        # TensorFlow assigns the variables in feed_dict_train\n",
    "        # to the placeholder variables and then runs the optimizer.\n",
    "        session.run(optimizer, feed_dict=feed_dict_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-functions to show performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dict with the test-set data to be used as input to the TensorFlow graph. Note that we must use the correct names for the placeholder variables in the TensorFlow graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "feed_dict_test = {x: data.x_test,\n",
    "                  y_true: data.y_test,\n",
    "                  y_true_cls: data.y_test_cls}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for printing the classification accuracy on the test-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_accuracy():\n",
    "    # Use TensorFlow to compute the accuracy.\n",
    "    acc = session.run(accuracy, feed_dict=feed_dict_test)\n",
    "    \n",
    "    # Print the accuracy.\n",
    "    print(\"Accuracy on test-set: {0:.1%}\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for printing and plotting the confusion matrix using scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_confusion_matrix():\n",
    "    # Get the true classifications for the test-set.\n",
    "    cls_true = data.y_test_cls\n",
    "    \n",
    "    # Get the predicted classifications for the test-set.\n",
    "    cls_pred = session.run(y_pred_cls, feed_dict=feed_dict_test)\n",
    "\n",
    "    # Get the confusion matrix using sklearn.\n",
    "    cm = confusion_matrix(y_true=cls_true,\n",
    "                          y_pred=cls_pred)\n",
    "\n",
    "    # Print the confusion matrix as text.\n",
    "    print(cm)\n",
    "\n",
    "    # Plot the confusion matrix as an image.\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)\n",
    "\n",
    "    # Make various adjustments to the plot.\n",
    "    plt.tight_layout()\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(num_classes)\n",
    "    plt.xticks(tick_marks, range(num_classes))\n",
    "    plt.yticks(tick_marks, range(num_classes))\n",
    "    plt.xlabel('Predicted')\n",
    "    plt.ylabel('True')\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for plotting examples of images from the test-set that have been mis-classified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_example_errors():\n",
    "    # Use TensorFlow to get a list of boolean values\n",
    "    # whether each test-image has been correctly classified,\n",
    "    # and a list for the predicted class of each image.\n",
    "    correct, cls_pred = session.run([correct_prediction, y_pred_cls],\n",
    "                                    feed_dict=feed_dict_test)\n",
    "\n",
    "    # Negate the boolean array.\n",
    "    incorrect = (correct == False)\n",
    "    \n",
    "    # Get the images from the test-set that have been\n",
    "    # incorrectly classified.\n",
    "    images = data.x_test[incorrect]\n",
    "    \n",
    "    # Get the predicted classes for those images.\n",
    "    cls_pred = cls_pred[incorrect]\n",
    "\n",
    "    # Get the true classes for those images.\n",
    "    cls_true = data.y_test_cls[incorrect]\n",
    "    \n",
    "    # Plot the first 9 images.\n",
    "    plot_images(images=images[0:9],\n",
    "                cls_true=cls_true[0:9],\n",
    "                cls_pred=cls_pred[0:9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Helper-function to plot the model weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Function for plotting the `weights` of the model. 10 images are plotted, one for each digit that the model is trained to recognize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_weights():\n",
    "    # Get the values for the weights from the TensorFlow variable.\n",
    "    w = session.run(weights)\n",
    "    \n",
    "    # Get the lowest and highest values for the weights.\n",
    "    # This is used to correct the colour intensity across\n",
    "    # the images so they can be compared with each other.\n",
    "    w_min = np.min(w)\n",
    "    w_max = np.max(w)\n",
    "\n",
    "    # Create figure with 3x4 sub-plots,\n",
    "    # where the last 2 sub-plots are unused.\n",
    "    fig, axes = plt.subplots(3, 4)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Only use the weights for the first 10 sub-plots.\n",
    "        if i<10:\n",
    "            # Get the weights for the i'th digit and reshape it.\n",
    "            # Note that w.shape == (img_size_flat, 10)\n",
    "            image = w[:, i].reshape(img_shape)\n",
    "\n",
    "            # Set the label for the sub-plot.\n",
    "            ax.set_xlabel(\"Weights: {0}\".format(i))\n",
    "\n",
    "            # Plot the image.\n",
    "            ax.imshow(image, vmin=w_min, vmax=w_max, cmap='seismic')\n",
    "\n",
    "        # Remove ticks from each sub-plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "        \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance before any optimization\n",
    "\n",
    "The accuracy on the test-set is 9.8%. This is because the model has only been initialized and not optimized at all, so it always predicts that the image shows a zero digit, as demonstrated in the plot below, and it turns out that 9.8% of the images in the test-set happens to be zero digits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 9.8%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu8VXP+x/HXp0IpRUqS6swoKSEjt4RcalzLvYw7MdTP/TYMjUsMSRmXEXoIk2uJLi5R5FahppTKpShqJl0Zkojv74+9vnuvfc6ps9fZ9937+Xj0OGuvtfZaH75nfc9nfdd3fb/mnENERFJTI98BiIgUE1WaIiIRqNIUEYlAlaaISASqNEVEIlClKSISgSpNEZEIVGmKiESgSlNEJIJa6Xy5UaNGrqysLEOhFIfp06evcM41znccuaIyLn0q42jSqjTLysqYNm1aOocoOma2KN8x5JLKuPSpjKPR7bmISASqNEVEIlClKSISgSpNEZEIVGmKiESQ1tNzkeoaOHAgAGvXrgVg1qxZAIwcObLCvhdddBEA+++/PwBnnHFGLkIUqZQyTRGRCJRpSk717NkTgBEjRlS63cwqrBsyZAgAEyZMAODggw8GoEWLFtkIUfLos88+A6BNmzYA3HvvvQBcfPHFeYupPGWaIiIRKNOUrPPZJWw4w9xll10AOOKIIwD44osv4tvGjBkDwPz58wEYPnw4ANdff33mg5W8mjFjBgA1asTyuWbNmuUznEop0xQRiUCZpmSNf5/5hRdeqLCtffv2QCKLbNSoEQD16tUD4Oeff47vu++++wLw0UcfAbBy5cosRSz5NnPmTCDxe3DCCSfkM5xKKdMUEYkg55mm74f3yCOPALDDDjvEt9WuXRuA0047DYDtt98egFatWuUyRMmQ//73vwA45+LrfIY5fvx4AJo2bVrpd30/ToB58+YlbTvmmGMyGqfk3+zZswG47777ADjzzDPzGc5GKdMUEYkg55nm1VdfDcDChQs3uI/vl1e/fn0A2rVrl5FzN2/eHIBrrrkGgI4dO2bkuFK5Y489Fkg89QbYaqutAGjYsOFGv/vss8/Gl8Ptm1KaPv30UwDWrFkDJPe4KDTKNEVEIlClKSISQc5vz4cOHQokuo+Eb73nzp0LJDq4Tpo0CYCpU6cCidfmvvrqqw0ef7PNNgMSXVj8w4jwcfxtum7Pc6Nly5Yp73vXXXcBidfpwnzXI/9TSseAAQOA2NQbUNjXpjJNEZEIcp5pHnbYYUk/w/wrdN7q1auBRObp//p8+OGHGzz+FltsASRe+Pev5wGsWrUKgJ122qlasUv2jBs3DoB+/foBsG7duvi2Jk2aAHDHHXcAsOWWW+Y4OsmG8MNgf03767Zu3br5CCklyjRFRCIo6Ncot9lmGwAOPfTQpPWVZanlPf/880AiWwXYfffdAejVq1emQpQM8a9chjNMz3c/8UPCSWl46623Kqxr3Ljwp5tXpikiEkFBZ5rVsWzZMgD69OkDJL/C59vLqupYLblz3HHHAYnXKr2zzjorvty/f/+cxiS54ac4CfMvnhQyZZoiIhGUXKb5wAMPAImMc+utt45v80/mJP98/9nJkycDibZM36Z1ww03xPf1w4RJaZgyZQoAw4YNi6/bc889AejatWteYopCmaaISAQlk2m+++67QKIvnzd69Oj4sh+WTPLPDy67YsWKpPV+WED1pS1dEydOBJJ7tvg+2n54yEKmTFNEJAJVmiIiEZTM7fnLL78MJMZePPzwwwHYf//98xaTVOTnBPKvxnpdunQB4JZbbsl1SJJjfrCesJNPPjkPkVSPMk0RkQiKPtNcu3YtAK+++iqQGLDj5ptvBhJDxUn+hGePvP3224GKo7F36NABUPeiUrZ06VIA3nnnHSB5MJ3jjz8+LzFVhzJNEZEIij7T9IPW+jayI488EoBOnTrlLSZJdvfdd8eXP/jgg6Rt/jVKtWWWvsceewyAb775Bkhcq8VGmaaISARFmWn6AWsBbr31VgAaNGgAwI033piXmGTDBg0atMFt/rVXtWWWvkWLFiV99kM/FhtlmiIiERRVpumfwl5yySXxdevXrwfgqKOOAtQvs9j4Mk2ll4O/m/D7/vLLLwB89913Ffb1r+gNHjy40mPVrFkzvnznnXcCmkYj28aOHZv0+ZhjjslTJOlRpikiEoEqTRGRCIri9vzXX38FEiOhfPnll/FtrVq1AhIPhKS4+HmbUnHKKacA0LRpUyDRdeWZZ55JKwY/22V4DE/JHN+Z3ZdXsVOmKSISQVFkmgsWLAASMxaG+e4sGn+xcPmHdAAvvvhitY/z3HPPVbmPf0hUo0ZyPtC9e3cAOnbsWOE7nTt3rnZMUrUXXngBSDy09aO0F+vsoso0RUQiKOhM03eG7datW9L6gQMHxpeLtdvCpmTUqFHx5QEDBgAVB+zw5s6dC2y8nfK8884DoGXLlhW2nXjiiQC0bdu2esFKxvz4448AvPLKK0nr/TBw4W5fxUSZpohIBAWdaT700ENAxdevwm0hZpbTmCQ9qc5r/dRTT2U5Esk2377sZ4Tt0aMHAJdeemneYsoEZZoiIhEUZKbp+3Xdf//9eY5ERKrLZ5p+nvNSoUxTRCSCgsw0/Rzm33//fdJ6//aPhhETkXxRpikiEoEqTRGRCAry9rw8P1PhxIkTAWjYsGE+wxGRTZgyTRGRCAoy07zuuuuSfoqIFAplmiIiEZhzrvpfNlsOLKpyx9LS0jnXON9B5IrKuPSpjKNJq9IUEdnU6PZcRCQCVZoiIhFstNI0s23NbGbwb6mZLQl93jwbAZlZu9A5ZprZ92b2f1V8p7eZLQ/2n2dm56YZw3AzO66KfczM/mlm881slpl1SOec+ZKnMm5pZpPMbK6ZzamqfIPvqIyrKR9lHJz3cV9mKe5fHGXsnEvpH3ATcFUl6w2okepxovwDNgOWATtWsV9v4J5geXtgBdCo3D61Ipx3OHBcFft0B8YGy52B97Lx/yCX/3JVxsAOQIdguT6wANhZZVw6ZRwc82BgH2BmivsXRRlX6/bczFoFWcKTwByguZl9G9rey8yGBstNzGyUmU0zsw/MbL8Ip+oKzHPOLU71C865pcBCoIWZ9TezJ8zsPeAxM6tlZoOCOGaZWe8gxhrBX5tPzOx1oFEKp+oBPBGc811gezMrmSeu2Sxj59x/nHMzg+X/AZ8AzVKNTWWcGdm+jp1zbwGrqhNbIZdxOp3bdwHOdM5NM7ONHedeYIBzbqqZlQHjgPZmti9wjnPuwo18txfwdJSgzKwV0BL4IhTnQc65n8ysD7DMObePmW0BTDWz14D9gN8B7YhlQXOBIcHxbiP21+flcqdqBnwd+rw4WLc8SrwFLutlbGa/B9oDH6YalMo4o3JxHUdWyGWcTqW5wDlXcU7dig4H2lhiWoptzKyOc+594P0NfcnMagNHA1ekGM9pZtYFWAf0ds59G5xztHPup2CfbkBbM+sVfG4AtAYOAp52zv0GLDazSf6gzrm/pnj+UpTtMq4PPA9c7Jz7IYXzqIwzL6tlXA0FX8bpVJprQsu/EWsT8WqHlg3YxzlX+fSDG3Y08L5zbkWK+z/pnLuskvXhOA3o45ybGN7BzI6PGBvAEqA5MDX4vGOwrpRkrYwt9gBiFDDMOTcmxa+pjDMv29dxVAVfxhnpchTU7KvNrLWZ1QDCwU8A+voPKT2dijmVcrfmZnapmaVzGzAe6ONvQ8ysjZnVAd4GegZtIs2INWBXZQxwZnCczsA3zrlSum1Lkskytljq8BixBwT3ltumMs6TLF3HFRR7GWeyn+a1xP5jJhNrF/D6AgcEDbZzgfODAPc1syGVHcjMtgIOAV4st6ktsDKNGB8CPgdmmtnHwIPEsu2RwFfE2kCGAfFJTczsNjM7qpJjjQWWmNmC4Dh9K9mn1GSqjA8m9kexqyW6vvwx2KYyzq9MXscjgHeAdma22MzODjYVdRkX1WuUZvYS0MM5tz7fsUh2qIxLX7GXcVFVmiIi+abXKEVEIlClKSISgSpNEZEIVGmKiESQ1hxBjRo1cmVlZRkKpThMnz59hduERvVWGZc+lXE0aVWaZWVlTJuWyhtYpcPMNqlpAVTGpU9lHI1uz0VEIlClKSISgSpNEZEIVGmKiESgSlNEJAJVmiIiEajSFBGJQJWmiEgEaXVuz5Z///vfAJxwwgkALFy4sNrHeu211+LLbdu2BaB58+bVD07yZuzYsQB0794dgPvuuw+Aiy66KL5PzZo1cx+YJFm2bBkAp5xyCgCdOnUC4IILLgBinekz4bvvvgPg7bffBuCII44AYLPNNsvI8TdEmaaISAQFmWmOHz8egHXr1qV9rDFjEnN2PfroowA888wzaR9XcmflytjMCOGMEuDiiy8G4Lzzzouvq1OnTu4Ck7jVq1fHl3fddVcgkQk2adIEyHyG+Yc//AGAFSticy/6V0Fbt26dkfNsiDJNEZEICirTXL8+NmXIyy+Xn8+9+jp27BhfHjRoEABr1sRmA61bt27GziPZ49uslixJnln11FNPBaB27doVviO54bM8334JiTuDvn1jc5T5tudM6d+/PwBffvklAA8//DCQ/QzTU6YpIhJBQWWab775JgCTJ08G4Nprr037mKtWrYovz5kzB4Aff/wRUKZZyMLt2T6zKO+MM84AIDaNuuSD7+kyadKkCtv69euXsfN8/PHH8eWBAwcCcPzxsWnZe/bsmbHzpEKZpohIBKo0RUQiyPvt+ezZs+PLvXr1AqBVq1YAXH/99WkfP9zlSIrHrFmz4sv+FtCrVSv2a3vkkUfmNCZJ8B3Yn3/++QrbfNe+xo3TnzHE35Z37dq1wjb/8stWW22V9nmiUKYpIhJB3jPN2267Lb7sH9AMHz4cgHr16lX7uP4B0FtvvRVfpwcGxWPUqFEb3FZZ1iG5deWVVwKJa9V3NAc4+eSTM3aed999F4ClS5fG151zzjkAnH766Rk7TxTKNEVEIshbpjly5EgguSO7b8vce++90z6+76YSzi67dOkCwNZbb5328SW7wncI3uabbw7A7bffnutwpBx/XfmfzZo1i2/z5VQda9euBRJl/MADDySdBxJtpvmiTFNEJIK8ZZojRowAEq80QsUBGarDDyP31FNPAYknrQA33HADkP2ho6T6/IsNU6ZMqbBtyy23BKBDhw45jUmqNm7cuPhyt27dgMQdXSrXte8c739OnTo1aXsm20nTpUxTRCSCnGeaflin8n9JAPr06ZP28f3L+8uXLwegXbt28W2HHnpo2seX7Prwww83uC0TdyKSGZdeeikAb7zxBgD/+c9/4tt8e7RzDoDRo0dXeTy/b/keLjvttBNQWO3YyjRFRCLIeabpB2JYvHgxkBjeK1MWLFiQ9Ll9+/YZPb5kV2WZpm8by8SdiGTGXnvtBSTe6Js5c2Z826uvvgrAgAEDANhuu+0AOOusszZ4PD/4yu6775603k+V4TPOQqBMU0QkAlWaIiIR5Pz23L9c77uNhAfs8K8+NmzYMPJx/QACviuTd8ABB1QrTskt/7qc7yoW1qBBAwB23HHHnMYkVdtmm20AOOSQQ+Lr/PKdd96Z8nG++OILIPFAyNcPfuzMQqJMU0Qkgpxnmn62QP/KpH+dEuDoo48G4IorrtjoMcKjOPsHP4sWLQIqdlmoUUN/F4qBn1fGZxphGqCj9N1yyy1A4vr1D5EyMbxcpqlGERGJIG+vUd50001AcmbhX8XygxFvSPivj//L5GfFK88PIyWFrXxbdHhQlQsuuCDX4UgOhMv88ccfB6B+/foAbLvttnmJKRXKNEVEIshbptm2bVsAnnvuufi6GTNmABU7qJd30kknVVjnO876QVE934Yqhcm/5FD+qXn4SXkmhgqUwvPKK69UWOefa4QHNS40yjRFRCLI+3QXYXvuuWfSzyh+//vfV7o+3A90t912q15gkjV+KLjyT8179OiRj3Akh8KZZt26dQG46qqr8hVOypRpiohEoEpTRCSCgro9T4e/vSt/m6db8sLmO7V7jRo1AuCyyy7LRziSA0OGDAGSZ5hs0qQJUNgPgDxlmiIiEZRMpll+djwpDuPHj0/63Lx5cyAxSIeUHp9phq/Vo446Kmmf77//HoDVq1cD0KJFixxFVzVlmiIiEZRMpvnTTz8lfVan9sL2yy+/ADB//vyk9bVr1wY0Y+imxs8a619OGTx4MJCYecG/ZlkIlGmKiERQMpnmsGHDgMRAD/369ctnOFIFP2Sff0Vyzpw5ALRu3TpvMUn+PPLIIwAMHToUgN69ewNw44035i2mDVGmKSISQclkmj5jufzyywHNcV7oatasCcBtt90GJJ6kFkM/PUnPfffdB8Df/va3+LqDDjoISMxt76fR2HzzzXMcXdWUaYqIRFAymebYsWPzHYJUww477ADAo48+mudIJFcOPPBAAN544408R1I9yjRFRCJQpSkiEoEqTRGRCFRpiohEoEpTRCQCVZoiIhFY+UF7I33ZbDmwKHPhFIWWzrnGVe9WGlTGpU9lHE1alaaIyKZGt+ciIhGo0hQRiUCVpohIBButNM1sWzObGfxbamZLQp+zOvyImdUys1lm9mIK+/YPxTbbzI5O89zvmlmHKvapbWYjzWy+mU0xs8KZxCSCfJWxmT1uZsvNbGaK+/f2+5vZPDM7N83zDzez46rYx8zsn0EZz6rqd6JQ6Tre6D6Rr+ONVprOuZXOuQ7OuQ7AEGCw/+yc+zk4qZlZNjLWK4CPI+x/VxDnqcBjVm6GNTPL9OAkFwBLnXOtgAeAv2f4+DmRxzJ+FIh6UTwZxHkIMMDMGoU3ZqGMjwWaB2Xch1g5Fx1dxxsV+Tqu1v8kM2tlZnPN7ElgDtDczL4Nbe9lZkOD5SZmNsrMppnZB2a2XwrHbwl0BYZFjc059zFgwDZBNvGgmX0A3G5m9czssSCOGWZ2bHC+Lc1sRJDBPA/UTuFUPQA/cclzwB+jxlrIsl3Gzrm3gFXVic05txRYCLQIspMnzOw9YhdZLTMbFMQxy8x6BzHWCLLGT8zsdaDRRk7h9QCeCM75LrC9mZVMVyRdx0A1ruN0au1dgDOdc9OqqP3vBQY456aaWRkwDmhvZvsC5zjnLqzkO/cAV5PaL3YSM+sE/OScWxX8kWoK7Oec+83MBgCvOufONrNtgPeDC+j/gNXOubZmticwLXS8YcA/nHPlbyObAV8DOOd+NrM1Zra1c+5bSkc2y7jazKwV0BL4IhTnQc65n8ysD7DMObePmW0BTDWz14D9gN8B7YAdgLnEsi7M7DbgPefcy+VOFS/jwOJg3fJM/vfkma7jiNdxOpXmAufctKp343CgTSjL3sbM6jjn3gfeL7+zxdqZvnbOzTSzwyPEc7WZnQ18D/QMrR/hnPstWO4GHGlmfwk+1wZaAAcBAwCcczPMbI7/snPunAgxlJqslHEaTjOzLsA6oLdz7tvgnKOdc3460m5AWzPrFXxuALQmVsZPB78Li81skj+oc+6vGYyx2Og6jiidSnNNaPk3Yqm0F06LDdjHt52koBNwgpl1D45T38wed86dVcX37nLO3VNFnAYc55xbEN6hXLNJqpYAzYGlFmtMr1tiWSZkr4yr60nn3GWVrC9fxn2ccxPDO5jZ8dU4ny/jqcHnHYN1pUTXccTrOCMNv8FfgNVm1tpijcnhX9AJQF//wap4muWcu8Y5t6Nzrgw4HXjN/482swG+/aKaxgMXh2LZM1h8G/hTsG4PYNcUjjUG8L8ApwCvpRFXwctkGW+MmV1qZunczo8H+vhbTTNrY2Z1iJVxz6BtsxlwcArHGgOcGRynM/CNc66Ubs2T6DpO7TrO5NOya4n9x0wm1vbj9QUOCBrl5wLnA5jZvmY2JOI5dgeWphHjzUBdi3VnmAPcFKy/H9jWzOYBNwIz/BfMbNgGfkEeBpqa2XxibSnXpxFXschYGZvZCOAdoJ2ZLQ5uyQDaAivTiPEh4HNgppl9DDxI7I5qJPAVsbbMYcCUUCy3mdlRlRxrLLDEzBYEx+lbyT6lRtdxFYrm3XOL5d6vOOeOyHcskj1m9hLQwzm3Pt+xSOaVwnVcNJWmiEgh0GuUIiIRqNIUEYlAlaaISARpvcfZqFEjV1ZWlqFQisP06dNXbEqjequMS5/KOJq0Ks2ysjKmTUvlZYLSYWab1LQAKuPSpzKORrfnIiIRqNIUEYlAlaaISASqNEVEIlClKSISgSpNEZEIVGmKiESQ6UmKRLJu9erVAHz11Vcb3Kdly5YADB48GID27dsDsPPOOwOwxx57ZDNEKWHKNEVEIlCmKQVv3LhxAIwdOxaASZMmAfD5559v8Dtt2rQBYOHChQCsW7cuaftvv/1W/isiKVGmKSISQUFnmv/73/8A+MtfYpPOzZkTm1xuwoQJ8X0222yz3AcmGbdgQWyOrAceeACAhx9+OL5t7dq1AEQZMPvTTz/NYHQiCco0RUQiKMhMc/jw4QDccMMNQMWnpD4DBdh2221zF5hkzeLFsTm87rmnstlbU7fLLrsAiaflUnjmz58PwIoVK+LrXnjhBSDRXl2jRiyfu/DC2MSknTp1iu/bunXrXIS5Qco0RUQiKKhM02cbl19+OZD4S1R+EviLL45Pecz9998PQMOGDXMRolRDOKPwmWTnzp0BOOKI2KSEm2++OQANGjQAoF69evHv/PDDDwD88Y9/BBJZ5L777gvAnnvuGd+3Tp06ANStWzfD/xVSXbNnzwYS7dWjRo0CYPnyqqeQnzp1KpD87ML3jPC/Q//4xz+AxO9QtinTFBGJQJWmiEgEBXV7PnDgQABWrly50f2eeeaZ+PIrr7wCJB4a+Vv3XKXqsmFr1qwBoGvXrvF1H330EQAvvvhi0r77778/ADNmzABiUzB4/kHgjjvuCCQeEkhhmjVrFpC4HX/22WcB+O6775L28+UJcOCBBwKJcr/rrrsA2GuvvQB4//334/v6+uHll18GEq/E+odG2abfPhGRCPKeaS5alJjfaNiwYUnb/F+QJk2aAPD6669X+L7/6+Wz1NNOOw2A7bffPvPBSkp+/vlnAP70pz8BiewS4Prrrwfg8MMPr/S7lc2K2KJFiwxHKJn25z//Ob7suw+Vf9Djy3y33XYD4Pbbb49vq127dtK+U6ZMAeDBBx8E4JxzzolvmzlzJpC4xvv06QPAiSeeCEDjxtmdSFSZpohIBHnPNP1fDUh0Wj/ooIMAeOuttwD46aefAHjqqacA+Pvf/x7/ju8ou3TpUgB69OgBJNo61RUpd3zXIJ9B+AE2wn/5r776agC23HLLHEcnmeSvyQEDBgDwyCOPxLf511232247AC666CIgUfapdAfz7Zbr168H4Oabb45v813P/GAsuaZMU0QkgrxnmuEhu3wndt+53fPtHeeeey4AI0eOjG/zAz34v24+g9HT89zzT8TvuOMOIDEQ8DvvvBPfx3del+LmX3f0T7nDg6k0a9YMSHRi32effao83q+//grA119/DcCZZ54JwNFHHw0kBp6uzBlnnAHA1ltvnXL86VCmKSISQd4zzaeffrrCupdeegmA4447rtLvTJs2bYPH22+//YDk1/AkNyZPnpz02b/eGO6PJ6XBtzXWrFmzwjb/yqPvW+nvDD/55JOk/fwrrwDz5s1L+tmoUSMg8ayiMr5Xje+jnathIpVpiohEkPdM89RTT40vjx49GoAPP/wQSPxl8i/8+/5f4fYN347h1/nBa307R7t27bIWuyQLtzVDogdD+Mln9+7dgeRBNqT4HHbYYQAccsghQHIfat/3+pJLLqn0u7Vqxaodn61WpnyGGX4L7IQTTgDg3nvvBaBp06aRYk+XMk0RkQhUaYqIRGBR5l0pr2PHjm5jD2VSsWrVqvjyTjvtBCRejfSxlR9PMzwAhB8U4JhjjgHgs88+A+CCCy4AYMiQIWnFV56ZTXfOdczoQQtYlDL25VS+vML8gwM/uIIfE9N3NWnVqhUAu+66a4Xv+jmi/OAe2XrApDKO7ttvv40v+y5n7733HpCYXcG/Duu7GYZfrw0PyFEZ30EeEi9PpNPFKJ0yVqYpIhJB3h8EhV9zHDFiBAAnnXQSUDHj9A3Ld955Z/w7vuO7bxz2r1iOHz8eSHR+h0QmK9lx1VVXAXD33XdvcB/fidnfIfifUfjX87p06QIkDxUo+RHO+nymWRXfgR0qZpr169cHYNCgQQCcffbZ8W2VdXPKJWWaIiIR5D3TDPNDR/muK36ADv9X7JZbbgEqDiMFcOONNwKJzrG++5L/DsDjjz+ejbAl4DOMU045BUgM0/fLL7/E9/HzQPmMszqWLVsGJO5MwjNP+o7OUrj8IB8bu0PwQ8L54QULiTJNEZEICirT9HzGuaGBaivjX8nq2bMnkMg033zzzfg+/km9hovLDt/WtPfeewOJngxhEydOBBLZ50033QTABx98EPl8vq17+vTpkb8ruTd06FAA+vfvDyTfgXj+rsEPKFyIlGmKiERQkJlmOnx72pgxY4DkdhM/R3q/fv1yH5gAidfvPD8Itc80/aAL4ekNzj//fAAGDx4MJNq6pTj4sr3yyisB+P777yvss9VWWwGJtswtttgiR9FFp0xTRCQCVZoiIhGU3O25Hw3lmmuuAZLn1/YPHXr16gXAzjvvnNvgpIJu3boBiVkq/cMBP1oVwOeffw4kRgsvz48ULoXJzxXl5wDzwnMF+ea0zp075y6walKmKSISQcllml6HDh0AuPXWW+Pr/Gt+1113HQDDhw8HkkeQltxq27YtkOgq9uyzz1bYJ9xtDBLjMfr5Y8Kv1Urh8A98fGf28k4//fT4sn8lthgo0xQRiaBkM00vPCjAQw89BCRmyfNtZbvvvnvuAxMgkeXfc889QCI7CXdY/+abbwAoKysDEmXq26ilsPzwww9A4i7i559/Ttq+xx57AIkyLzbKNEVEIij5TLNx48bx5QkTJgCJ+bj9ABPqLJ1/fmbBcePGAfCvf/0rvm3KlClAIrP0Q8NJYXrjjTcAWLJkSaXb/XBvlQ28UwyUaYqIRFDymWaYH27fT5fh+4bNnTsX0MyVhcTPJlp+WQqfH6ZAjD8/AAAEVklEQVSxPN93+tBDD81lOBmnTFNEJIJNKtP0/CDH/ine/PnzAWWaIpkQniwREm3Ql112WT7CyThlmiIiEajSFBGJYJO8Pfcz3X355Zd5jkSk9FxxxRVJP/2DoaZNm+YtpkxSpikiEsEmmWmKSPZcfvnlST9LjTJNEZEIzM/oV60vmy0HFmUunKLQ0jnXuOrdSoPKuPSpjKNJq9IUEdnU6PZcRCQCVZoiIhFstNI0s23NbGbwb6mZLQl93jxbQZnZFWY2J/h3cQr79zaz5UFc88zs3DTPP9zMjqtiHzOzf5rZfDObZWYd0jlnvuSxjBeb2ezgPO+nsL/KuJp0HW90n8hlvNEuR865lUCH4OA3AT845waWPymxttHfqjpZKoKgzwI6AuuB18xsnHOuqp7oTzrnLjOz7YGPzWyMc25F6Li1nHPrMxFj4FiguXOulZl1Bh4ADsjg8XMiH2UccqBz7tsI+6uMq0HX8UZFLuNq3Z6bWSszm2tmTwJzgOZm9m1oey8zGxosNzGzUWY2zcw+MLP9qjh8W2Cqc26tc+4X4G3g+FRjc84tBRYCLcysv5k9YWbvAY+ZWS0zGxTEMcvMegcx1gj+2nxiZq8DjVI4VQ/gieCc7wLbm1nJPHHNchmnRWWcGbqOgWqUcTptmrsAg51z7YDKh2iOuRcY4JzrCJwC+ELY18yGVLL/bOBgM2toZnWBI4HmqQZlZq2AlsAXoTgPc86dDlwALHPO7QPsDfQ1sxbAScDvgHbAOUCn0PFuM7OjKjlVM+Dr0OfFwbpSkq0yBnDAG2Y23czOixKUyjijdB1HLON03gha4JyblsJ+hwNtYtk/ANuYWR3n3PtAhbYs59zHZjYImAD8AMwAfk3hPKeZWRdgHdDbOfdtcM7Rzrmfgn26AW3NrFfwuQHQGjgIeDq4NVlsZpNC8fw1hXOXqqyUcWA/59yS4DbsdTOb55ybXMV5VMaZp+s4onQqzTWh5d8AC30OT/5hwD7OueQp6TbCOfcw8DCAmQ0A5qfwtSedc5UN2BeO04A+zrmJ4R3MLOXbhpAlxP5yTg0+78jG/1IXo2yW8ZLg51IzGw3sA1RVaaqMM0/XccQyzkiXo6BmX21mrc2sBsltFxOAvv6DpfB0ysy2C36WAd2BZ4LPl5rZhWmEOh7oY2a1guO1MbM6xNpbegZtIs2Ag1M41hjgzOA4nYFvnHPL04itoGWyjM2snpnVC5brAl2Bj4PPKuM80XWcWhlnsp/mtcT+YyYTaxfw+gIHBA22c4HzgwA31t71YrDvi8CFzrn/BevbAivTiPEh4HNgppl9DDxILNseCXwFzAWGAVP8FzbSFjIWWGJmC4Lj9K1kn1KTqTJuCrxnZh8BHwAvOOcmBNtUxvml67gKRfUapZm9BPTIcJcDKSAq49JX7GVcVJWmiEi+6TVKEZEIVGmKiESgSlNEJAJVmiIiEajSFBGJQJWmiEgEqjRFRCL4fy63uy42kCxvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd2a7fad1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 1 optimization iteration\n",
    "\n",
    "Already after a single optimization iteration, the model has increased its accuracy on the test-set significantly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimize(num_iterations=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 15.9%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xn8VnP+//HHqyylTSlJqs+MkhIysm9ZaoiRvQwiYsjXvg1DE8qQlHWEfmIma0TKEkVZKqlpoWxFEZO0MJbK9v79cV3v65zrs/S5zufar5732+1z+5zrrK96f877ep1z3uf9NuccIiKSmlr5DkBEpJio0hQRiUCVpohIBKo0RUQiUKUpIhKBKk0RkQhUaYqIRKBKU0QkAlWaIiIRbJLOxk2bNnVlZWUZCqU4zJ49e6Vzrlm+48gVlXHpUxlHk1alWVZWxqxZs9LZRdExs6X5jiGXVMalT2UcjS7PRUQiUKUpIhKBKk0RkQhUaYqIRKBKU0QkgrSenovU1NChQwFYu3YtAPPnzwfgqaeeqrDueeedB8A+++wDwGmnnZaLEEUqpUxTRCQCZZqSU7169QJgzJgxlS43swrzRowYAcCkSZMAOOiggwBo3bp1NkKUPProo48AaN++PQB33nknABdccEHeYipPmaaISATKNCXrfHYJVWeYO+64IwCHH344AJ988kli2XPPPQfAokWLABg9ejQA11xzTeaDlbyaM2cOALVqxfK5li1b5jOcSinTFBGJQJmmZI1/n/mZZ56psKxTp05AkEU2bdoUgPr16wPw008/Jdbda6+9AJg3bx4Aq1atylLEkm9z584Fgr+D4447Lp/hVEqZpohIBDnPNH07vAceeACAbbfdNrGsTp06AJxyyikAbLPNNgC0bds2lyFKhvz3v/8FwDmXmOczzIkTJwLQokWLSrf17TgB3n///aRlRx11VEbjlPx79913AbjrrrsA6NOnTz7D2SBlmiIiEeQ807ziiisAWLJkSZXr+HZ5DRs2BKBjx44ZOXarVq0AuPLKKwHo0qVLRvYrlfvTn/4EBE+9ARo0aABAkyZNNrjtE088kZgO39+U0vThhx8C8MMPPwDJLS4KjTJNEZEIVGmKiESQ88vzkSNHAkHzkfCl98KFC4GggeuUKVMAmDFjBhC8NvfZZ59Vuf9NN90UCJqw+IcR4f34y3RdnudGmzZtUl731ltvBYLX6cJ80yP/W0rHkCFDgNjQG1DY56YyTRGRCHKeaR566KFJv8P8K3TemjVrgCDz9N8+77zzTpX733zzzYHghX//eh7A6tWrAdh+++1rFLtkz4QJEwAYMGAAAOvXr08sa968OQA333wzAFtssUWOo5NsCD8M9ue0P2/r1auXj5BSokxTRCSCgn6NsnHjxgAccsghSfMry1LLe/rpp4EgWwXYZZddAOjdu3emQpQM8a9chjNMzzc/8V3CSWmYOnVqhXnNmhX+cPPKNEVEIijoTLMmVqxYAUD//v2B5Ff4/P2y6hpWS+4cc8wxQPBapXf66acnpgcNGpTTmCQ3/BAnYf7Fk0KmTFNEJIKSyzTvueceIMg4t9xyy8Qy/2RO8s+3n502bRoQ3Mv097SuvfbaxLq+mzApDdOnTwdg1KhRiXm77bYbAN26dctLTFEo0xQRiaBkMs0333wTCNryeePGjUtM+27JJP9857IrV65Mmu+7BVRb2tI1efJkILlli2+j7buHLGTKNEVEIlClKSISQclcnr/wwgtA0PfiYYcdBsA+++yTt5ikIj8mkH811uvatSsAN9xwQ65DkhzznfWEnXjiiXmIpGaUaYqIRFD0mebatWsBeOmll4Cgw47rr78eCLqKk/wJjx550003ARV7Y+/cuTOg5kWlbPny5QC88cYbQHJnOscee2xeYqoJZZoiIhEUfabpO63198iOOOIIAPbdd9+8xSTJbrvttsT0zJkzk5b51yh1L7P0PfTQQwB89dVXQHCuFhtlmiIiERRlpuk7rAW48cYbAWjUqBEA1113XV5ikqoNGzasymX+tVfdyyx9S5cuTfrsu34sNso0RUQiKKpM0z+FvfDCCxPzfvnlFwB69OgBqF1msfFlmkorB3814df9+eefAfj2228rrOtf0Rs+fHil+6pdu3Zi+pZbbgE0jEa2jR8/PunzUUcdladI0qNMU0QkAlWaIiIRFMXl+a+//goEPaF8+umniWVt27YFggdCUlz8uE2pOOmkkwBo0aIFEDRdefzxx9OKwY92Ge7DUzLHN2b35VXslGmKiERQFJnm4sWLgWDEwjDfnEX9LxYu/5AO4Nlnn63xfp588slq1/EPiWrVSs4Hjj76aAC6dOlSYZv999+/xjFJ9Z555hkgeGjre2kv1tFFlWmKiERQ0JmmbwzbvXv3pPlDhw5NTBdrs4WNydixYxPTQ4YMASp22OEtXLgQ2PB9yrPOOguANm3aVFh2/PHHA9ChQ4eaBSsZ8+OPPwLw4osvJs333cCFm30VE2WaIiIRFHSmed999wEVX78K3wsxs5zGJOlJdVzrRx99NMuRSLb5+8t+RNiePXsCcNFFF+UtpkxQpikiEkFBZpq+Xdfdd9+d50hEpKZ8punHOS8VyjRFRCIoyEzTj2H+3XffJc33b/+oGzERyRdlmiIiEajSFBGJoCAvz8vzIxVOnjwZgCZNmuQzHBHZiCnTFBGJoCAzzauvvjrpt4hIoVCmKSISgTnnar6x2dfA0mpXLC1tnHPN8h1ErqiMS5/KOJq0Kk0RkY2NLs9FRCJQpSkiEsEGK00z28rM5sZ/lpvZF6HPm2UjIDPrGDrGXDP7zsz+r5pt+pnZ1/H13zezM9OMYbSZHVPNOo3N7Hkzm2dmC8ysTzrHzJc8lXEbM5tiZgvj/3cbLN/4NirjGspHGceP+7AvsxTXL44yds6l9AMMBC6vZL4BtVLdT5QfYFNgBbBdNev1A26PT28DrASalltnkwjHHQ0cU806A4DB8enmwJooxyjEn1yVMbAt0Dk+3RBYDOygMi6dMo7v8yBgT2BuiusXRRnX6PLczNrGs4RHgAVAKzP7JrS8t5mNjE83N7OxZjbLzGaa2d4RDtUNeN85tyzVDZxzy4ElQGszG2Rm/zKzt4CHzGwTMxsWj2O+mfWLx1jLzP5pZh+Y2StA01QOBTSIT9cnVsC/Rvi3FbRslrFz7kvn3Nz49P+AD4CWqcamMs6MbJ/HzrmpwOqaxFbIZZxO4/YdgT7OuVlmtqH93AkMcc7NMLMyYALQycz2Avo6587dwLa9gceiBGVmbYE2wCehOA90zq0zs/7ACufcnma2OTDDzF4G9gZ+B3QklgUtBEbE9zcYeMs590K5Q90BTDCzL4llSye4+NdVCcl6GZvZ74FOwDupBqUyzqhcnMeRFXIZp1NpLnbOVRxTt6LDgPYWDEvR2MzqOufeBt6uaiMzqwMcCVyaYjynmFlXYD3Qzzn3TfyY45xz6+LrdAc6mFnv+OdGQDvgQOAx59xvwDIzm+J36pz7WxXH6wHMJHYJsgPwkpnt7Jz7PsV4i0G2y7gh8DRwQYr/byrjzMtqGddAwZdxOpXmD6Hp34jdE/HqhKYN2NM5V/nwg1U7EnjbObcyxfUfcc5dXMn8cJwG9HfOTQ6vYGbHRowNoC8wMP6t9KGZfU7sP/0/NdhXocpaGVvsAcRYYJRz7rkUN1MZZ162z+OoCr6MM9LkKF6zrzGzdmZWCwgHPwk4338ws84p7vZkyl2am9lFZpbOZcBEoL+/DDGz9mZWF3gd6BW/J9KS2LdOdT4DDo3vpwXQFvg0jdgKWibL2GKpw0PEHhDcWW6ZyjhPsnQeV1DsZZzJdppXEfvHTAPCD27OB/aL37BdCJwdD3AvMxtR2Y7MrAFwMPBsuUUdgFVpxHgf8DEw18zeA+4llm0/Rew/byEwCkgMamJmg82sRyX7GggcZGbzgVeIPZFck0ZsxSBTZXwQsS/FbhY0ffljfJnKOL8yeR6PAd4AOprZMjM7I76oqMu4qF6jNLPngZ7OuV/yHYtkh8q49BV7GRdVpSkikm96jVJEJAJVmiIiEajSFBGJQJWmiEgEaY0R1LRpU1dWVpahUIrD7NmzV7qNqFdvlXHpUxlHk1alWVZWxqxZqbyBVTrMbKMaFkBlXPpUxtHo8lxEJAJVmiIiEajSFBGJQJWmiEgEqjRFRCJQpSkiEoEqTRGRCFRpiohEkFbj9mz5z39iPc0fd9xxACxZsqTG+3r55ZcT0x06dACgVatWNQ9O8mb8+PEAHH300QDcddddAJx33nmJdWrXrp37wCTJihUrADjppJMA2HfffQE455xzgFhj+kz49ttvAXj99dcBOPzwwwHYdNNNM7L/qijTFBGJoCAzzYkTJwKwfv36tPf13HPBmF0PPvggAI8//nja+5XcWbUqNjJCOKMEuOCCCwA466yzEvPq1q2bu8AkYc2aYISInXbaCQgywebNmwOZzzD/8Ic/ALByZWzsRf8qaLt27TJynKoo0xQRiaCgMs1ffokNGfLCC+XHc6+5Ll26JKaHDRsGwA8/xEYDrVevXsaOI9nj71l98cUXSfNPPvlkAOrUqVNhG8kNn+X5+5cQXBmcf35s8Ep/7zlTBg0aBMCnn8YGjbz//vuB7GeYnjJNEZEICirTfO211wCYNm0aAFdddVXa+1y9enViesGCBQD8+OOPgDLNQha+n+0zi/JOO+00AGLDqEs++JYuU6ZMqbBswIABGTvOe++9l5geOnQoAMceGxuWvVevXhk7TiqUaYqIRKBKU0Qkgrxfnr/77ruJ6d69ewPQtm1bAK655pq09x9uciTFY/78+YlpfwnobbJJ7M/2iCOOyGlMEvAN2J9++ukKy3zTvmbN0h8xxF+Wd+vWrcIy//JLgwYN0j5OFMo0RUQiyHumOXjw4MS0f0AzevRoAOrXr1/j/foHQFOnTk3M0wOD4jF27Ngql1WWdUhuXXbZZUBwrvqG5gAnnnhixo7z5ptvArB8+fLEvL59+wJw6qmnZuw4USjTFBGJIG+Z5lNPPQUkN2T39zL32GOPtPfvm6mEs8uuXbsCsOWWW6a9f8mu8BWCt9lmmwFw00035TocKcefV/53y5YtE8t8OdXE2rVrgaCM77nnnqTjQHDPNF+UaYqIRJC3THPMmDFA8EojVOyQoSZ8N3KPPvooEDxpBbj22muB7HcdJTXnX2yYPn16hWVbbLEFAJ07d85pTFK9CRMmJKa7d+8OBFd0qZzXvnG8/z1jxoyk5Zm8T5ouZZoiIhHkPNP03TqV/yYB6N+/f9r79y/vf/311wB07NgxseyQQw5Je/+SXe+8806VyzJxJSKZcdFFFwHw6quvAvDll18mlvn70c45AMaNG1ft/vy65Vu4bL/99kBh3cdWpikiEkHOM03fEcOyZcuAoHuvTFm8eHHS506dOmV0/5JdlWWa/t5YJq5EJDN23313IHijb+7cuYllL730EgBDhgwBYOuttwbg9NNPr3J/vvOVXXbZJWm+HyrDZ5yFQJmmiEgEqjRFRCLI+eW5f7neNxsJd9jhX31s0qRJ5P36DgR8UyZvv/32q1Gcklv+dTnfVCysUaNGAGy33XY5jUmq17hxYwAOPvjgxDw/fcstt6S8n08++QQIHgj5+sH3nVlIlGmKiESQ80zTjxboX5n0r1MCHHnkkQBceumlG9xHuBdn/+Bn6dKlQMUmC7Vq6XuhGPhxZXymEaYOOkrfDTfcAATnr3+IlInu5TJNNYqISAR5e41y4MCBQHJm4V/F8p0RVyX87eO/mfyoeOX5bqSksJW/Fx3uVOWcc87JdTiSA+Eyf/jhhwFo2LAhAFtttVVeYkqFMk0RkQjylml26NABgCeffDIxb86cOUDFBurlnXDCCRXm+YazvlNUz99DlcLkX3Io/9Q8/KQ8E10FSuF58cUXK8zzzzXCnRoXGmWaIiIR5H24i7Dddtst6XcUv//97yudH24HuvPOO9csMMka3xVc+afmPXv2zEc4kkPhTLNevXoAXH755fkKJ2XKNEVEIlClKSISQUFdnqfDX96Vv8zTJXlh843avaZNmwJw8cUX5yMcyYERI0YAySNMNm/eHCjsB0CeMk0RkQhKJtMsPzqeFIeJEycmfW7VqhUQdNIhpcdnmuFztUePHknrfPfddwCsWbMGgNatW+couuop0xQRiaBkMs1169YlfVaj9sL2888/A7Bo0aKk+XXq1AE0YujGxo8a619OGT58OBCMvOBfsywEyjRFRCIomUxz1KhRQNDRw4ABA/IZjlTDd9nnX5FcsGABAO3atctbTJI/DzzwAAAjR44EoF+/fgBcd911eYupKso0RUQiKJlM02csl1xyCaAxzgtd7dq1ARg8eDAQPEkthnZ6kp677roLgL///e+JeQceeCAQjG3vh9HYbLPNchxd9ZRpiohEUDKZ5vjx4/MdgtTAtttuC8CDDz6Y50gkVw444AAAXn311TxHUjPKNEVEIlClKSISgSpNEZEIVGmKiESgSlNEJAJVmiIiEVj5TnsjbWz2NbA0c+EUhTbOuWbVr1YaVMalT2UcTVqVpojIxkaX5yIiEajSFBGJQJWmiEgEG6w0zWwrM5sb/1luZl+EPme1+xEz28TM5pvZsymsOygU27tmdmSax37TzDpXs06ZmU01szlmNs/MDk/nmPmSrzI2s4fN7Gszm5vi+v38+mb2vpmdmebxR5vZMdWs09jMno+X7wIz65POMfNF5/EG14l8Hm+www7n3Cqgc3znA4HvnXNDyx3UiD1Q+q26g0V0KfAesEWK69/qnLvdzDoBr5nZ1i70lMvMNnHO/ZLB+AYAo51zD5jZLsBYoG0G958TeSzjB4F7gPsjbPOIc+5iM9sGeM/MnnPOrQzFmekyvgCY65w70syaAx+Y2aMZPkbW6TzeoMjncY0uz82srZktNLNHgAVAKzP7JrS8t5mNjE83N7OxZjbLzGaa2d4p7L8N0A0YFTU259x7gAGN49nEvWY2E7jJzOqb2UPxOOaY2Z/ix9vCzMbEM5ingTqpHApoGJ9uBHwZNdZClu0yds5NBVbXJDbn3HJgCdA6np38y8zeAh6KZzbD4nHMN7N+8Rhrmdk/zewDM3sFaJrKoYAG8en6wErg15rEXIh0HscORcTzOJ2u4XYE+jjnZpnZhvZzJzDEOTfDzMqACUAnM9sL6OucO7eSbW4HriC1P+wkZrYvsM45tzr25UkLYG/n3G9mNgR4yTl3hpk1Bt6On0D/B6xxznUws92AWaH9jQLucM6Vv4wcALxsZpcQ+xY9NGqsRSCbZVxjZtYWaAN8EorzQOfcOjPrD6xwzu1pZpsDM8zsZWBv4HdAR2BbYCEwIr6/wcBbzrkXyh3qDmCCmX1J7MQ6IZz1lAidxxHP43QqzcXOuVnVr8ZhQHsLxjhubGZ1nXNvA2+XX9li95k+d87NNbPDIsRzhZmdAXwH9ArNHxO65OgOHGFmf41/rgO0Bg4EhgA45+aY2QK/sXOubxXHOwW43zl3h5ntD/zbzHYusZMqK2WchlPMrCuwHujnnPsmfsxxzjk/HGl3oIOZ9Y5/bgS0I1bGj8X/FpaZ2RS/U+fc36o4Xg9gJnAQsAPwUryMv8/gvynfdB5HPI/TqTR/CE3/RiyV9sJpsQF7Oud+SnG/+wLHmdnR8f00NLOHnXOnV7Pdrc6526uJ04BjnHOLwyuE/hCiOAvoCuCce9PMGgKNqeElZ4HKVhnX1CPOuYsrmV++jPs75yaHVzCzY2twvL7AwPgJ9KGZfU6s8vxPDfZVqHQeRzyPM9LkKP4NsMbM2plZLSD8BzoJON9/sGqeZjnnrnTObeecKwNOBV72/9FmNsTfv6ihicRu7vtYdotPvg78OT5vV2CnFPb1GfFU3sx2Amo550qpwkySyTLeEDO7yMzSuZyfCPT3l5pm1t7M6hIr417xe5stiWWP1QmXcQtiDwg+TSO2gqbzOLXzOJPtNK8i9o+ZBiwLzT8f2C9+U34hcHY8wL3MbETEY+wCLE8jxuuBehZrzrAAGBiffzewlZm9D1wHzPEbmNmoKv5ALiF2cs4DRgNnpBFXschYGZvZGOANoKOZLYtfkgF0AFalEeN9wMfAXDN7D7iX2BXVU8ROkIXEHkxMD8Uy2Mx6VLKvgcBBZjYfeAW43Dm3Jo3YioHO42oUzbvnFsu9X3TOFWV7SEmNmT0P9Cy2Zj2SmlI4j4um0hQRKQR6jVJEJAJVmiIiEajSFBGJIJ12mjRt2tSVlZVlKJTiMHv27JUbU6/eKuPSpzKOJq1Ks6ysjFmzUnmZoHSY2UY1LIDKuPSpjKPR5bmISASqNEVEIlClKSISgSpNEZEIVGmKiESgSlNEJAJVmiIiEaTVTlMkH9asifXO9tlnn1W5Tps2bQAYPnw4AJ06dQJghx12AGDXXXfNZohSwpRpiohEoExTCt6ECRMAGD9+PABTpkwB4OOPP65ym/bt2wOwZMkSANavX5+0/LffMj1SrWwslGmKiERQ0Jnm//73PwD++tfYoHMLFsQGl5s0aVJinU033TT3gUnGLV4cGyPrnnvuAeD+++9PLFu7di0AUTrM/vDDDzMYnUhAmaaISAQFmWmOHj0agGuvvRao+JTUZ6AAW221Ve4Ck6xZtiw2htftt1c2emvqdtxxRyB4Wi6FZ9GiRQCsXLkyMe+ZZ54BgvvVtWrF8rlzz40NTLrvvvsm1m3Xrl0uwqySMk0RkQgKKtP02cYll1wCBN9E5QeBv+CCxJDH3H333QA0adIkFyFKDYQzCp9J7r///gAcfnhsUMLNNtsMgEaNGgFQv379xDbff/89AH/84x+BIIvca6+9ANhtt90S69atWxeAevXqZfhfITX17rvvAsH96rFjxwLw9ddfV7vtjBkzgORnF75lhP8buuOOO4DgbyjblGmKiESgSlNEJIKCujwfOnQoAKtWrdrgeo8//nhi+sUXXwSCh0b+0j1XqbpU7YcffgCgW7duiXnz5s0D4Nlnn01ad5999gFgzpw5QGwIBs8/CNxuu+2A4CGBFKb58+cDweX4E088AcC3336btJ4vT4ADDjgACMr91ltvBWD33XcH4O23306s6+uHF154AQheifUPjbJNf30iIhHkPdNcujQY32jUqFFJy/w3SPPmzQF45ZVXKmzvv718lnrKKacAsM0222Q+WEnJTz/9BMCf//xnIMguAa655hoADjvssEq3rWxUxNatW2c4Qsm0v/zlL4lp33yo/IMeX+Y777wzADfddFNiWZ06dZLWnT59OgD33nsvAH379k0smzt3LhCc4/379wfg+OOPB6BZs+wOJKpMU0Qkgrxnmv5bA4JG6wceeCAAU6dOBWDdunUAPProowD84x//SGzjG8ouX74cgJ49ewLBvU41Rcod3zTIZxC+g43wN/8VV1wBwBZbbJHj6CST/Dk5ZMgQAB544IHEMv+669Zbbw3AeeedBwRln0pzMH/f8pdffgHg+uuvTyzzTc98Zyy5pkxTRCSCvGea4S67fCN237jd8/c7zjzzTACeeuqpxDLf0YP/dvMZjJ6e555/In7zzTcDQUfAb7zxRmId33hdipt/3dE/5Q53ptKyZUsgaMS+5557Vru/X3/9FYDPP/8cgD59+gBw5JFHAkHH05U57bTTANhyyy1Tjj8dyjRFRCLIe6b52GOPVZj3/PPPA3DMMcdUus2sWbOq3N/ee+8NJL+GJ7kxbdq0pM/+9cZwezwpDf5eY+3atSss8688+raV/srwgw8+SFrPv/IK8P777yf9btq0KRA8q6iMb1Xj22jnqptIZZoiIhHkPdM8+eSTE9Pjxo0D4J133gGCbyb/wr9v/xW+v+HvY/h5vvNaf5+jY8eOWYtdkoXvNUPQgiH85PPoo48GkjvZkOJz6KGHAnDwwQcDyW2ofdvrCy+8sNJtN9kkVu34bLUy5TPM8Ftgxx13HAB33nknAC1atIgUe7qUaYqIRKBKU0QkAosy7kp5Xbp0cRt6KJOK1atXJ6a33357IHg10sdWvj/NcAcQvlOAo446CoCPPvoIgHPOOQeAESNGpBVfeWY22znXJaM7LWBRytiXU/nyCvMPDnznCr5PTN/UpG3btgDstNNOFbb1Y0T5zj2y9YBJZRzdN998k5j2Tc7eeustIBhdwb8O65sZhl+vDXfIURnfQB6ClyfSaWKUThkr0xQRiSDvD4LCrzmOGTMGgBNOOAGomHH6G8u33HJLYhvf8N3fHPavWE6cOBEIGr9DkMlKdlx++eUA3HbbbVWu4xsx+ysE/zsK/3pe165dgeSuAiU/wlmfzzSr4xuwQ8VMs2HDhgAMGzYMgDPOOCOxrLJmTrmkTFNEJIK8Z5phvuso33TFd9Dhv8VuuOEGoGI3UgDXXXcdEDSO9c2X/DYADz/8cDbCljifYZx00klA0E3fzz//nFjHjwPlM86aWLFiBRBcmYRHnvQNnaVw+U4+NnSF4LuE890LFhJlmiIiERRUpun5jLOqjmor41/J6tWrFxBkmq+99lpiHf+kXt3FZYe/17THHnsAQUuGsMmTJwNB9jlw4EAAZs6cGfl4/l737NmzI28ruTdy5EgABg0aBCRfgXj+qsF3KFyIlGmKiERQkJlmOvz9tOeeew5Ivm/ix0gfMGBA7gMTIHj9zvOdUPtM03e6EB7e4OyzzwZg+PDhQHCvW4qDL9vLLrsMgO+++67COg0aNACCe5mbb755jqKLTpmmiEgEqjRFRCIouctz3xvKlVdeCSSPr+0fOvTu3RuAHXbYIbfBSQXdu3cHglEq/cMB31sVwMcffwwEvYWX53sKl8Lkx4ryY4B54bGC/O20/fffP3eB1ZAyTRGRCEou0/Q6d+4MwI033piY51/zu/rqqwEYPXo0kNyDtORWhw4dgKCp2BNPPFFhnXCzMQj6Y/Tjx4Rfq5XC4R/4+Mbs5Z166qmJaf9KbDFQpikiEkHJZppeuFOA++67DwhGyfP3ynbZZZfcByZAkOXffvvtQJCdhBusf/XVVwCUlZUBQZn6e9RSWL7//nsguIr46aefkpbvuuuuQFDmxUaZpohIBCVOoGbZAAAE5ElEQVSfaTZr1iwxPWnSJCAYj9t3MKHG0vnnRxacMGECAP/+978Ty6ZPnw4EmaXvGk4K06uvvgrAF198Uely391bZR3vFANlmiIiEZR8phnmu9v3w2X4tmELFy4ENHJlIfGjiZaflsLnu2ksz7edPuSQQ3IZTsYp0xQRiWCjyjQ938mxf4q3aNEiQJmmSCaEB0uE4B70xRdfnI9wMk6ZpohIBKo0RUQi2Cgvz/1Id59++mmeIxEpPZdeemnSb/9gqEWLFnmLKZOUaYqIRLBRZpoikj2XXHJJ0u9So0xTRCQC8yP61Whjs6+BpZkLpyi0cc41q3610qAyLn0q42jSqjRFRDY2ujwXEYlAlaaISAQbrDTNbCszmxv/WW5mX4Q+b5atoMzsUjNbEP+5IIX1+5nZ1/G43jezM9M8/mgzO6aadRqb2fNmNi8eZ58NrV+o8ljGy8zs3fhx3k5hfZVxDek83uA60cvYOZfSDzAQuLyS+QbUSnU/KRynMzAPqAtsCrwG/K6abfoBt8entwFWAk3LrbNJhBhGA8dUs84AYHB8ujmwJsoxCvEnV2Uc3+cyYMsI66uMi6iMS/k8rtHluZm1NbOFZvYIsABoZWbfhJb3NrOR8enmZjbWzGaZ2Uwz27ua3XcAZjjn1jrnfgZeB45NNTbn3HJgCdDazAaZ2b/M7C3gITPbxMyGxeOYb2b94jHWMrN/mtkHZvYK0DSVQwEN4tP1iRXwr6nGWeiyXMZpURlnhs7j2KGIWMbpNG7fEejjnJtlZhvaz53AEOfcDDMrAyYAncxsL6Cvc+7ccuu/C/zdzJoA64EjgLdSDcrM2gJtgE9CcR7onFtnZv2BFc65Pc1sc2CGmb0M7A38DugIbAssBEbE9zcYeMs590K5Q90BTDCzL4GGwAku/nVVQrJVxhD7Y33VzBzwT+fc/0s1KJVxRuk8jljG6VSai51zs1JY7zCgvZn5z43NrK5z7m2gwr0s59x7ZjYMmAR8D8whtW/3U8ysK7EC6uec+yZ+zHHOuXXxdboDHcysd/xzI6AdcCDwmHPuN2CZmU0JxfO3Ko7XA5gJHATsALxkZjs7575PIdZikZUyjtvbOfeFmW0DvGJm7zvnplVzHJVx5uk8jljG6VSaP4SmfyN2T8QLD/5hwJ7OueQh6TbAOXc/cD+AmQ0BFqWw2SPOuco67AvHaUB/59zk8ApmlvJlQ0hfYGD8W+lDM/uc2H/6f2qwr0KVzTL+Iv57uZmNA/YEqqs0VcaZp/M4YhlnpMlRvGZfY2btzKwWyfcuJgHn+w9m1rm6/ZnZ1vHfZcDRwOPxzxeZWWWXeqmaCPT3lyFm1t7M6hK739Irfk+kJbFvnep8Bhwa308LoC1Qst0mZbKMzay+mdWPT9cDugHvxT+rjPNE53FqZZzJdppXEfvHTCP2ZNQ7H9gvfsN2IXB2PMC9zGxEFft6Nr7us8C5zrn/xed3AFalEeN9wMfAXDN7D7iXWLb9FLH/vIXAKGC638DMBptZj0r2NRA4yMzmA68QeyK5Jo3YikGmyrgF8JaZzSN2afSMc25SfJnKOL90HldTxkX1GqWZPQ/0dM79ku9YJDtUxqWv2Mu4qCpNEZF802uUIiIRqNIUEYlAlaaISASqNEVEIlClKSISgSpNEZEIVGmKiETw/wH6zwoLIGTQ3gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd2a7fa7cc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The weights can also be plotted as shown below. Positive weights are red and negative weights are blue. These weights can be intuitively understood as image-filters.\n",
    "\n",
    "For example, the weights used to determine if an image shows a zero-digit have a positive reaction (red) to an image of a circle, and  have a negative reaction (blue) to images with content in the centre of the circle.\n",
    "\n",
    "Similarly, the weights used to determine if an image shows a one-digit react positively (red) to a vertical line in the centre of the image, and react negatively (blue) to images with content surrounding that line.\n",
    "\n",
    "Note that the weights mostly look like the digits they're supposed to recognize. This is because only one optimization iteration has been performed so the weights are only trained on 100 images. After training on several thousand images, the weights become more difficult to interpret because they have to recognize many variations of how digits can be written."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztnX94VdWV979LSQwQLKkgPwSMCkqxKqgIovjQUZFW2nFaLfrUdpwWO7a1M9bad+zUt48dO1P71r61U9vavj6V6eO0drRqK23xR6eMCGIRRUGNAkIFDDbRBIJyS4L7/WPt79n7ntyQQO7NyQnr8zx51v2xz7nnrntyznevvfba4pyDYRiG0fcckvUBGIZhHKzYBdgwDCMj7AJsGIaREXYBNgzDyAi7ABuGYWSEXYANwzAywi7AhmEYGWEXYMMwjIywC7BhGEZGDNqfxtXVI9zgwfUVOpT+x+7dm7FnT7P05WeajyvLweZfANi5c3Wzc25kX32e+bjn7NcFePDgesya9dT+fkZuWbHi9D7/TPNxZTnY/AsAS5bIn/ry88zHPcdCEIZhGBlhF2DDMIyM2K8QRF/zl7+o3bUrvDaoiyOurVV76KGVPaaBhvm4sph/K0+efWwK2DAMIyPsAmwYhpERmYYg2HXo6FDLbgOfs0vB5zHvfa/ampriNlu3hjbcH7sb/Lz06wOZnvp49+6wzRtvqKVvuc3w4cU2fu9g8vHgweExz8P6erXp87C1tdgCQKGglj6inw87TC39H293MPk3zUA+h00BG4ZhZETFFfDevWp5tzrhhPAelcT27WpbWtRu2KC2qUnt+98ftmEQnXcw3uEI9wUcHOoA6OzjuMdAddDcrJY+LRTafYs9AIBJk4Ym26R9Onq02hEj1J53XnjvdJ/Gu3Kl2scfL/7cgUDav3wOAEO92y6/XG311lf0AU/EyZMBAHtq351ss3mzWvps3Tq1VMZHHBH2z93wPZ7/A42RfgrDxIlqS53DhL0OXgPoI/oTCOf566+r5TlNPzY0hLbvepfaYcPU9qWPTQEbhmFkRJ/FgKmiYvVApbt+vVrGXqi0Fi1Se/bZYZvqXW/qg6VLi/b/3MQPAyiO7TzlJ+OkY8g8hvhYSJ5VM+/uVLvx45YWBhYZKPtz0bYbNpyaPHZO1fGwYVUAgsqliqCCA4CFC9VOnaqWcTP+PKWUcJ59DIR4LxCUW3VhJwDgnfpjAQCL16ndsEjf5/kPAOPGqWXMl78Re4dxDHjOHLVLlqhlbJn7S/dWgHz6l//zVJ9UwkA4tybgVX3gv/TbtUcCANau7bwN/+fpn1Wr1H7/+2obG9uTto2N+j8hcnjRfvrCx6aADcMwMqLiCph3NN5NYjVKJcU4zYIFar/9bbVDVv63Prhrc+eNGDjznDxT5fSoT/2vTscQKzYgKI5SWRZ5VA88fsbQ4+/b3r7TP9qKYnysEvq+c29F700AALS16Y/34osalLzsMn2XagUIvYxHH/Wfkhr9pyqPVUTefEz/sncV98joix/frerpttv0Oc+xmTO73h+VNBXx6tWdt+FpzteohOlf/n/l2b9A8Anjuz50DgCYsMFfB+gMHySvuexjAICqquJtAWAsXtMH/ocYfv7JAEKvuhjtcjin/wutrfqD0KeV9LEpYMMwjIyomAJmPJd3eaqHWG1yNJ0xRt7BhjQ8rQ94m1+8OGyUvi1RxvphzVF/+mPS9FOfOgNAiEcuX66Wccr0LvIGfUw3bdyotr09CiImSvclb71c6KSIq8KjqqMBhF7L9Olq6bdYaVBR8Cfi70yfUt11NTW0P5POP2VsMM7ppeL/ylcYU+Twuiri++/nkHo0OIFGb1VxTZ48BUAYhY8zeZhlMWaMWqpvjuKnc2PzBn3M78PvwZ4TABQm/5W22aWW59rKb6rleXruuWGbK64YCwAY1aw7HuR7KvRjQ8Oe6Cj4m2mjpqYR/lj0WpPOJS4npoANwzAyomL3Td6pGSPjnTrOv7voIrWM9xzb7NUrJcaLL/qjjA7zwgvVMiDJWyWDY9EQ9Uv+sxa8T0f8q6p01PSRR/T1eLQ5j/Cr010h3hulQSRKd423h3vL6VynAACGDQuBx4svVsvfjgqDyjfukPz85zwW/eymJg2OjRyp0i2vygwI/mWeKOPcMc88Q5+zh0F1y2yTo7m3aCsGEzXtoaFhk3/O3yaoszPPVMkWZ14AoWexNd2RyRn0MVU/M6P4rw8Ab/nhCeaYp88p9rriGYpU1rwQrbm7+PPCbwAA7KVUe6v+7+ioKfl55cQUsGEYRkbYBdgwDCMjyi6u6+rUstva1qaWqWbs3gIhtebwpb/WB+yHsE/xd3+n2x46NtmGKVYPfZ2vDAEAnHCCTsRY0PxC0vacgu92N2hX4sPJAIaGIvIagkjXP21krxfMjYlTytidrUYxR3r7PgBhQgUQQkMMG3GQlClnd90V2hYKz/pHnD9b7V8fWrSPPEH/povmkLh7DHAlGp53/h8gCSdw7n0cK2A/mH1m/l4cCA1VZTgAlJ4OzWPLo3+Brs/hgv9ijY0hRFBI4gY6Y6u9nSEDPcc2bNBzec2aZJNkAktzc3XR/sMkpZ2hceJvfmbBf67+hpX0sSlgwzCMjCi7Ak6XiWRQnQH0eDBh7tlv64N771XLEYWrrgIAPLlFle9nPhO2SZeg453tyivVLnhfNEvg9tvVco6nr9Zx3HE6WYN3Q6r0vFCqxKHCudVxlvg73lJdjfJWB3fOP19fjwvsnDPRJ7Ez8X2Qdh0WL9beRmNj6GWEe3jxZ6dVY54mB9CvPD+oeNm7KxQao9YcQKMSpoqiEvb/ADi20+fU1Wnblhb9Tao4oyBRzyH1jf9PnLbPHmVey1J2fQ5rj61QeCl6zc8wSnpZnBikfuK5G5/D3C/TJNm5Dp/3TmgMFqLib9d357ApYMMwjIwouwJm7Dcdo5o3T+0/XBUlQF99jVre5n2w8b5BHwUAfGQmVdWmsE0SS6Na0PjZ976nAeVbbjkyaZlEPb/7XbU+trzgcf3cl15Kx0XzQamykwonA0SVqZOJGIyj0W/aU2B89wMTXw6bXHWdWv+7PIy5AIDvfIf7iH8Pnw+VKAr9zVgcO49paF35t7GRgwbxCuS+EkwSx6XleV6Teg7QRy0t6k8qYaaWxUVlmArIsQ9OtqEaoyrPG/t3Dmu8VmQOAODrfvyHvW3auIxkutwAz8PSMeA0fXcOmwI2DMPIiLJf25nAwHAu70rXeLGLG24IjXl78oHhFRM/AQC48So28FOSi0onUn0wRrSnqM11112etPx3/+Hv+MBZcrfxFVPGjbu2+y+UK9hjaIleo8o/NNVG/cZJBuEHAgq/+x0AoOYXvwAA3HEH3/mFtyFGCXAeN5W1qjlOqx1Y7I9/aamIn0aAMUdV1IMG6WSY+fP11VgBs8g4Y5i0jAnnVQF3Df0Wn2OaLcIMKireD53uxyr8xebljjM67Y2F8r/1LbXOsRcX7z/unYRj6Itz2BSwYRhGRlQsusH4Cackv/vnvhLysmWhEW8x/tZ/1hGMsbGgDp/HOZRUwFQhVBgai4uXJcE8/Xq+hHtynxvrlffWnBbhIexdtCSCbHeJVhw55r2WSk1H3rncU+H63yVb1IwfDwB48miNxd9zj8/TRin1UJ2yxZIsjzFg0tm/9F28Zs0Eb4em2rDXxt8kTjqnj2YDAK6+Wp+xE/LAA6Elx1KeeILP9YWamspPk+0LOvuYxOeY/o/zu35onO9N3O+d4lNCjr8+VIl6YbNm7HBpopB9xatAHAPmOEbpNAeLARuGYQxAyn5tZ54d7zgfcL/RBxy+3bYtNL71VgDAp6/ngoU+HzhRvLxLxXmXcaGZGFUVXJ4EADBH1QK/ZHK38dV/OopruueG9LLcYaYQMxHi7A5ffy9RaFocZvr0YwAAJy/SOHicjllzxRUAgJkzmcP6K29P8vbIuLW3VA/p0f/8Qf8yA4Hn8sqV2ptoa5sVtabPeV6mpRyVb5w7reUnzz9f7aWXFm/B3HkgFAAqFKioa1I2n3R9DqdnBQLDhmkeP9PSf7BSl8+a/0G1HEraHvV+uV8W8vGdOnzpS6qs16wJCpsFwhhfD0WtKu9jU8CGYRgZYRdgwzCMjCh7CIKDYD++1U8zvvF/1HIu8pYtobHvJ9x9N184NGXZHYnDDkzUZveAU4+1a/3lL0dNfT3PQ1JbMFcurrqfJ5iCFFbyYLefL8QhCD7mwJH6Ken2blAnDOfyvkBIFbzpSf8Cu9ksHhM7joWSiu/llVxFoNJwQgTTndIrphQKYbCG9Y/f//4xvq1aTpzgtk88EVadrqnRcBB/g+Mnqn//e6n6kClT+lkMxzE9q/KrNPQF3Z/DYTC33f/Lr12r593nPrfTW4Ymh6YsMH26/g4sMkVfM0221NpwW7YUT87oCx+bAjYMw8iIsl/bk6mFHEmgFOBtLM5u9hLjqKP0aUMDlS6nz3LgIU5tohKgGmOwXqciFw1o3KZDS9zrsZMm6YOpUwEAWxft+7v0VzqXIkz7JC40wvd00O1971NfsuQkbvfKmPM7Afx4EVUzRzS5nlmp+zVfKx4A5FpmeVJorIVD1ZSevsrJRfGabRygG+VrHDHxn+qJPcLLLw/qjPvzY514u6A+vPlmfV4oxFO9B45/Y7o+h2tTz4FCgaOSO1OW1wn6dlSyzapVOhg6bpwOdLL07bFP/RcA4NorQuWeu+9mEgD3o9eqvvCxKWDDMIyMKNu1PV1gOap6obCmJGPBAN4crXenvcnNjutnUYHxThfuhkEVMxXqrwEAV16pCu/4EW+Gpg8+CADg/Y1Vmp/eqtt2Tv7u3yTrXHl2J/Mu0uu8rY9aqe+mT1fle/31+uqxVA9ckC9a6vi6RBz4modIz3eNC74zzUqPQUSlH6fI5qlMIgsTzZih9iU/250TI6hmGxtDDPzSS/X7sgB4eh09KuQh219Jtkkk1Xa1ix7QOPojj3C/cfpTVF4VgIjK9Dz6F+j+HB4zRr9YY2P8P8/8MirhdOpjqZ6ynpf336/XkLPP1v2fPEfneb9dk1wVoundur/Bg9X2hY9NARuGYWRE2aMbyZQ/Bnc47Mj1bDg3OXopwDsYpwYy4+G1qM0CAMDkyTrKSWH9Dxf7NndE6+X4SR/DOd/WF2hfcnMPvkg/hq7lKrB79+otup1xdoTlYWtqdPIEY75zJ3olxh6Kl30vDDo52SaMTPP3SE/0iKc8sxtRvHpvHmOTzH5gLJgJ+nQV3583LyjUf17o1ZefU3vfEp0CSwUclHAoyH6I9yNjv2EpHfoyKDkqXq4pMG2aWhZR4hBL3ujqHOb3aoznXhWNaQDhfOTvwOvFlBJttLd2222qgOfN02yUeNIGL1E8Jv5mfXEOmwI2DMPIiLJf45OiyIwtcvhx6dJiC2DuLVqEZ/58jYEtXjwdQIinMfYyZswpyTbp2FqyDMmiJWprovgZ81l9asTDj+r9pvMyKPmCd+b0dE4q06qqUM/wkkvU/vP1XkX8TtfXeeU9FwIIfrz+omQTOMfYO2O9VGZxgRSkXtNYJdVEHhUw4ZRXjhEwrkvL4jkxP713SNFzqmbui/8GAFBfX6x8Qz46H4TyiKNH6/nMThzP3XhGfx7h+cHrBS2TpBiPBYBC4Rj/iOcae8QcMyqV0M/sKE1l2LSJC23WeBta8hrC4am0Iq4kpoANwzAywi7AhmEYGVH2jiIHcH7xhNZJXeAnPSTEowZ+NeT/e5UuGHfDDccDCF0B2hFRJg5XDWB44pCCn/LMkAdjFEDSX3uyRffLNCIOruSVdAiClmNwdAVQvFIsAOw5X0MPq+7X55wvw0kGSnq9rHS+3qjosQ6AjBypXb48T5Fl7VieQqykxe/EUzeuVrZmjaY0sl4vu9D8DTqHiYC77ireD8elx43TrjYroMXbcwCQ3WP+nyXjrjmD34upf/w+9P1FUUjsxhv1y48YoXbyZA1J0qf8v25vj1PXigfqPv95DlSjaNtS8FjiNeYqhSlgwzCMjCibTjnsML9Dv8fly/mOpn2ceZ7aCUt/GjaijF2iA2jvrlkKAPiQX0K5vn6Ct2ET3pV4l3obOvgxhLfSiCebNPVn9Wp9zrQ3BuDzlsROH/P4074491xNX2LtUyAM2nzzW3qv9YsHJINEfP+ZZ+J199JTPllwh5NfxiUtRTT97Ai/8EYelS950tceYl0innfsHbDnFM0lSpTu88+rTU9fpmq+K8qO/MMfVvlHen7efLM67xQ/1hx34rgfKl+qcT5va+v2a/Ur0ucwvw+ncsfnLrnxRrX0C893nrvjktMx/EOPG6eDbzwvZ+viI8lvGuUCJL8nfRnXpao0poANwzAyomLFeHhn43RO1sHBnE8kbSfs8qsE8Fbm5cTr7TpNkAovLn7COyfvgvyc1ladvhzHMvk4jqkB+VZpQPjO6RUbeJc/99zQ9ve/V8vye3R1UxOnEBevkqzwx/LZ/8mqGnz9mKQlV/DltM08+5bTYjn1mN+Fr1MBx8MYPA+pSJl2xrZhSu1LYSNE05IBbNumMo2/3xvR8nH8jdPx/r6IT1YSnsP8f/ad3kQh743CuewF0Bcnnqh2xw61PP/jTjD3y2sHf6cf/Uit73QDCNesLM5hU8CGYRgZUfZrPUd203dov/xbcvcCgEmTphS1KWxWW7SyMYLKAjoXxz7uOLW/+lXnz83rNM3uoErgd6V6YD2dUgotVAXlHE+VGDU1GkCrCktwJUpj06az/Csa+x02TCVCHJMfCBMv0nCknL0FqqgtW5hy8HrSdv16xh2ZqsNVu3kSM4MkHnBgj0JjwDzf+XnxOc7Y6JgxGFDwHKa6ZRbJxz+udtaIlztt885EzWZi/JbnPye5xD3lxYvV8rxPlxSN52tleQ6bAjYMw8iIsl/zGS+jpbLi3X1oqEudxHCo4HiXYvyGqxfFd7Z0G8ZveDfknRUIWQ55y3boKVRqzO4o1ft45hm2ZVBNp3NOm6Y/BH+n14Ooi2JiGgPmKDOVcezjgQzPVX7/2lo9mbduDVkgTU30K+e3c/o2C+qw+xa2YR51XZ3uj+cy1VnsXyq4gdqbi68HQPhff6zj+OS1JL7uly6jAk6vZhyr2vR1gr8hbX85h00BG4ZhZETFox68AzG+snFj5zbMUuBdiwqs1IgvVR+Vdd5Hg8sBlRNVUqlY1vTp2g1obh7qbXHbcZFAo4+pPA52H9NH9Ec8M5Ox3dZWzWRoblbb0XFK0bZ2Du8b+oIFiuJsJp6rVLpdXSdiWNaSv1V/9bEpYMMwjIywC7BhGEZGVDwE0ZOBMHYP0gUy4qB6V+yrG3KwYD6uLObfyrN+ffdtBqKPTQEbhmFkRL+4H/CuxID5vpQGpyim72gDNdWsXJiPK4v5t/IMRB+bAjYMw8gIcc71vLFIE8Jcy4OBo51zfVicznxcaQ5C/wLm477ggHy8XxdgwzAMo3xYCMIwDCMj7AJsGIaREQd8ARaR74jINdHzh0Tkjuj5t0Xk2m72saIHn7NZREaUeH2OiMza3+OOtj9NRNaKyAYR+XcRkQPdV6UYAD7+VxHZIiK7um+dDXn2sYgMEZHfiEiDiDwvIjcfyH4qTZ597LdfIiLPeh/fLiJly6XojQJeDmAWAIjIIdCCqFG1X8wCsE+nOecO2CkA5vDzD5AfArgSuszDJADzerGvSpF3Hz8I4IxebN8X5N3HtzjnJgOYBuAsEXl/L/ZVKfLu4486504B8F4AIwFc0ot9FeOcO6A/6EqNW/zjkwD8B4CHoXX4DoPW56v2738JwCoAzwH4WrSPXd4eAuAHABoAPALgtwAu9u9tBvA1AE8DWAtgMoB6ANsBbAOwBsBs75R1AJ4F8Fg3xz4GQEP0/DIAPzpQX1TqL88+Tn2PXVn7cqD72H/GdwFcmbVPB6qPAVRBRcWCcvnmgCdiOOdeE5EOEZkAvbs8AeAoAGcC2AFgrXNuj4jMhSrMMwAIgF+LyDnOucei3X3YO2oKdPmFFwH8JHq/2Tl3qoh8FsB1zrmFInK7/1FuAQARWQvgAufcNhEZ7l8bC+AO59wHUod/FICo3hK2+tf6FTn3cS4YKD72bT8IvQj3KwaCj0XkIX9cvwNwbxncAqD3g3AroA6lU5+InnNh+rn+7xnonWkywuqO5GwA9zjn3nHObQfwh9T793m7Gur8UiwHsEhEroSvEeicey2vF4YI83HlybWPRWQQgJ8D+Hfn3CtdtcuYXPvYOXcBtOd8GIC/2tcX3R96OxWZsZ2ToJJ+C4AvAtgJ4E7fRgB8wzn3o158zl+83Ysujtk5d5WIzABwIYDVInKac+6NUm2h3ZF4iYJx/rX+SF59nCfy7uMfA1jvnLu1F8dWafLuYzjnCiLyKwB/DQ1/9JpyKOD5AN50zu11zr0JYDi0a8Gg+kMAPikitQAgIkeJyJGp/SwH8BEROURERkGD5t3RBmAYn4jIcc65J51zXwXQBGB8Vxs65xoB7BSRmT774RMAftWDz8yCXPo4Z+TWxyLydQDvAnDNvtr1A3LpYxGpFZEx/vEg6EW7oQef2SN6ewFeCx3RXJl6bYdzrhkAnHMPA/gZgCd87OVeRM7w/BIah30BwF3Q7seObj77QQB/IyJrRGQ2gG+JppWtg/6gz4rIWBH5bRfbfxbAHQA2ANgIje30R3LrYxH5PyKyFcAQEdkqIjf2+Fv3Lbn0sYiMA/AVaDz0ab+PhfvzxfuQXPoYwFBoLPo56CDenwHc3tMv3R39ZiqyiNQ653aJyBEA/gjgLB/jMcqE+bjymI8rz0Dycb8oR+lZ7EckqwHclFeH9nPMx5XHfFx5BoyP+40CNgzDONiwWhCGYRgZYRdgwzCMjLALsGEYRkbs1yBcdfUIN3hwfYUOpf+xe/dm7NnT3KdV0szHleVg8y8A7Ny5utn14YoY5uOes18X4MGD6zFr1lP7+xm5ZcWK0/v8M83HleVg8y8ALFkifbo8kPm451gIwjAMIyP6Ux5wJ/7iZ3Xvisp5D+riiGtr1fa3Zaf7O+bjylBVpXaerzJ9+eXhvc2b1dbUqN3us1jv8CXKm5oqfngDijyfw6aADcMwMsIuwIZhGBmRaQiCXYeODrXsNvA5uxS7d4dt3vBF49h94zbDhxfb+D22nTpVLbuAB0NXr698zC4dPy/9+kClK/9Om6Z2zhy1hzf8Mdnm5EWL9MFCrZtTU38qAGDmTH15a7RUwIYNatvaij/vYPEvMLDPYVPAhmEYGVFxBbx3r1rerWIOO0zt6NFqm5vVPv642vb2P/uWr0db8XbExU/rAADHHKOjHqNGhZZUumeeWfy5vCsOFNI+jn1NdcDX0oqgtVXtpk3t0R53+7aHAwDGjOm8X8JBDVrur1DYzy/Rj+mJf3nuDh2qln6eghf0wbp1YaPTfeqd75J9/ari/dKXMVR0pd4bCOyPj9lzLRR4zrZ42/11osqPjo6I1k5+17vUDvOFL/vSx6aADcMwMqLPYsClFBHvNCt9iebGRrXt7c/6FqzdHN/Z/K0Sx3p7AgBg61YNoPEuCQBH+WU2qcpIHGNLk+eYGn0c+4CPZ8xQO3iw2hdfVLtp01u+5ZpoT2/5/U33bdgTUZ+feWZV0pI9jssuU0tfc/87fKns7VHBQP7OeWNf/m1pUTU2ebL6hrHfPROnAACqJ09OtvnJItU9nzqUufv8DVSlTZs2JmnL/5Gn/LyGiRPVstdYqjc3UM/hlhauGkTFy+XvNnlb6jrBlce0N9fertK3sTGsSNbYqItuiKj/+9LHpoANwzAyouIKmLGcFn/TGhnNlmaMdtMm3q1e8vaVlG1BgFJ6Q9HntLcP9XZw8tr48Xor44gnw3C8y1JdxHe4PKqHtI/pVwCor1f7nveopTJ9ia4G1W0UFINPF0G1t6rQRFTdUSEAQemdM3NP0YfPm3c8AGDpUn2ZvRwgfwp4X/5tb1ffDBum5x97AvyO3Ob73w9a56ab6IyHvaUa09/gmWfS68UCVHStraqOec4eDOdwe/vOVGt/riVxXvroiKhNOj7MXgb3FXeD1afOaW+ltVX31xc+NgVsGIaRERVTwOnpgRzljGOBIc7zTmpr3q0OTT0vBdVze6d3tmxRy1gO4Qgoj62raYv9HfqYcdeNG9XG3/fcc9WO8yKBSmP5crVtbXr3r6oKt3fmS1M9j/MbM947vsQasq9uV7U8wTu1eqv2Xnbt0rhxnASQF7ryb3t75xXM581TBczMHsbav/99tbcWLRj/dGprqjL27qIAaBKbV3XW1KRxykGD9PdKZ7Xkjek6zJD8L158sdp4nGbiRI3fcqp2oUClqz5573v1WTzWQ3/wesMYens7e85hHIOxd11/M5z3tFTjlfCxKWDDMIyMqNh9k3FWWsZMtm0LbdoT0cpYL+OQVLVVqdcBjmaGNid5e4q3IV7EOxjjkMzRZEwnr6qB0Le88zMeOXz40KTNSd49nFG1xguqtra1voWq24svDgr4H/9RbZ0XBozLUQHGGS1UB+zhTJ2qs7o2N+jzhobO2+SFzv7luRXOsaqqY4q2SeezMvbd1hbHHOnrQsqWCiyOST3f4/ev+8j7OUw/MUnk0wt9b5izBaM3P7nES10v+59rqEZMPDYxpPU1feD/2V9t1evGvfdqo0ceCW3f8h3sZcve8scU/n+AyvrYFLBhGEZG2AXYMAwjI8ourtODbwymMxzQ1rY3aSuiXa5jjtHJFJs2MWVkirfsqsUDbExB0VGOqqrpRftvbq5LWqYH/tIDUaWm1uaBtI+Z8jRypHadWNQFCN1nhmGeeYb+17DOZZepvy69NGzDMBEH6vgbcsAnHiDhtFB+Dgc76FumAMbdQ6bC9Ve68i8HaUJ6Xhjk5MAln7/u5wSE0EupgeQjUu/x/I/jNXQcP7u9aL95PYdZHuCCC9TOmKb/128X1LdDmN8IhH9c/iP7UfTJk7Vt9SANW+zpCHryNYwFAIwtaJrlhA4d2Lz2In1//fpjk7a/+pXa8ePVxxzUSxf9qQSmgA3DMDKi7AqYSoiWNy+Wihs2LAw0XHRRcZvt21WN3XknB92oBOL7BNXw0QDCRACmnFGkFWMyAAAgAElEQVQJA2HAiXR1R8tb4nrax0x5OvtstbEPOODI7/zBD+qXranRwR2u2BAXIOF+OcB2221qnePAXVytRH+8l14qLnLCVDjaOP2wv5P2L6mro+/CCfPBD5Zuy15C+N6xqqX6YroZJ8McmnofALSHJ6JKOO3fvJWl5Eoh/L+dMd4Plq1TRw0pMUL+wgZVujU1EwAASxbp6/z/7ujorCPZ81qwQKcZz1h+l77gL0Rf+MJXkrarVxdvc+KJatkTYgplJXxsCtgwDCMjyq6A0/FV3tA46eKss0JbxioZz2KiNcshhhhwDFOAjix6lSUTmfYEAE1NWqRD5JiiY6AqYawnb6R9zB4EJ0rEKV+8a19xhdr589Xydxm7VQuFvzr6jGSbu7xYoMJwjsnrr3obJ7FroHTz5pMQky53ObQ4s6dfk/YvyxQyBh73MNLpdeyNsC3T/4rTyRjX5XgGG1FihdS2SZO0V8gKllRjLDTF8z5v07sTgctAu5efrw5S9f/oo6EtH6dT+9I92niqcPq3mvHAA/rAn/gTvxzasjfBOD7T0tIF4CuBKWDDMIyM6LM0biqxMZEQoALmzampieX5qHIZ742WO/Xviaik4l2KSquxMc6Y0ORr3sFaWlSujBw5sCqyU81yinC8Omy6GM/x6+7TB6x675fFCUqteHuFyuzw9BtgNkVajTB+HKvFvMLvQLUbfyd+73The/bEqK5qa0OPLSgrPYdHjNA4LzNF4uVyQnaPWsb52Tth27woYE6+opKfuVCLNlHl8jxk1k78XksLY+XM5ElnjewJG/ksqZUrdYLWP1xzjb7sHRUX+6EP+fvyGHiMcfH2cmMK2DAMIyMqpoDTy3owjsZ4JRDuLKFUIe9wxVMMi4v1aKCTdy0qXyqQuroQn+zo0DzLoICL95r3aZz0Me/YzARhEXQgKKhkqaZW73Q/DH1fg+ZcfyUMCie/VYhvUnFM8Db+fVTZUdVx9J+/bZ59TP8e4dN1mdsbK1R+b44n8PtzEc1Sy9tQ6XL6LdswMyDOmeb+qHjT8cp0pk9eoN+YR85lgTg+E+ea85qxa5eea+3t7CnzvGSPNi5ipL2L5Bw+7zzfRNvEY0U8V9nLoOJmHDldzKucmAI2DMPIiLLrEyoe3lV4Z2Nhl3jUmGHIEGM52ltKVarZUGSdo8lUC+mYG/NagXA3pUoYM2ZgFDDpKp+ZCi2O5zLOPmrY20UbvTDxQwCAu27Ql2PV3NCgPRERLtUysehzY0VApcIyjIUCZ3XlKO0hRfp7phfEZAYCEJQvVSvPac4mZK+EvQognO/8X2DvjTbOKWYbCjj+5uliV3mDqvbJJ9X+8IdqmZMb/4+y58piU2FsiIFvlgd9LfoEHa9IeiuphPg4js85CvztCHsZlfSxKWDDMIyMsAuwYRhGRpS9M87ANbsNfJ4uzgOEbgi7ZJMna5d3x47igZ14UIJdB3b9liwp/blA50r2DIewSE9eQxFpn7KrzK5rqYGfPYOGAAir865dpq+z29XUFBeL0e6bc+pMrhbLgj1x6IOhn/C7Di06xjz6mMfO7ivPI/q1KpqHwnOKv8EJWldqn93WxYvV8rznOX3vvcX7BIArr1Q7ZfSbAIA38W4AIazGQay8wfNlrZ/dzpANB8Kc+3Pc2tu9qedpG/LFZs/W2JvPskxyyvZMPhlAGAAFQoiJoTueszy34+tPuTEFbBiGkRFl1ye883NAiNP6GNCOB3DS1fC5Le88VB7xNmybnia4fr3atWvDRIw4JQ0IyjfP6gzoXIqQye1UxPEAA1P8qN7GjdMyfRx042Dmgw+GQbOwcnXxhBUqtlITBe6+W+0RRxRtkksfp9Ur/ZuedAGEwWWqJ56zPMc4ASbeJj0Vns9//nP9Zzn//PBbJEqtVQ9qkP8dqYB5TuelGA/heUl/0QfOpVczjuEErfSEDKZFTklafu5zao87Tu07I1X5rvN+Y28DAO65h70/3W97uw70Dx1a+UF7U8CGYRgZUbZrO0u3EaZ2pKe1xilSjAMx7sPnjGVSabEQCRBUGJOlGcMM63UF+dLSovKkpqaq6Fi4j7yphrSP6R9O7+b3i5PMqSx4xw+pPFQPVFthLuuWLdqGhfIZA+PUzLhWNtVBerouyZOP0/6l8mWPIF3QHwg+5zlMRcx4ORVeKDQVClLRr1SzdXX6W7BwEgAMKWjsl93AF1fpUxa+z1ORI6BziUcW5y8UeD7SxtOKqYb/lHpvXJEVuTDZgmshjtrxsj7wPb5du3Tqc3HKGS9KQ/1+VPnyt6zkOWwK2DAMIyMqVo6SZfmohHnHi4tgOEfVpbcaFslhzPdjH/PvhlWGEqVxzz28Uz5ctI9SBWMGD9asirzHfgl9TCXPSRDsOTRHMzKbmn7vH/FFSlRWRXqft3ERI+2KbNo0xu9ffcp4Y6xy09M1S00myBv0L1eBprpnD43jGUA4n/m9+d7c83T6/MsbVOPEmSncD3uD7OEtWODtJdHU+0dV6v73oLkAgFtv3d9v0z9JXyf27lWZ2Z4slR6XH0hXGuL/OP/ntafGjCgAmFKvE49ebVbFO6GgSpgZKMuWxVkW/N/Q6wR/7764TpgCNgzDyIiKTUWmZcxn0ybexZ6OWjNArFKuqUkV17hxOprJ6Zuxorv5Zj7yiazJ3bFE8qu/UzJ2lz62vMLj5wg51SeXx7n66rj1S95SLbBwOuNnu1IWCNkPGlffuFH9WGrJIypdqnHGnPPs43QxeSp+qt34fOQUYfo8KTblpdbxfi54zUWhHCX3zzguf8dk2yhZ/ukRqnxv+3rxW3EmSh5J/y+GuDrjveuj1j6lKsnzjbrEAIYNU8fxtwCAFxo0733RIn1+2mmqhEMPPJqQkChq3T99awrYMAxjAGMXYMMwjIyoeAiCtr2dXYu4YlFrymoXgF1cBtXjtKq2tqX+EfuBO1HMqOTR+PE1Rftj1zHP3WOgcwgive4bu10AsGoVk9M5KYXdt3S6T9yt4zb6uzDF7/TTtasWT83kb8RBJnbXS02Hzgvpc3eVT/0q+C+3ZcsLSdva2lMBhO7vhOH+fPSjcXuGa+hhQsfbyTZvQ7vH9CMtz/MHHpiQtOVqMQw99OUAUSXp/joRT8TggFzp+d1f9+GZQ6KBu7vvVm3JdNUweM80zDi3TEOfI0fq/0hfDtabAjYMw8iIsl3jQz1YtVRAoZYn00vidCeufswguM5j5d1q2TKqtP+ItuGdcay3TKfSbYcNC3NhqSw4QJTn1Cigs4+pmKi+hnSo+rr55pCKd801c4rapldh4MBPPLD24INqt21TX3K1DE744OAREAZPOJkgz8qsu3M4DGiGAZxlyzR5/4YbNBWq4fLD/bZqw6SiIck29BE/73vfU/t7nzEYD/IxvY1TavPsX6C31wn24jghYw6AcA7+9K6gJznVme+FVZY5bTlcJ0T0xOY0+r70sSlgwzCMjKhYOUqmcvBuXihwzuSRUeviOG5VlcqxZct49/udt3EiNlfN8JVMfOySqShxkny6jFze1QOhj6lqqZImTlTVFU8VpjJl+g2T//m7UCFwogwATJqk9vOfV0t1zM+LexJppVFTXL8nl3R/DsfTZB8DAKxatdzb9FRvxhqjxRBxrLc8/+tSNsSAJ03S7TkZaaCdw/t3naDyVflcVaUnKHu48TnMc5WTXPjetGmqotesCb8HrxNZ+NgUsGEYRkaU/VrPGE96umxLC5VrOmsB4Kh7GAHdmbLHRm011jZsmE4o4PpcVG1xLJOsX9/5tTyT9jFVLUfM4/Kdx7bqxJdj/YtDT9HY+W9+o+9zdD2eXsxiMenC1FQRcVF9MhCUL9m/c/gVbxlb3Jtqw3h8POrOTCCOY6jiHTZMJVh8DvflpIC+ZP98TJ9y+rCeoDfeqM84QSg+h/k/QAXMsQ62if2ZZYkCU8CGYRgZUbFrPsvk8c5TW6uxl61bT41a6ePTTtNnzz+vdssW3ukYxD0h2WLaNFUJLPLD5WEYB92X2uVdd6BAH/POnV7VFQDOO8/724fRONqcXsopLpLEx+xdsKh+egXk+BgGIj05h5uaTvGPGBhPL5PD0fY4Bqy56lwwIL0010A7T/fF/vi4rk57EVxBecH4FfqgoN2EmTNDQXZmV7A3wSnyPLe5WnXWmAI2DMPIiIpHPajOGMviXR4I6isdtx006ISi5/GsqrTyzfOMq3JBH9OfzHwAwqKHLCqeLhfJ3+PEE8M2VAnPPlu8/4EWh+wp+zqHGdttbWUOu9qOjlOKto3PU6a62jkc2JePa2vVx4z1LnjrJ/pgyWa1Po1h3PyggAmVL/834hzr/oApYMMwjIywC7BhGEZGVLxTyUIxpdZVYleM6U3pteB6ktqUXh35YGRfPibp9fYI03JKpZal2+S9Bu2B0hP/MowQrxcH2DncU3ri4yTNLC78C+C5Vk3ja40mYnS1vuS+zvMsMAVsGIaREf3inss7f09WK+a6ZGnVkKfVd7PAfFxZzL+VgwPvnHD0mW+o4o1TJ9Nw0Dld6rOpqfzH1xtMARuGYWSEOOd63likCaEixsHA0c65kX35gebjynIQ+hcwH/cFB+Tj/boAG4ZhGOXDQhCGYRgZYRdgwzCMjLALsGEYRkYc8AVYRL4jItdEzx8SkTui598WkWu72ceKHnzOZhEZUeL1OSIya3+Pu8R+fi0i67pv2ffk3ccislREXhKRNf7vyO636lsGgI+rReTHIvKyiDSIyEcOdF+VIs8+FpFh0fm7RkSaReTWA9lXKXqjgJcDmAUAInIIdE35qKQLZgHYp9Occ725gM7h5x8oIvJhALt6s48Kk3sfA/iYc26q//tz9837nLz7+CsA/uycOx66ssH/9GJflSK3PnbOtUXn71Rodsd9vTiWTh9wQH/Qcv5b/OOToEsXPwxd2OowaIHUav/+lwCsAvAcgK9F+9jl7SEAfgCgAcAjAH4L4GL/3mYAXwPwNIC1ACYDqAewHcA2AGsAzAZwCYB1AJ4F8FgPjr8WwOPQk3bdgfqhkn8DwMdLAZyetR8HuI+3ABiatR8Hso+jYzje+1vK5ZsDngnnnHtNRDpEZAL07vIEgKMAnAlgB4C1zrk9IjIXwCQAZwAQAL8WkXOcc49Fu/uwd9QU6Gp8LwL4SfR+s3PuVBH5LIDrnHMLReR2/6PcAgAishbABc65bSIy3L82FsAdzrkPlPgKNwH4NoC3D9QHlWYA+BgA7hSRvQB+CeDrzp/J/YU8+5jvA7hJROYA2Ajgaufc6+XxTnnIs49TXArgF+U8h3s7CLcC6lA69Yno+XLfZq7/ewZ6Z5oMdXLM2QDucc6945zbDuAPqfcp+VdDnV+K5QAWiciV8EVanXOvlXKoiEwFcJxz7v6efc1MyaWPPR9zzp0EVR2zAXx8n980O/Lq40HQpTZWOOdO9cd9S3dfNiPy6uOYSwH8vJs2+0Vva0EwtnMSVNJvAfBF6Ip6d/o2AuAbzrkf9eJz/uLtXnRxzM65q0RkBoALAawWkdOcc290sb8zAZwuIpv9/o4UkaXOuTm9OMZKkVcfwzm3zds2EfkZVNn8tBfHWCny6uM3oD04XnTuAfCpXhxfJcmrj/XARE4BMMg5t7oXx9aJcijg+QDedM7tdc69CWA49ALHoPpDAD4pIrUAICJHlRgNXw7gIyJyiIiMggbNu6MNwDA+EZHjnHNPOue+CqAJwPiuNnTO/dA5N9Y5Vw+9o77cTy++QE59LCKDOCItIlX+O/TLbBPk1Me+K/xg9DnnAnihB5+ZBbn0ccRlKLP6BXp/AV4LHdFcmXpth3OuGQCccw8D+BmAJ3zs5V5EzvD8ErqK4QsA7oJ2P3Z089kPAvgbnxoyG8C3RGStaErZCgDPishYEfltr75h9uTVx4cBeEhEnoMOfmwD8P96+qX7mLz6GAD+CcCN3s8fh6rK/kiefQwAH0UFLsD9phaEiNQ653aJyBEA/gjgLB/jMcqE+bjymI8rz0Dycb+oB+xZ7EckqwHclFeH9nPMx5XHfFx5BoyP+40CNgzDONiwWhCGYRgZYRdgwzCMjNivGHB19Qg3eHB9hQ6l/7F792bs2dMsffmZ5uPyMmLECFdfX1+p3eeS1atXN7syrpBhPu5MT328XxfgwYPrMWvWUwd+VDljxYrT+/wzzcflpb6+Hk89dfD4syeISFmXCzIfd6anPrYQhGEYRkbYBdgwDCMj+lMecCf+4md174oq9g7q4ohra9Ueemhlj2mgYT42jOwwBWwYhpERdgE2DMPIiExDEOz+dnSoZdeXz9kt3r07bDN4sNpRo9SO9IkeTU3F+4z3xy4z30u/PpA5EB+/4Qvz1dQUbzN8eLGN3zuYfWwYB4opYMMwjIyouALeu1ctFRctENRXc7NaqtgxY9Qyt3vGjLDNSSepPe00/960PQCAh5dWAwAWLQptW1rUxqp4IHIgPi4U2n0L7yTEq9hQtnKB2ToAQFVVlb4arTv7rnepHeaLBnKgzjCM7jEFbBiGkRF9FgNm3DBWSIwx8rWzz1b74eYf64PbblO7qS3ZZuvtmwEA47waQ7squbmjR6v9139N2n518ycBACt9Cejt24uPhZ8fk+eYZaGglmo3ftzSwhVXqHhf8XaTt7EC9pIa47w9HADQ3q7St7FxXNKysVEXLBBRlTxxor7uf44B52PDKCemgA3DMDKi4gqY8ULGCuOaHXx85plqJzylawvu/Pu/BwBwbkA0RwAUdzu98p0ye7a+0OZVcjQn/V8WTgUA/Pr0UwEAN96or7e2qqUSjice5FGdMebLmPfmzeG99vad/tFWbxu83eCtl80YGu2Ry3DVeUvFO6JE21cBAM69BQBobdW26d5NrITz6GPDqASmgA3DMDKiYgqY+bpUuYwNDo3E07PPql1Q5VfV9jHfnSjm+EsuCY8pW48+GgDw+sKvAABG/emP+vq6aOHd228HAHzo8ssBAA9MPQcAsGZN8THFWQPMEsgDzO6got+4UW17e7zCNvsP7Dsc7a1PpE7UbJTagD3e/jm1bWNqGwA4oeiYmppUUQ8apJI3nUtsGEbAFLBhGEZGVEyXUJVxNHzOHLWxQH3ve/0Dr1Rx1VUAgJo//AEAMOKyywAAO2//WbLNwoVqz/NlZEc/qXb37jMAAAsaHwkf0ODjnT4ufMMNqoDvuENfpvJtC0kWuVLAzHpI+7qp6YioDdXs4d6WzvEFVpf4BJY0rU7tY0rUhvthFoV+XkeHSl9TvobRNaaADcMwMsIuwIZhGBlR9g4iB4Y4AYDdYw7GxIVcxjG76W//Vu3kyQCAEV/+MgBgz43/BgDYEIUtLr1ULQfQNvhsqve/3zcY8bXQmDM7/EHwGObNU/v442rXru3JN+s/pAvenODHwV5/vXPbrVt1XrdzHEhjyOEtbxvSmwDgNOXBqbacoFFIWo4ZoxNiGjk+598rFDRcEQ9wGoZRjClgwzCMjCi7AqbiTU922LJFLUUpEKmj2ukAgG/efzwA4J8uYiqTvh1P3uDAE9Xs/Plqq+fPLX4BAK6+Wu3MmQCAW27Qp15ol5wmmwfoW/Yg0j6Jv5dzL/lHVLFMUWOyHwfW4tQyKmDWqHy9qG1V1TFJyzDIRmVdSL2u2OQLw+iMKWDDMIyMKLsCpqqlCnvxRbULFqg9pOGFpG31Jl8IZts2AEBdnSpgSt5DWt8EAKxf/+5km1kz39EHnE3xSGPxB0+dGg7Gy+/PfHEIgM7x6TglLk9wei+tn5WdlJ5cv749ak3ly6nHvohRyWnFhOr4jZQ9D0Cxwg4TaxgEVpXMAu+WhmYYXWMK2DAMIyMqpk84yYLF1R96SG19fUjiP2e2D2JecQUA4NPX+OBss1dnixcDAGbEQeBv36v2EZ1wsccHRKsZbPaTOQDg4WYtwvP888XHxhgqy1OyUFBeiDNJgLAsE7MgWDgdANrbmf3A8pP+B+mUDfEWApyAwfgxP1CL9MQK+I141jMAZkqwQLthGF1jCtgwDCMjKqaAGV9lDJjL5iTTjwE8dbbGC69lagQDxSn5tHn9+uQxdRwjmIwOj/vCFwAAb5730aTt9RqyTHKFw1RdtVSOeYUxYJb8ZBg8XmCzoSFdbpLpCCy0w8axlKUC5jYzvdUPOiLMdEZDQ1oC1xU9sxiwYXSNKWDDMIyMKLs+SS97zuLgjBuuX783afvoo6rGrmm9FgCw84tfBAA87oOzVLdrov1T+TJ7dZyXtU9e8FUAwPn1oS2zAhgzZeyX5FUBb/W11Zn9wO93ui9QFBdkB6Z7y9gvlS8zHXxWCSLZDL/yaVJqUlVtTY3GlllqVGG8mcqaxX9ymmRtGH2IKWDDMIyMsAuwYRhGRpQ9BMFQA7v/7e0acujoYBc1rHcxfLh2bVkU5xy/vtvWZcsAAFzdLb5LnOrt6ePH64Mf/hAA8J//qU/b2sIkhLq6Kv/Z+rylRQekxo/Pd61a+pgrirzlM8gYgti1q6i1t756URK8aUQxcchgaMrqe4WC/pbbt8fzivl7Ft/LbSUMw+geU8CGYRgZUXZ9wsIwVJ11dYcWPW9rOzxpe911as+p15V1OXpErUZNFq+/cAZT1Fhox+e3+bkcaGgIkxCYfsZBKw4UcfpsXtVZ2sdMq2PqHyeYKDtTls54NbXXMehMe+q5Su3Gxnj6MgdVOZinq2fk3ceG0ReYAjYMw8iIsukTFrohLJiejgkvXBjih5+9WFOiOkbpSr3UZLO8HUK1+41vhB37wj2JDPR1DjnJIp6mS/UVSiGqOq6rS7+eD9I+5lp2VMJMq0sK3QNoa1PfFgo+YJys73akt4emLBDUMKcxp5XwC9Fjvqc9GxH9wfPqY8PoS0wBG4ZhZETFylF2VSw8rpeOGTMAhEKJjA4PuesuAMA/P/8xAMC93w2b+Po8OP6qv9IHvlAP455x3Z6VK9VSjXHqbt7jkvQxJ0Swd8GYd2NjIWrNmC/jtoyRUyazbZggE5QvYbeixduXovc49VhVM/2fdx8bRl9gCtgwDCMjKjYVmZYKmMpowoPfT9q+4rMeJjOAe8stAIBP/l6V7513Mlc1TEZeulRX3zz+KZ8l7NcXerz1EwCKSyWyxk+DX3dyoCjgtI+piBuTlTFXRa25nNAkbxn7ZfnJtNoFQo4w48WcvlxKLbPfopPEGYPPu48Noy8wBWwYhpERdgE2DMPIiIqFIBgKYJf0wgt9g//4TdI2iRZceSUA4F82aujh0Uf5BitrjU22+fQ8TVbb/veagzV6zhwAQIcfa4pTsFgnl9GKUqsG55F0CIK2vZ2DZK9HrVkCjgNnnJDReWp4gGEJOpP7Ze3feNKGthk5Ugf3bAqyYfQcU8CGYRgZUTadcthhatNpZ1xvzdfZAZ6cnGwzllV4/DLFF/ytPuWgWU2NTiK45Zajk23ePEIABF03eqau1jBiqT6Pa/7yGDgZIO+qLO1jDiq2eIHazgLBRRMnWOeX6Wdc7aIFxcQzJqpTloNu6RQ2QETVMFfJyLuPDaMvMQVsGIaRERUrR8n4K1UaJwmMnTcvNL7/fgDA076kZJW3P/Bv+03xxzvDJqy0eI63e0ZPANA53Q0A7r5bLSdiDBR1lo6v+w4ECgVOtjgyas14LpUv47iMr7N3ERfYYTkkFv/kEhiMFx8TWvqmA83HhtEXmAI2DMPIiLLrFcYpqdL4nHHdifPnJm3H/u//DQA49VOfAgD82r/um4ILKMfj9Md7W/P887rNEr9fr8RCBkUoQ5n3rIc09CknYLC30dJCNRt7jPFbTklm2UhOL+Y28fRljavPnq3x3TXJPBhV1nEvwyZeGMaBYwrYMAwjIyqmW1b52bDpEoo+7AsAmD79kwCAE+9Wu+AkX+bwhhvUeol3MqcqA3ju6h8DAD7zveL9Fq8ErAwd2vm1gQS/H3Ofa2s1S2Hr1lOTNk1Np/hHTA+hEuYUYlW1IsFZLJTP/bIHQcVNBW4YRu8wBWwYhpERFY/cpQvG7I3quLBc5BIfx/1y8xTf9r6ibWs3h23adb1OVPmUVGZZpGPOBxP0E+OxjAkrmt/b2qqJus3NahkzJ3GcfOrU0u8VL/ZpGEZvMQVsGIaREXYBNgzDyIiKhyA4DXhfa4MxjMAwBelJ+hi3OZjToA7Ex5y8USq8sMyHeTi5wpdcjtLRDMMoB6aADcMwMqJf6EaqVw4e7UvJcRAvrXxt9d19sz8+3rhRrfnYMCqLKWDDMIyMEOdczxuLNCFUdTkYONo5N7IvP9B8XF4OQn/2hLL63Hxckh75eL8uwIZhGEb5sBCEYRhGRtgF2DAMIyMO+AIsIt8RkWui5w+JyB3R82+LyLXd7GNFDz5ns4iMKPH6HBGZtb/HHW1/mYisFZHnRGRJqc/ImgHg4wXev8+LyDcPdD+GMVDpjQJeDmAWAIjIIQBGADgxen8WgH3+8zvnDvifG8Acfv7+IiKDAHwXwPuccycDeA7A1b04lkqRZx8fAeBbAM51zp0IYLSInNuLYzGMAUdvLsArAJzpH58IYB2ANhGpE5HDALwHwNMAICJfEpFVXg19jTsQkV3eHiIiPxCRBhF5RER+KyIXR5/1eRF52ivWySJSD+AqAF8QkTUiMltELhGRdSLyrIg81s2xi/8bKiICrc34Wi98USny7ONjAax3zjX5548C+EivvGEYA4wDnojhnHtNRDpEZAJUJT0B4CjoBWMHgLXOuT0iMhfAJABnQC96vxaRc5xz8T/whwHUA5gCLVD7IoCfRO83O+dOFZHPArjOObdQRG4HsMs5dwsAiMhaABc457aJyHD/2lgAdzjnPpA69nYR+QyAtQDeArAewOcO1BeVIs8+BrABwAn+Qr4VwEUIyywbhoHeD8KtgF4YeHF4Inq+3LeZ6/+egaq1ydCLRczZAO5xzr3jnNsO4A+p9+/zdjX0IlKK5QAWiciV8DUYnZmR6ZUAAAG9SURBVHOvlbgwQESqAHwGwDQAY6EhiC93/3UzIZc+ds61QH38CwDLAGxGWB/JMAz0fioyY5QnQbvHWwB8EbooGdcyFgDfcM79qBefw3U19qKLY3bOXSUiMwBcCGC1iJzmnHujVFsAU/02GwFARP4LwPW9OL5Kklcfwzn3IIAHAUBEPg27ABtGEeVQwPMBvOmc2+ucexO62uOZCINDDwH4pIjUAoCIHCUiR6b2sxzAR3ycchR08Kc72gAM4xMROc4596Rz7qsAmgCM38e22wBMERHOVDkf2iXvj+TVx+AxiEgdgM8CuGNf7Q3jYKO3F+C10JH5lanXdjjnmgHAOfcwgJ8BeMLHEO9F9E/t+SU0TvgCgLug3egd3Xz2gwD+hgNEAL7lB5DWQS9Mz4rIWBH5bXpD59xrAL4G4DEReQ6qiP9tP753X5JLH3u+KyIvQC/+NzvnXu7ZVzaMg4N+MxVZRGqdc7t8+tIfAZzlY5VGmTAfG0b/ol+Uo/Qs9iPr1QBusgtDRTAfG0Y/ot8oYMMwjIMNqwVhGIaREXYBNgzDyAi7ABuGYWSEXYANwzAywi7AhmEYGWEXYMMwjIz4/yX7jtygnqx/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd216e500b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 10 optimization iterations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have already performed 1 iteration.\n",
    "optimize(num_iterations=9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 66.2%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecVNX9//HXBxFBsKBYiAprvihgiSUI2I09mthi1K8aS0RjCXZJYixYfwa+ImpsWKKxRINYwYbGEhU0IEhV0UQRFEUFFSMWOL8/5n7m3ll2d+buTl3ez8eDx87cueUsZ++Zzzn3FAshICIihWlT6QSIiNQSFZoiIimo0BQRSUGFpohICio0RURSUKEpIpKCCk0RkRRUaIqIpKBCU0QkhbYtObhLly6hrq6uSEmpDRMnTvwkhLBWpdNRLsrj1k95nE6LCs26ujomTJjQklPUHDN7r9JpKCflceunPE5H1XMRkRRUaIqIpKBCU0QkBRWaIiIpqNAUEUmhRU/PRUQa88033wCw3XbbATBp0iQA9ttvPwAeeuihyiSshRRpioik0OoizQULFgAwe/bsRvfp3r07AFdddRUAm222GQAbb7wxAFtssUUpkyjSqnmEecYZZwAwefJkAMwMgB//+MeVSViRKNIUEUmh5iPN0aNHA/Doo48C8NxzzwEwa9asRo/p2bMnAO+++y4QfzO6pUuXFjmVIsuPa665BoCbbroJgN122w2Aiy++GID+/ftXJmFFokhTRCSFmog033nnHQCuu+46AEaMGJH97OuvvwYgzVLEb775ZhFTJyJJH374Yc773XffHaj9CNMp0hQRSaEmIs05c+YAMHz48Badp1evXkD8tFyqz9tvvw3AJ598kt324IMPAnF7dZs2me/6E088EYj7AQJstNFG5UimNGHRokUAtGvXDogjzdZCkaaISAoVjzSTEYVHkjvssAMAe++9NxB/Y6222moAdOrUKXuMf6vttddeQBxF9uvXD4Ctttoqu2+HDh0A6NixY5F/C2muqVOnAnF79QMPPADA/Pnz8x47fvx4AFZcccXsNu8Z4X9DV199NRD/DUlpfPDBB9nXt9xyCxDXALbeeuuKpKlUFGmKiKSgQlNEJIWKVc+/+uorAPbYY4/sttdffx1YdiD/tttuC8QD/pPrmfhwyfXXXx+IHxJIdZoyZQoQV8fvu+8+AD7//POc/Tw/AXbccUcgzvehQ4cC8XC8V155Jbvvp59+CsBjjz0GxENi/aGRlMall15akvOOGzcOiB8GJ3ne+vDnclEJIyKSQtkjzW+//RaAww8/HIijS4Bzzz0XaLyLQkMr5nXr1q3IKZRi+81vfpN97d2H6j/o8TzffPPNAbj88suzn7Vv3z5nX48+brjhBgCOPfbY7Gc+OcS6664LwMknnwzAL37xCwDWWmu5WWSyrMaMGbPMtgEDBqQ+z0knnZRzPp+A57///e8y+6666qoAnHnmmQCcf/75qa/XHIo0RURSKFuk6V2DPILwCTaS3/znnHMOACuvvHK5kiUlsHjxYgCGDBkCwM0335z9zIe7rr322kAcWXjeF9IdzNstv//+ewAuuuii7Gfe9cwnY5HS8gjwu+++y27z9uhjjjmmwWM831577bXstgMOOACAefPmAfHfiZcPydqnH+fPM3xikKOOOgqIp34sFUWaIiIplC3S9CfiV1xxBRB/G/zzn//M7uOd16W2+XBHf8qdnExlvfXWA+JO7H379s17viVLlgDw/vvvA3FEse+++wJxu1dDfvWrXwGw+uqrF5x+KZx3ZP/oo4+y25Jt2EneAd4n3LnkkkuW2cf/PjzfvE062ZvC+bIZ3v7pE4Uo0hQRqSJlizRffvnlnPc+vLGhbxCpbd5mtcIKKyzzmQ959L6V999/PwBvvPFGzn4+5BVg5syZOT+7dOkCxO1fDVlnnXUAOO+883KuK8XlfaeTGps0xfty3njjjUC8/AXEExUPGzYMKGxSnR49eqRLbJEo0hQRSaFskaZHFO7xxx8Hcp98ehtFcpINqT0eNfzkJz8BYOzYsdnP3nvvPQBOPfXUBo9t2zbzJ+nRakPqR5jJUWAHHXQQEC+50LVr11Rpl3SSE3U05q233gLg3nvvzdl+wgknZF+3ZGIVHxlWrolBFGmKiKSgQlNEJIWyVc992Jw3/voKkMnquTcU++QKPiemdzXxht9NN910mfNPnz4diCf30AOmyvGHOD5kcuHChdnPvMvZSy+9BMCaa64JxMNh/e8iObw2OSFHQ5JdXHzwhLoYlccXX3wB5HYrq79e17XXXgvEfwdHHHEEEA+DbS4fMONNOuWaM1WRpohICmWLNM8++2wArrzyykb38U7MPm2Y/0zDh+ftsssuwLKNz1J+yajPI818vAM7LBtp+kQN3j0lOVyvoW5OUjpec0x2H0q+hvhhkW8v5OFRU/x471jvk7GUiyJNEZEUyhZpeoRxyCGHAHG7RnKgv0806hFnc3z88ccAjBw5EsjtJOsdnaV6+SQfTdUQvC3MpxeU6ubDJn2Ai/9MTv/n7dLext0U71bmE/ucddZZxUtsARRpioikULZI09uattlmGyDu8Jr0zDPPAHH0OXjwYABeffXV1NfzJ3gTJ05MfayUn7dPeQ+KZA3Eea2h3G1YsixvV/RJMpri0aNP6eaDWJKTBj/55JMAjB49GoBVVlkl531yOQ0fuuk1x/79+zfzt2geRZoiIilUfN3zJB9+53zpAo80fdKF5PIGxx9/PABXXXUVAPfcc0/J0ynF43nr7VJffvnlMvt41OFtmSuttFKZUieN+cEPfgDEi5r58FiAf/zjH0DcTultjz6k9V//+hcQR5EAvXv3BuK+nP734DWQ5MTkHmGWa3mL+hRpioikoEJTRCSFqqqe17fnnnsC8SqV/nDAuzAAzJo1C4hnC6/PZ4KW6uRrRflwPJdcK+iRRx4BYIcddihfwqQgt956KxDPog/xTOp+//pqkfVnnEoOWvDuR77NH+T27Nkz53OAAw88sHi/QDMo0hQRSaGqI01vHD700EMBuO+++5bZ59lnn81574P3/ZvvT3/6UymTKM3kD3y8M3t9Rx55ZPa1D4mV6uMT4zzxxBPZbT6Pqq9P/8tf/jLnGI8i6w+3TPKHvf73UUin93JRpCkikkJVR5o+xdjw4cOBODpJdlj3VfDq6uqAeKIH7xgv1cWn8/JaxLfffpvz+RZbbAHEeS61IdleOX78eCCuGb799tsA3HzzzQAcd9xxQO6M+84/69WrV+kS20KKNEVEUqjqSNP5yoLeGfbOO+/MfubtJh5Z+tRwUp284/PcuXMb/Nyne2vfvn3Z0iTF5VMB1l//fOjQoZVITtEp0hQRSaEmIs36fvWrXzX4WqpfY0PfBg0aBMCuu+5azuSIpKZIU0QkhZqMNKV2ffbZZznvvQ369NNPr0RyRFJTpCkikoIKTRGRFFQ9l7LyyRv8pz8Yqj+Zg0i1UqQpIpKCIk0pqzPOOCPnp0itUaQpIpKC+TRNzTrYbD7wXt4dW5fuIYS1Kp2IclEet37K43RaVGiKiCxvVD0XEUlBhaaISApNFppmtqaZTY7+zTOzuYn37UqVKDObY2ZTo+u8UsD+A8xsfrT/TDP7dQuvf5eZHZBnn85mNsbMXjez6WZ2VEuuWSkVzOMzo/+36WY2sID9y57HiX23NbMlhe5fbSqYx2uY2QNm9kaUZ33z7F8T93GTXY5CCJ8CW0YnHwwsCiH8X72LGpm20aX5LpbSjiGEhSn2vzuEcLqZrQtMM7NHQgifJNLZNoTwfRHTNxCYHELY18zWAd4ws3uKfI2Sq0Qem9mWwNFAH+B74CkzGx1C+E+eQ8udx5hZW+ByYGwxz1tOFbyPrwUeCSEcFBXOHQo4purv42ZVz82sh5nNMLO7genABma2MPH5YWZ2S/R6nejbZoKZvWpm/ZtzzUKFEOYB7wLdzOxSM/urmb0E3G5mbc1sWJSOKWY2IEpjGzO7PvpGHAt0KeRSwCrR607AJ8CS4v9GlVHiPO4NjA8hfB1C+A54ASh4XdYy5jHA6cC9ZPK3VSllHpvZGkC/EMLtACGEb0MInxeatmq+j1vSptkLuCqEsAnQ8DTcGdcAQ0IIfYBDAM+EfmZ2YyPHBOAfZjbRzI5Lkygz6wF0B/6dSOduIYQjgROAj0MIfYFtgFPMrBtwMLAhsAlwLLBd4nyXmdk+DVzqamBLM/sAeB0YGFpfV4RS5fFUYOeo+tYR+CmwQaGJKlceR8ftC9xcaNpqUKny+IfA/Kiwm2RmI8xs5UITVc33cUtGBL0TQphQwH67Az0tXq6zs5l1CCG8AjTWXtk/hDA3CtHHmtnMEMLLea5zhJntAnwDDAghLIyu+XAIYXG0z55AbzM7LHq/GrARsBPwt6hqMsfMnvOThhD+2Mj19gFeBXYGNgaeMLPNQwiL8qSzlpQkj0MI08xsGPA0sAiYRGFRernzeDgwKISw1JpYbrbGleo+bkum+WUgMJFMVf0c4KI816n6+7glheZXiddLgeRfVXKBFwP6hhBylx1sQghhbvRznpk9DPQF8hWad4cQGpqUMZlOA04OITyT3MHMCq4aJhwLDI6+ld40s/fJ/Ke/1oxzVatS5vEIYASAmQ0B3i7gsHLncR9gZHTTdgH2NLMlIYRHm3GualWqPJ4DzPYC2cxGkWnqyKfq7+OidDmKSvYFZraRmbUht33qaeAUf2OZhwCNMrNOZtYpet0R2AOYFr0/zcxObEFSnwROtkzjPmbW08w6kGlTOzRqE1mPzLdOPrOB3aLzdAV6APkeZNSsYuZxtM/a0c86YD8y7YZVlcchhG4hhLoQQh3wEHBCKyswcxQzj0MIc4CPomo2ZO6VGdGxVZPHNOM+LmY/zd+R+WVeJvMt404Bto8abGcAx0cJbKwtpCvwkpm9TiZsfjCE8HT0WW/g0xak8SZgFjDZzKYBN5CJtu8n8583A/gLMM4PaKItZDCZdrkpZJ6snh1CWNCCtNWCYuUxwEPRvg8BJ4YQvoi2V1MeL4+KmccDgfuie2RT4IpoezXl8WBS3sc1NYzSzMYA+9datx4pnPK49av1PK6pQlNEpNI0jFJEJAUVmiIiKajQFBFJQYWmiEgKLVojqEuXLqGurq5ISakNEydO/GR5mtVbedz6KY/TaVGhWVdXx4QJhYzAaj3MbLlaFkB53Popj9NR9VxEJAUVmiIiKajQFBFJQYWmiEgKKjRFRFJQoSkikkKLuhyVy8SJEwF48MEHARg1alT2szfffBMAn3jEZ5b+8Y9/DEDv3r2z+/7hD39YZpsU36JFmUmv33//fQBuuOGGZfb59a8zCw1uuWXeqTdFqooiTRGRFCoeaY4YMSL7+o033gDgn//8Z84+Hml6FJmczs63/eY3vwHgwAMzk03vueeeJUqxNMYjzKFDhwJwySWXNLrvjTdm5q099NBDAbj66qsBWGONNUqZRJEWU6QpIpJCxSNNjxAhjhpXXjmz0qe3PZ5+emadpV69egHQpUu8nPFBBx1UlnRKfpdffjkAV1xxRZ494fvvM5N233333QA880xmjazbb78dUE1hefPee5lRjddccw1AdljnddddB8Bmm21WmYQ1QJGmiEgKFY80k5HiQw89BMQR5r/+9a+KpEmaZ8MNN8x57zWH3/72t9ltm266KQDffptZCfaCCy4AYN68eQDsv//+APzud7/LHjNo0CAgroFIbXvrrbcA+POf/5zd9te//hWAzz//PGffvffeG4DRo0dnt3mvjO7duwPwox/9qHSJbYAiTRGRFCoeafpTVIDXXsusz+7tG7NnzwagW7du5U+YpOb9aN0hhxwCxE/GG7LFFlsAcY3j008zK7tefPHF2X3eeecdAG677TYAVlxxxSKlWMph6dKlAMyYMQOAPfbYA4hrF02ZO3cuADvvHC9h/sUXmdWet912WwBefPFFANq0KU8MqEhTRCQFFZoiIilUvHq+1lrxjPPHH388AOeddx4An3zyCaDqea14/PHHgfgB0B//+Me8x+y4444APPzww0A81DU5wMG7JfmgBu+W1LZtxf98pQnz588H4NprrwWaHuyw+uqrA3HV26v0zrcn+WAY31fVcxGRKlRVX9X+jeERhTccJ4dN1ufdk9QdpfJ23313IO6o3qlTp4KP3W677QAYMmQIAPvss0/2swULFgBwzz33ALDffvsB8YMmqU5e07j55ptztrdr1w7IfUDo3dUGDx4MwPjx4xs9r9dOvXZS7hqHIk0RkRQqHml6uwfArbfeCsRtYkcffTSw7LRvycjTJ+g44ogjAA2rrCSP+j3SbMgtt9wCxFFjchht0uGHH5597UPpnHeOluqRbIM8+OCDgTgS9LZG74TufwNjx47NHuNDpb2dsilbb701EHc5KjdFmiIiKVQs0vQIc6eddspu807t9ScQ3mGHHXKOTbaReIf4Bx54AIijUR+CmZxwWO2epdWnT5+c91OmTAFg8eLF2W0+pNKHUT733HOpr+M1Ep/AxTtLA6y22mqpzyct5xNtwLKDHDyffv/73wPx/Zz8u8hn4403zr6+6aabmp3OYlCkKSKSQsUiTW+78OUqAH7xi18AMHLkyCaPPeGEE7KvvS/nXXfdBcSTfmyzzTYAbLLJJtl9/bxa7qI0DjjgACCefGHXXXcF4KOPPsru0759eyCONJvDayT+9DxZg/BaiE/8odpFaX333XcA/OlPf2p0H7/XDzvssJztyQmnBw4cCMDTTz8NwEsvvZSzry+PAvFEHZWiSFNEJAVrqg9kPn369Ak+WWi18WU0ku2fHqH4yBVvO03DzCaGEPrk37N1KHYe+xRfHvV/9tlnAIwZM6Zo1wDYfPPNgXg0UZpJbJXHhVuyZAkQj+wCGDduXM4+HTp0AGCllVYC4nbtM888M7uPT/fWr18/IG7v7N+/PwBPPvlkdt9VV121WWlNakkeK9IUEUlBhaaISAqttnru/EERxN2bvEp4/fXXA+k6xKvqVlxevfvyyy9zticfHnk3srXXXjtnnwsvvBCI59kE+Oqrr3L28e5I/qCikHXWlcfpLVy4MPvam2B8eKP/n3vXI+erlwIcddRRQNxdaZVVVgHitYKSXY6KQdVzEZEyqfgwylJLrlzps8SfddZZAJx44olAPEO8D+WS0vHI34dC+kQdPjWYq/++IT7hg6+dDnDSSScBMHXqVCAequdD+fwhoBRXMr+OPPLIgo4ZNWpU9nX9DvGep8WOMItBkaaISAqtvk2zIR7teBund7D39rWmqL0rvUcffTT7+rTTTgPgww8/BODee+8F4s7oLeVtoz6pg68v5N1U/Hq+ymFDlMel5c8Udtlll+w2rxn4hOOzZs0C4mnkik1tmiIiZdLq2zQb4u2c3iG3kOmopPmST8Y9wvzmm2+AuOeCryjY0um+/KmrTz3nbaa+XII/RW8q0pTS+tnPfgbE0WXSBRdcAJQuwiwGRZoiIiksl5HmzJkzgXhyj+SkHlJ8yQmFP/jgAwAGDRoExBNKF9KenIZPS1d/gS6fCFfK79///jcA06ZNW+Yzjz6POeaYciapWRRpioikoEJTRCSFilfPr7rqquxrX2Wu0M6xafksR75Kng+5e/7550tyPVmWz4XqncyfffZZIB5G591QfJZvyN/BObmqoa8/8/bbbwNNr2Qq5TF37lwAdtttNyB+MOjdiyBeB2qFFVYoc+rSU6QpIpJCxSJNX9PHhzRCvDJhcyJNX3Oo/nCs5HtfT8gj2jvvvBNYdiIBKR3vZO4rFfqDGe+KdPvttwNx3kA8BLIxPnt4U/r27QvEXVqkfPy+e/fdd3O2J2djT0ad1U6RpohIChVv00y2Ofkqcz6Q3zs++z7eCX3NNdfMHuPdhhpbGz25HpCvjX7uuecCuZN5SHl16tQJiLuh3HHHHUA8zDHZ8dm7KaWx/fbbA7DXXnsBcPzxxwO5fztSWq+++ioQt1c7n8F9n332KXuaikGRpohIChWLND2KfOKJJ7LbPGp03h758ccfA3EndI8mIW4H9ajxwAMPzDlHsr1SKxNWr6OPPjrn57x587Kf+dNWX+/Jn7A3NEGtr/vkbWQe1Uj5eK8UnyQ6OUExQOfOnYG4tlFrFGmKiKSwXE4N1xKaNqz1Ux63jPe9Tq42CbDuuusCcR/dQpYeKRVNDSciUiYVf3ouIq2Lj+rxJTDOOOMMIO7B0LVr18okrEgUaYqIpKBCU0QkBVXPRaSoTj311JyfrY0iTRGRFFRoioikoEJTRCSFFnVuN7P5wHvFS05N6B5CWKvSiSgX5XHrpzxOp0WFpojI8kbVcxGRFFRoioikoEJTRCSFJgtNM1vTzCZH/+aZ2dzE+3alSpSZzTGzqdF1Xilg/wFmNj/af6aZ/TrfMXnOd5eZHZBnn85mNsbMXjez6WZ2VFP7V6tK5LGZdTSzV6NrzDCzvAv3mNmlibRNNbN9W5iGF82syWl2zKzOzJ43s0lRPu/dkmtWSgXv4zOje2O6mQ0sYP/auI9DCAX9AwYDZzew3YA2hZ6nwGvNAVZPsf8AYHj0el3gE6BLvX3apjjfXcABefa5ALgser0OsCDNNarxX7nymMyXdcfo9YrABKBPnmMuBU6PXm8GzCd6kNnMPH4R2DLPPrcBx0evfwS8Xek8qqE83hJ4HegQ5fGzwIZ5jqmJ+7hZ1XMz6xFFCHcD04ENzGxh4vPDzOyW6PU6ZvaAmU2Ioov+zblmoUII84B3gW5RdPJXM3sJuN3M2prZsCgdU8xsQJTGNmZ2vZm9YWZjgUIWDwrAKtHrTmQyeEnxf6PKKGUehxCWhhC+it62I3NTFdyNI4QwjcxN3jmKJm4ws1eBy82sk5ndHqVjkpn9PErjymY2MopgRgHtC7kUsGr0ejUg/WJFVazE93FvYHwI4esQwnfAC8CBeY7Jqub7uCVjz3sBR4UQJphZU+e5BhgSQhhvZnXAaGAzM+sHHBtCOLGBYwLwDzMLwPUhhFsLTZSZ9QC6A/9OpHOnEMJiMzsZ+DiE0NfMVgLGm9lTQH9gQ2AT4AfADODG6HyXAS+FEB6rd6mrgdFm9gGZG+vgEH1dtSIly+OoWvgq0AO4OoQwsdBEmdl2wOIQwmeWWfqkK9A/hLDUzIYAT4QQjjGzzsAr0Q30W2BBCKG3mW1FJrr18/0lSsPkepe6AHjKzM4AVgZ2KzSNNaRUeTwVuNDM1gC+AX4KvFRooqr5Pm5JoflOCKGQ6Z53B3pavK5PZzPrEEJ4BWisvbJ/CGGuma0LjDWzmSGEl/Nc5wgz24VMBg0IISyMrvlwCGFxtM+eQG8zOyx6vxqwEbAT8LcQwlJgjpk95ycNIfyxkevtQ+am3xnYGHjCzDYPISzKk85aUrI8DiF8C2wZFWwPmlnvEMLMPNc5x8yOAb4EDk1sHxnlHWTy+Kdm9vvofXugG5k8HhJde5KZTU+k5dhGrncEMCKEcLWZ7QDcGeVxa/pyLEkehxCmmdkw4GlgETCJwmpiVX8ft6TQ/CrxeimZ6pJLVn0M6BvdJAUJIcyNfs4zs4eBvkC+QvPuEMLpedJpwMkhhGeSO5hZwdWGhGOBwdEN9KaZvU/mP/21ZpyrWpUsj10IYYGZvQDsBeQrNIeGEIbnSaeRacd6J7lD4mZP4zhglyidL5rZqkBn4LPmnKxKlfI+HgGMAIhqAG8XcFjV38dF6XIUlewLzGwjM2tDbtvF08Ap/sbyP7HsZGadotcdgT2AadH708ysoep8oZ4ETvZqiJn1NLMOZNpbDo3aRNYj862Tz2yi6pqZdSVTzfxPC9JW1Yqcx2ub2WrR65XJRDFvRO+HeDtkMz0JZJ/URlVxyOTx4dG2LYBNCzhXMo83JfOgpDUVmDmKmcfRPmtHP+uA/YB7o/c1fR8Xs5/m78j8Mi+TefrtTgG2jxpsZwDHRwnsZ2Y3NnCersBLZvY6mbD5wRDC09FnvYFPW5DGm4BZwGQzmwbcQCbavp/Mf94M4C/AOD/AzC4zs4ZWtR8M7GxmU4CxZJ5ILmhB2mpBsfL4B8DziTweE0LwtZx/BMxr4JhCXQR0tEy3pOlk8gngz8CaZjYTOJ9MdZEonX9ppBA4g8zN+TqZJ7HHtCBdtaJYeQzwULTvQ8CJIYQvou01fR/X1NhzMxsD7B9C+L7SaZHis0wd+vEQQk32h5TC1Pp9XFOFpohIpWkYpYhICio0RURSUKEpIpJCi1aj7NKlS6irqytSUmrDxIkTPwnL0azeyuPWT3mcTosKzbq6OiZMKGQwQethZsvVsgDK49ZPeZyOquciIimo0BQRSUGFpohICio0RURSUKEpIpJCVRWaw4YNY9iwYZgZZsa4ceMYN25c/gNFRMqkqgpNEZFq16J+msU2fHhD88uKSC1auDCz3NCqq2aWWWrTpnXEaK3jtxARKZOKR5rvv//+Mq+vvPJKALbddtuKpEmq09ixYwEYNWoUAH//+98BWLAg/9zPHuW88kpmOZs+ffqUIomS8Mtf/hKAjh07AjBgwAAAfvazn5Xkeh9//DEAa6yxBgBt25ameFOkKSKSQsUjzZEjRy6zbf31169ASqQa3HfffQA8+uij2W2PPZZZddXbyHzi7I022giIIxiAfv36AbDZZpsBca3l5ptvBuIoVZFm6W299dYADBkyBICddy5kyZ7m82ci3333HQBDhw4tyXUUaYqIpFCVkabaMpcfgwYNAuDaa68F4JtvvgHiaBKgZ8+eAOy1114AnHHGGQBstVVmockVV1yx0fP37dsXgFmzZgFw6aWXFi3t0rQNNtigLNfxtu5hw4YB8d+QIk0RkSpQsUjTn5SPHz8+u82/mcr1DSWVd8cddwCwePFiAA455BAAzj777Ow+W2yxBQDt2rVLff6f/OQnABx55JEArLDCCs1PrKRy/fXXl+U6zz77LBBHmN6WWiqKNEVEUlChKSKSQsWq51ddddUy2w4++OBmn88n9kh2lofc6r93ttWDpuqx/fbbA/Dggw8CsO+++wKwzTbbFOX8//M//1OU80jhpk2bBsAHH3xQlus9/fTTOe8vvPDCkl5PkabE8eMJAAALWElEQVSISAoVizTnzJmzzLb+/funPo9HmIceeiiwbKSZ5NHtyy+/DCjirKS33noLgCeeeAKADTfcEIADDzywYmmS4vB78vPPP8/Z7sMpi8Uf/Hhn9g4dOgCw0047FfU69SnSFBFJoeKd21vKo8f6EWZDk354NHrmmWcCaILjCrrxxhsB+PrrrwHYe++9AVhllVUqliZpvkWLFmVf+73nvPZwwgknFPWaDz/8MACTJ0/OOf/qq69e1OvUp0hTRCSFmow0kxFi/WGYPuGDd5JO8jbThoZuSnl5hOk23njjCqVEisGHtgK8+eabOZ+V6mn2bbfdVpLz5qNIU0QkhZqMNBvq49lUhNkYn8Q2zTFSHGPGjAHiNswDDjigksmRZnrkkUeAhmtvdXV1QDzhSrH4U/mPPvqoqOctlCJNEZEUKhZpNjTRcEN9N5P8CXnyW83bKRUt1gZ/yvrFF18AcRTi2300SUN69OgBQPv27UuZRCmA598ll1wCLNsnE+JRXsXOr//85z9A/NTcHXfccUW9TmMUaYqIpKBCU0QkhYpVz72LQvKhzllnnQXEnc/r88+TfBKOQnj13ufrVJW+/HwIq1fnJkyYAMDmm2+e91ifqf0Pf/gDAD//+c8BVdcrwVd+9PxL8s7sheRpMa255ppluY4iTRGRFCoWaXq0l5ykw6dx87U+Gos4k/KtXOndipLnrz/MS8rHJ+bw9X5WW201YNkp3JIPBV977TUAJk2aBMQ1hP/93/8Fcjs5K+osrSeffBKAc889N2e7rwwKcN111wHxLPm+3tNXX33V6Hl9nSeffMP5JB9m1uixHtn+8Ic/zP8LFIEiTRGRFCreuT05/Mon1PC2S48MfZ80K1d6hJlca8aj2zTtoFJcHpH4lHCF8CGXM2bMAODiiy8G4G9/+xsAvXv3zu57/vnnFyWd0jCfJMOjf+fTtAFcdtllOZ8tWbIEiCdpaYi3V3ttwt19991A3H4N8NRTT+Xs4xN0NBWNFpMiTRGRFCoeaSafYHs71vDhw4E4smxqgg2fvMN/3n///Y0e409utdplbfHJZdddd10A3n333ZzP11prrXInabk1atSoBrfPnj07+9rbNNPwwQ3dunUDYNVVVwXgiCOOAKBPnz7ZfefPn59z7EknnZT6ei2hSFNEJIWKR5pJ/rTc2xy9D2dDE3Q4bwetz5/KJ5+eK8KsTc8//zwAp556KgBTpkwBYMcddwTiaERKz9uMBw4cmLO9e/fu2dfrrLMOsOzyFrvuuivQ8KJ5ffv2BeC///0vAF26dAHghRdeAODaa6/N7ut9Q7fcckug/NMKKtIUEUlBhaaISApVVT13Xo32Tu7+0xuJG1px0rsladaj2vLtt98C0K5dOyCunl100UXZffzBgneO9jz2vwutK1Q+J554IgD9+vXL2d61a9fs6zXWWAOAlVdeOfX5O3funPN+jz32AOKuTkne3dAHSJSLIk0RkRSqMtJszMEHHww0/GDIuyt5lyOPVrW2eXX58ssvAXjssccAeOeddwD48MMPARg9ejSQ263Ih0YOHjwYiB8YKsIsv7ZtM0VGQw9zSim5zpDXKpPRbTkp0hQRSaGmIk1vw0ryyNJ5e1e+iTyk9Hz4XHKClPPOOw+AXr16AfDWW28B8TC8Nm0y3+PJiVyuv/56IB5qJ8uf5ACGSg9mUKQpIpJCTUWaLhlxNhR9SnXwyYKHDh26zGdTp04F4inBvHOzPzXfe++9y5FEkdQUaYqIpFCTkabUhu233x7I7Vf73nvvAXDaaacB8XrnK620UplTJ9I8ijRFRFJQpCkls//+++f8FGkNFGmKiKSgQlNEJAUVmiIiKajQFBFJQYWmiEgKKjRFRFKwEELzDzabD7xXvOTUhO4hhOVm+UPlceunPE6nRYWmiMjyRtVzEZEUVGiKiKTQZKFpZmua2eTo3zwzm5t4365UiTKzNczsATN7w8xmmlnfPPsPMLP5UbpmmtmvW3j9u8zsgAL33dbMlhS6f7WpYB7PMbOp0XVeKWD/suexmXU2szFm9rqZTTezo1pyzUqpVB5H125rZlPM7KEC9r00kbapZrZvC6/9opltmWef9mZ2v5m9bWbjzKxbvvM2OfY8hPApsGV08sHAohDC/9W7qJFpG12a72IpXAs8EkI4KMrUDgUcc3cI4XQzWxeYZmaPhBA+SaSzbQjh+yKmETNrC1wOjC3mecupgnkMsGMIYWGK/cudxwOBySGEfc1sHeANM7un2H9HpVbhPD4TmAYUujTl0BDCcDPbDHjWzNYOiQcvJcjjE4B5IYQeZnYk8P+AI5o6oFnVczPrYWYzzOxuYDqwgZktTHx+mJndEr1eJ4oaJ5jZq2bWv7HzRvuvAfQLIdwOEEL4NoTweaFpCyHMA94FukXfXH81s5eA26NvvWFROqaY2YDomm3M7Poosh0LdCnwcqcD9wKf5Nux1pQyj1uqjHkcAF+9rROZfF5S/N+oMkqdx2bWHdgD+EvatIUQpgEGdI5qBTeY2avA5WbWycxuj9Ixycx+Hl1vZTMbGdVERgHtC7jU/sAd0eu/A3vlO6AlbZq9gKtCCJsAc5vY7xpgSAihD3AI4JnQz8xubGD/HwLzoxthkpmNMLOCF1A2sx5Ad+DfiXTuFkI4ksy3yschhL7ANsApUTh+MLAhsAlwLLBd4nyXmdk+DVynG7AvcHOhaatBpcpjyBRI/zCziWZ2XJpElSuPgauBLc3sA+B1YGAy6mklSpnHw4FzyOR1Kma2HbA4hPBZtKkr0D+EMAi4AHgiyuNdgSvNrD3wW2BBCKE3cCmwVeJ8f2mkqr4e8D5kAjTgKzNbvam0tWRquHdCCBMK2G93oGcm+gcy3xwdQgivAA21ZbUF+pCpGk0kU1U/B7goz3WOMLNdgG+AASGEhdE1Hw4hLI722RPobWaHRe9XAzYCdgL+FlVN5pjZc37SEMIfG7necGBQCGFp4ndrbUqVx5C5AeZGVe2xZjYzhPBynuuUO4/3AV4FdgY2Bp4ws81DCIvypLOWlCSPLdNe/H4IYbKZ7Z4iPeeY2THAl8Chie0jE00HewI/NbPfR+/bA93I5PEQgBDCJDOb7geHEI5NkYYmtaTQ/CrxeimZUNolw2ID+kaleCHmALM9I6Mw+/QCjrs7hNDQfsl0GnByCOGZ5A5mdmCBaUvqA4yM/oi6AHua2ZIQwqPNOFe1KlUeE0KYG/2cZ2YPA32BfIVmufP4WGBwFF2+aWbvkyk8X2vGuapVqfJ4O+AgM9svOs+qZnZHCOHoPMcNDSEMz5NOAw4IIbyT3KGZwctcYANgnmWen3TM185elC5H0TfAAjPbyMzaAMk/0KeBU/xNIyFy8lxzgI+iKhjAbsCM6NjTzOzEFiT1SeBkyzzAwcx6mlkH4AXg0Kjdaz0ykUWTQgjdQgh1IYQ64CHghFZWYOYoZh5HbVKdotcdybR7TYveV00eA7PJ/P1hZl2BHsB/WpC2qlbk+3hQCGH96P44EnjKC0wzG+LtkM30JJmaqKfFq+EvAIdH27YANi3gXI8AXpAfAjyV74Bi9tP8HZlf5mUy0aI7Bdg+apSfARwPedtCBgL3mdkUMr/4FdH23sCnLUjjTcAsYLKZTQNuIBNt30/mBplBptF6nB/QRHvX8qhYedwVeMnMXidT/X0whPB09Fk15fFgYOfo73AscHYIYUEL0lYLinkfN+ZHwLwWpPEioKNluiVNJ5NPAH8G1jSzmcD5wCQ/oIk2zRFAVzN7m0yb6Ln5Ll5TwyjNbAywf611+ZDCKY9bN8vUoR8PIdTsGs01VWiKiFSahlGKiKSgQlNEJAUVmiIiKajQFBFJQYWmiEgKKjRFRFJQoSkiksL/B9Qcb9EdJpR2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd258064908>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztfX2UXVWV5++ESlEWlaISKqQSQigCxBBIDDEIBMKkNXwpKiK02NoOw8As2tYZWnG1jq5esKCVUVxiS3fTmkEcacUGFQ2NQWJDQyBgEhJSIRQQoCQfJKaSFKkilPk688fev3vPO3WrUh/v1X2vsn9rJfu9+8792vfUPb+9z977OO89DAaDwTD8GJX3BRgMBsPhCnsBGwwGQ06wF7DBYDDkBHsBGwwGQ06wF7DBYDDkBHsBGwwGQ06wF7DBYDDkBHsBGwwGQ06wF7DBYDDkhKqBNG5oaPRNTc0lupTyw9atbejoaHfDeU7TcWlxuOkXAF56aVW79378cJ3PdNx/DOgF3NTUjLvvXjnQc1Qsrrlm7rCf03RcWhxu+gWAc891fxjO85mO+48BvYBLjf37RVZVZW8PEbcxDA2m48GBeov1R91Rmn5Lj0rUsfmADQaDIScM2/iQNTplbQu3d3f3bBePaH2NcL21LfdRcbDoTZ/9aZul495YXV/HGck6jvtl/BkA6upE8v47Onq24281NYX79sWWuY37jFQ993XvsQ762qc/v/WG+PilhDFgg8FgyAn2AjYYDIacUHIDpi9Tl2YZzbQjjihsO3p0YbuwTXNzody8WeQrr/TcnyZFbyZfpSPWcaiv2Hzr7Z7D7dy/s1Pk2LEi+Zyy9juUWV2JoM6OPLJw+9FHp59PPlkk++Go9j9mHyxUhPoptu2qBgC89JJspt67utKm+/aJfNe7Cq8p67CVjKG4z4ijjko/79ghki4h7sM+HPZT6jB2ecbunlLAGLDBYDDkhJK92+MJi/5M4JCxctTKcoZPniyyoaGwTVNT4flCkDUfOJD9va9JvnJGPLqTLbW3p2044m/fLvJPfxJJfU2fLvLss9N9qjuUxTU2FpxvT7eM12TG4fHa2kRm6T9Gues4thqoKzJTsl0AGLdprXz42n0in3lG5IYNImOzDkg674SZM0Vefz0A4OXmeQCA1ta06datBbskYL+vxNAroKdFxq52zDEiyfjDNrXYIx+oWx6EyuEfBIA3Jh5b0ISY2iTH2Nldm2xbtkzkiy8WnpvPOdZ9eE1DhTFgg8FgyAlFGyt78/Vmhe4QZGqbNonkAEZW9u53i5zwpzfSndh4g9IRpQi1SuEaZk9LmpKVUZK5cYQLBsweqAQWQZCZUZ9kTeFnqo1tyYxpUVRXHUx34o+kBkpPaudK1lrbjpQ9kB3wOOG5gb7Dispdx7w+3lvtBmW79yxNGz34IACg68knAQDs5vUqq8eMkQ+kdkA6YXHOOSJVadOaXpPznj41acrnxT580kki33lHJA8f6nI4fJeDBQ2CmFWSbda2PicfyHKB1CRg52JH5w3yezBJMeWKK+TDwoUAgN0nzwEAbOuUvhsybF7LW2+JfOQRkdrdMXt2z2uOww0HC2PABoPBkBOKPkb2xoQ5YnOUAdIBjaM8mcZ554ms379TPrwQjIY80AsviKTjU+lt7YdTx/HkyVMApAMkGXDsOw1Rbqyhr9lh6i1mwGRL4bZ16wq/kxFzVD/mmHQsPutE3Ujnp1KBN7bKrH1ITgg+FkZMVIqPMuv6Ykustm29fKB/V1kvgISd1aki6+bPl+2nny6Spt9735vuw8kHOt6V/q1trS44P9DTvbltW+F3Mrly0mmMcC6nN+ZY26XzDuy8oY5XRnUlSEn5gPjiCEHFKSvmNbDvhn58fuY18Rp7i4ooJowBGwwGQ04oGQOO2VlywuCMJFgccWI/4jtHjwMATJgbVMyK/T30pzHwL3BC7q8TBhxN5idxnSQiWfGt5Yw4ppEy9vcCKfNtaZGbdU6ccK+/Lt+bmuR7GAUR42BVIfMNoyx4LrIFxsWSLWRFtJQrW4tjmWur9hZuOO44kaGyLrpI5AUXiGRf5UOKFQRg7f4ZAIB1Suw+EMQVA8B996WfuRv/NmLLgpZHueoUSI1UoGdUFN8BqNIbW7Cg5wFuuEGkstq9zTLPQ7JMghz+HS9Qcsz+yOkMnnfc/jReu7tKIiZOO62wDY/H/h76gLMiIwYDY8AGg8GQE4Y0bob+s9iXxpGaDJhMKBwNJ04s3Ie+GB5rwg71vYXDDYenM88UyZlkDnHBMMhRblyT7LOzSRg1RzTKvu4jb4TMprfoAd7ym2+KDCMRUn+w6MJ7SRcaM0bm6elvv/C8PelOX75VJB/I2fMKjhv6gFevFjl+fOE1ktlQ9icOPA9kJKgl18VMtQlRQOjBSz6U7HPTTSK3LtJ97+JxZd/GRoloCH2OZGzs1pR0G4c+4Ng6o/XGa6QVF6Ic9Apk5wAwbjzOLqyaLP1xq/5Nbqi5PPmt5bHCfRkUEWcOhpYu+92MOomgWrBArOHEOgz+SCZNb9DjyzOjtcFnlmUhFyuSxxiwwWAw5AR7ARsMBkNOGBSBHoiJQyc49wkjRmhCxK6H+k3qeqAfI7AtaBb+aol8p6nxkSv0w733pidgZR61j8fpccapo3/XLhl/QpOvEuqsxuExnIjJMkepws5OiQ8bO1Y2MILnU5/Shrfdluyz+3vfAwDU/9mfAQBGfe5zAIBWDZP65S97nicMbAf6V7e5HHSc5X6iycnU1OOPl37yzDPiwnr88XSfxYvpU9ulUvQ7fryY1FmTNXx+dF/wb4Tg8wR6ptizEFBYeAbInuQsB/0C2TqOXVN0y6xZIzKMQmP03759r+sW+hGoXJnU/Pzn07TvSy/VD3fJgUZpIsyUE08EAOyZPS9pu3RJ4fVSl0wa4eunWBNvIYwBGwwGQ04o2hjJkSwemRmYT+d3OBpy5K/eqqnGHAZJk3WYfHZ1dbLPz38ucvFikccfz18kxfAj4TBFqkL6EFXwmTh5XsG1VQri4jtxEaOsQP729sIocjKpCZsl9XPvLbckvzHs/f0aw/Zym+h/iTKFV19Nj8OoqwkTCs9diWFonMxh4SImPTz/vEh2p+XLQ1NDKRt2q6zXY0xUKbobM+bYZA/mEbD/s7uTeYfWBPXG58X+zraVktZNsD/sUoOBfZeTxWS7Tz4Z/lHqRugsM2gRiwKvvFKo6j98LSgFeu2NIjljx/A2nbwPGTZZN/9WyHjjQkEhrBiPwWAwVDgG9R7va+2q2BdI0M87tTko+sKhh/FTSgleq5sFAHhI3bl33pnu8sorWr1a/UCtrScAAGpqZN+P3BEkbTz0kEhSNwbSqwO0Ssv/hRUDGepSCYyCPt/eiqEDaciY96KvXbvERHn7bTVNNBZvabiPyvervuhWb2kRGTJssjdaOjELH871tYoFhtSRAVOHTGpJNQQAXI2c7IwmIBmxMF/64AHgtNOE9tEwozV3yikiw7yjOBmAhC5eZKAcS6pmvSfidwYrCjAqLM06bglakdlyAulCAMBXviK6/vr1akFfdXW6Cx34110n8qyzAAAPb5AkjgceSJvS50799+anLgWMARsMBkNOKPq7PS6QTtS3S6k9/ObFdCMdP1qYen2VMN/bbpLNP/4xfW2/C45EBkzGIbOaDz0kDPjpL6el/ObR2UYGzOFWqUzj1fJ148b06OXCHvoCr5HMPWZBYaqw90yBpRTHcUeHMjKlVK9lnejznwcA3Kt5Gd3dku7d3JyWVuwtASerDnk5IivRhfMY/E7W2dlJVvuHdCdoCnzSH0n56Th8W2XqNx6tjk/mDtHnfOqpIkMLoyf7LkRW8ka5WB19MWD628l8qeN0Piad96Gvt6ZGmO8//qNsvWb/9+XDFYsKDwYAZ5whkg7cSy6R89whX8Nkot7KJdC6MwZsMBgMIxBDerdnsQeOvvV16uvlkMZpx3CUYuMPfxgA8MA98vXHPyZbY8Dp+uCs6phL/EJSMMV7GdLWrEmDKudFJemSsAG92LQIeWWMQ/FITH876xDFRfABwDnRsfdkYG+jAEvF+xuQZixUuXbuNQCA11+nP0787SwCDqQzx335oSsNLJnK7pLqnawsDPVhdAP1e0TURh7S6NGpYsh8qSumg8fdFehZe4rWGp9BmOJcrshikPR/Zy1EKkhDD8aPfw+A1D885cY/BwDsv/9+OYa2q6MTF0gD3WmKKcWePl1ihsPY6zgKqq9FJIqNynjzGAwGwwhE6Qqyx6ktdPKEw/t/+S8AgG1HSqGM1C9TGOlQ6HNjxhHZCGeX63scHkfr9Ga8DkpUcfnAgXSJnUqKq4wZL2VWHPCuXdSX6OLaa+XbG/9LpuDHBcdljtBlX+MW+jknFZwn/hx+z1rwNG5Trjrm9ZEJM8b5xBOFsm7adG7SNs00lL50xhmygZEM/J1JmQDwthohbHPVVSLpriRDBtL+HC+vRd9vFvj8e7LKfJD1nOMk19iC2rcvtRhYyz4p4HPxxbKP0tg6KpKmRIhHHxWpyvjg7C0iH0gnqb5+m/BQWhdhwbDeYMV4DAaDocJhL2CDwWDICUU3AhPHNW0uxtjQ0x3GSKkpsWqVfKW3Ip1o4/gQzuxwcoPrzkoI25lnymRIUoQDAB5U01nD3JJF4ZjXqfZO6GwvV7M4BK+R181A8qz6qzTxdu1iira4IuiCWP+/RKaJssCo//bfAACP/pRbNF9UJzzDedQ4jZYhXHFt3XJMFOgNvHaa+ewunPBiNCOQ3v/FF0tf+swCTQrQfr6lSVbjDdfpYz9nlCRDNrk97I80yelK4qrIfK5cbPnoYFWNcqkHnIV4sj5O2OHcWUfHu5N9mORzzz0iJ06UyeF1eqy7rhbZ2Zn6Dq68UvrsvfdKtanq+/8VukFkMHv5v78sK27QFUHXDV8XpVzX0BiwwWAw5ITScRFW2fiDTqAtXy4yDBXRoYXsqaUlThogyw32ScJ9uJzGAgDpRMask4OVHTiUkT6wIIeWWdzZMSq8DADlzc56m3TjZFFW2Eyasio0gpmZtXd+E0AaftYc7nSrZF50/5ChfpzVqy44bwiqmhNM8bVkrTxRDsiaUGSIEud2yF41Yi9JVQZSFnt5uyYFXPQdkVr2cNIVV4gMGNe8y4TmrW2XSU3q6sknRYbp/Jw/5vwSdcfzxosvA9mrVueJMCmHn3kfZP+0JHgfocXASUkmo6QheSu0xeMq0zqe998v6/Y1N0vyxjc/rVawWncFq1TrGn+XXSbTz3HyC89XioJSxoANBoMhJxRtTTgiGYk5RDD8LB7qgISZ1mjT8eNliNm+/QRtEKd1htuEAvzN3whLTgqLp47k9AJJF0gjlDZsaiu81HJEX2FccWF2ImRDHL25dN5Xv6o/nCkVjth0VrD/Exsm6ScmbdASEQdvqK/erqE3tl5uCK+LflV20VHtYgEcc4x4yNmNwi48dd2v5QNrVXKhQyoprHtIqCU2Sycs3qiSAjEs8xle09VXi+RCzNX71cJjev2LwvqquUYigCo9XrmAxjCQ9hdaEbScYl93aCVt3LhbJQuy0wpmAv1u9IT4g6mmb96qzmVOUoSOfH3wTfr8qX+y8KxVkYuVcGQM2GAwGHJC0VKRObIlIwOnMyk5pRhmSujwd7YEQyQEdenS9+guLHCSUrrRo8W3dv318v1v/1bkhLdf63lRceVr/b6tUxIvwqVfKglxQZOYhYYMimxNXZGY8O93yz4ada5PB7V/+ZfJPvSBjRkjbKGzkwxYmEeoYl4DiUVW8ep4n3JCeF1kOKP2S7QHKdCEJlHwhA61rtoDBZP5suNT4aElBqR/B+GJ9OQ0Dslyw0JW589WdnenFpyhI5q0jMcNGHC5WRthYkPGKmMA0r99ss7C4kN0ijMC6hkUYo7KsAKUKJOvgEQp1H3oyNffxjVIaYLW1sJoiPjaw8NZIobBYDBUKIrOS5IRoUmZgKYbJ2gJCi0rG+ZoqDWTk0GqrW1iwe9AOnn5yU+KnPDWy/IhqyrJZZcBAPZUCYMjU+RMPYuvlzOyCh4RZBaUXOYlZFAkZKoK4DZhZjWqr+kM6r3xxmSftgcKj9PZSebRs8Yk28QF2ftakqicEOo3SXWlTkiBuCIAWW14M2zLTsv4d64ddNFFIj/2sXQffSjPbdD0+TbZfNppIkNyljgxf/ADkay8xIB3hkEEfyRVw1BEZiAIF4uNl8+q3yp/v1Ony4aXJ8v8Q7i27vHHy04bN9KZzKj1OCdgQrLP/PkSIJ30+7ieZ+jQ1RfDa23CRzltxb83vlpKYcUZAzYYDIacYC9gg8FgyAlFD0Oj1fZau5gFUzUgvYftASTpydUP/hsA4K/+Sup8cjmnl7QoWuhdSCYlOJmn5tzBKkkS2H5UuiLGaK4erJNtTFigqU6U6wQR0HflMZr9tEr5PbSQe1io9PPQN0F7K5gkiitTsZYtn0MYhsXP0ULWyTFiWc5ITOU48p7f6V54O6ipTL1xFumjHxUZLfC2uyqtN3fffSI52Um32vvP1hCzsBwaXRzz54vkA6VtrYqn+QyU3+Ry2B+Tic7uKJxO9ThtunTya6+dkuzDCfcUMuF4/PHS4fl3EfZLTjpffp4mE92vSRtMxGDmFpD8cXSrqlk9gf29lIlDxoANBoMhJxQtDI3gRBdJw57Zkt5XW/Nc4Q9AOrrr5Ea9xqCcpbEjZy1QdlEwgyOfd9bJCLlPo9tG63nJBoGehT8YuUOWnsXOyo2pZV0Pt8Xr7jHaLkzE4CjObdVcK4thS/o81rama3CRLXNffqcMGQGvgTIqtZwZrlNuOibYHSedrv2OMVEnaGLQBReIZM4w0LOKjFKvZ1cIt/ndXbI5jEoj8yVLu/JK/eHBJYXnBdJixGTYrL6jit3TLecJ/6zCSa9yQObKElQ2f+Ssm+rzL2hJADh92YcAAMuWyXOgLuOayOFrIikNzOOzv2tH3d2QMmzqLl2RWRCv8B3CUpENBoOhwlGyFTHiVU+nkp6Fw9QzGlBNpxV9a2QAdBgFCzjt3C++ZVa3jBlWGDzNkSskFCHKlYkdCrxu3l/o+wIyI2xS/7H+uEVXt9jAiKugSmicO7N5s0gy4CRcC2m0Vbw2XNaKuOUO6urZ1WINnHqqpPTWM0mFHYn5wQHemCyW3oMascauTbYbrjsWMyv22XqaHEn2AFLTgjRNH8I2XeFkh15Sf1ZxKAek/VBvnh2HCqJcsSLZZ9YpPxLJ6khzGwr3pd7Cztag2+oKc5vf6BJf/MqladPYIubq1DQ+GK5qYWgGg8EwglC0d3rMeMgmGP+8fYIw11NOmZHsM+5qoVp798s4QLbMwZ4jU13AHkhCSJoZtM6BM0za4BpPYXk/oGdSQKUw4Tg6Ifa/xtYHkLoMqdMOZb4MfuDvIcjMWFeGYCBLqGOSEKYis025r/uWhbjvksXW6XxDY5PIUL+cxGfbxYvpgGXgPxcXCDuhpNo//rj8TbAvn3qqpNTWBf7cpqTIvqTPd4fLVyN9rpWiZ87DNDZKMsW4hboGd/zCCE0GsmH+8XMfMmLuy1xupBbCgSidmIcI9UUCTcuOPnT25VImbBkDNhgMhpxQslTkWLLUHmN7AeDII+X9T6bF0ZGDX1yqDkgHOxZ2ZtQDWRv9lkA6glUKO+gv4vvpzYUOpLp8U+saMRaa4Ogf7kNGTR8YQ1r5HLL0SZYQs4VK0n1v18p+GVto4Wf21YkTpWO++San6GVpnenT005MfVK/fG58VuGsO2NSx4wRmRbY7/uayx20wI44QuL2J172PwEAU67V+OBQyZHptxuF80DJEl2t6S5xxFMcnRPOk9CiC98dwPD41Y0BGwwGQ04oWRREzIA5ymRldpF9hYMekB1/x+PE/lAyYjKFrGuIt1cqqDcyMjKBrOLosT831A+QMoQwkoJ+YVoQMdvqT0xvpesY6BltksWemMXGmNS0D0uHfOcdkWFRciIuQp7Vh3tjupXoY88CLSZaaG++Wavba5M2cb8mU42jE8L3RWw98xh1UfBF1vGI4YjgMQZsMBgMOcFewAaDwZATSm7AxCFfWbQ+NiEG4jLoz28jzQURI3bHZCE2o3s7Rvw5/N5XWvThgHgCLES84AIxkGJE/Wk7UvQdvwf6CvWKQ1wZVkpXTdY7JX6XxMeI3Z1hm+GEMWCDwWDICaVbEWMAv/fGgAfiBB8pzKA/GMzEV1yYyHTcOwZjMZl+B4aB3OuhLNlK1rExYIPBYMgJznvf/8bObQfwh9JdTtnhBO/9+EM3Kx5Mx6XFYahfwHQ8HBiUjgf0AjYYDAZD8WAuCIPBYMgJ9gI2GAyGnDDoF7Bz7jvOuRuC74845xYF37/tnPvCIY7xdD/O0+aca8zYvsA5N2+g1x3s/17nXItzboNz7h+cc26wxyoVRoCO/945t9E513Xo1vmgknXsnKt1zv27c67VOfeCc+62wRyn1KhkHev+S5xzz6uO73LOHTHYY8UYCgN+CsA8AHDOjQLQCOC04Pd5APpUmvd+0EoBsIDnHyT+GcB1AE7RfxcP4VilQqXreDGA9w1h/+FApev4du/9dABnADjXOXfJEI5VKlS6jv/ce/8eAKcDGA/gykO07z+894P6B2ASgI36eSaAHwH4LYCxAI4E0AGgWn//EoAVANYCuDk4RpfKUQD+CUArgEcBPAzgCv2tDcDNAJ4D0AJgOoBmAFsBbAawBsB8Vco6AM8DeOIQ1z4RQGvw/ZMA/mWwuijVv0rWcXQfXXnrcqTrWM/xXQDX5a3TkapjAKMhpOITxdLNoMOSvfdbnHP7nXNTIKPLcgDHATgHwFsAWrz3e51zF0IY5vsAOAC/ds6d771/Ijjc5aqoGQCOBfAigLuD39u993Occ58FcKP3/lrn3F36UG4HAOdcC4CLvPebnXMNum0SgEXe+w9Gl38cgKDkPjbptrJCheu4IjBSdKxtPwx5CZcVRoKOnXOP6HX9BsADRVALgKFPwj0NUSiVujz4/pS2uVD/rYaMTNMhSg5xHoD7vfcHvfdbATwW/f4Llasgys/CUwDucc5dB60F6L3fUqkvhgCm49KjonXsnKsC8FMA/+C9f63PO80PFa1j7/1FEMv5SADv7+tGB4KhJubRtzMTQuk3AvgigN0AfqhtHIBveO//ZQjnYamOA+jlmr331zvnzgLwIQCrnHPv9d7v6OV4mwGEawlP1m3liErVcSWh0nX8fQCveO/vGMK1lRqVrmN477udc78C8FGI+2PIKAYDvhTATu/9Ae/9TgANENOCTvVHAFzjnKsDAOfccc65Y6PjPAXg4865Uc65CRCn+aHQCSApXe2cO8l7/6z3/u8AbAdwfG87eu/fBLDbOXe2Rj98BsCv+nHOPFCROq4wVKyOnXO3AjgawA19tSsDVKSOnXN1zrmJ+rkK8tJu7a39QDHUF3ALZEbzmWjbW977dgDw3v8WwE8ALFffywMIlKH4OcQPux7AvRDzI1q9rAcWA/iYc26Nc24+gG85CStbB3mgzzvnJjnnHu5l/88CWARgA4BXIb6dckTF6tg5903n3CYAtc65Tc65m/p918OLitSxc24ygK9C/KHP6TGuHciNDyMqUscAjoL4otdCJvH+COCu/t70oVA2qcjOuTrvfZdz7hgAvwdwrvp4DEWC6bj0MB2XHiNJx+VUnO0hnZGsBnBLpSq0zGE6Lj1Mx6XHiNFx2TBgg8FgONxgtSAMBoMhJ9gL2GAwGHKCvYANBoMhJwxoEq6hodE3NTWX6FLKD1u3tqGjo31Yq6SZjkuLw02/APDSS6va/TCuiGE67j8G9AJuamrG3XevHOg5DoneFtXra9G94Vhc75pr5pb+JBGGS8ex/g4XHZdKv/FS9d3dhd+54CyQ6jpuUyqce64b1uWBSqVjgvorlz4MDF7H5oIwGAyGnDBsccBZo1PMAGLG29VVuD3+HH7vz0g3kLaViCwd98Z8uT2WWW3jY/X1PA61vZJBltvUlG5r1PLf1d27AQC7UQ8g1dW4revlQ3t7uhMP0NxYeOAsk09/O6hcqVWTYDs6Bn8f5YyBLDHP90dfffdQll9fGI4+bAzYYDAYcoK9gA0GgyEnlIxk92ZKhNtjE4KWGK21vswsTmrQTAgnOXpr09Ag8p13RB440PvxKwl9mW3UMe+dbWMddwWrtsWuml27RI7RsijTp6dtaYLzPLGMJ6cqGexPRwQrglV37ZQPesP7a8QFMa5Dy/KuWycyVPDjj4s8/XSRfAj0L4SdeeFCAMCo5mYAQFPTpPB0wzaRV2rEfTj8HrvJ2C957/x7zsLo0YXfs/rjoVyTpXRFGAM2GAyGnFCyd/uRR4r8k5ZH5igSEgEO/BzJ+NumTYWyL4ZHssB5jfPOS387+WSRHPVixs1rC1FJk0e9TaBlTZJt3VooW1pEbtYy9CEzOOYYkWQPZMtHHy0ynIRSYpbImGHzmWZZPuWKWJ9hn41/H9M8Tj7QetM+O65RTQ7e7IQJ6U5z5xYemBSODytUME0XlV3thYfNM/SqGIj7cBazj98PlHGfDi3m2Oql5PuC38PPVHvMiHsLeysGjAEbDAZDTij6O53+MfoLKTk6dXYGJ9ezx75fus2eCUs3g/tzaBQH7hlnHAUgHb0mBwsNTe3WEKBNemBlHq+hFkA2iyjlaFcs9BZCRl9tyOx5jzFTJROgO3LZsnQf6pBEjc/nwx8W+cGzd6aNf/pTkU/psl6rVwMA6i+4QL5/+tMAgD2np6vTb9ggMmaW5QZeH/suJf3eQMq+4nmMo48WZnxg4WcAAE8+me6zQ3VNC23hJ2XF9FE4CAB4eUPKi2gFbljG44p817sKz5eV6FEJfZiWMq+/rU1kyGa3by/87YUXRPL9sG8fcyCyVhWqVylmXU3NWADp3wEAnKKrzvF50FiZOFFkYxQtCBQvpNUYsMFgMOSEIb2/s/xP8SxjzNLGjk1/I2Mja6Yvhgyps/MlbRku9FoYGrF69RUAgAULxGEZztDjoXUFB95bVZt5raG/aSCB4MOBvpIrYn8ZmS9ZU/gb7zW0EADg7LNFckIe6Om3veoqkRdOVovi1kVp45UfjyqdAAAgAElEQVSScrpfKd4e3VzPh6k0unb27GSXmprqgvPkib70S2uN10mWlqVfyn37RDKgQQ0C/CpYcZDPgCqh9dHWJnxoyZKex1+woFDGvs1yjjbp628qfB8A6f2EOt6hxJZzEuyzZMQbNzIs5e3gSAyN0JcMxAru7hZG3No6NWm5eXOh8ngN8XMvhY6NARsMBkNOKLmXKPb1UAIp62Kbe+8V+frrf9QW6ldEMBxC6TE4dM5Q+R4AkU+GVEOdOymzFsk44KxZ+Urwn/UWexsyS47e7363yPFar4n+rikNhSm04f70w7EtnlFfehhqctJJAIAqbVRPB34cQkG6AqCqalqf95c34iiZOO6UPskQ9MmSuT37rEj6KbP6GP3vr7zC/r5G5e6glTC1NWvmAACOO0620od51FGF1xxebzkijs1n3xq1dQsAYFqzKOr9cwNHu3bEnXVTAKRd6b3vFbljx2Tdnpp3PD79t/H3MAqC7wMybc4nsU3s5w+PN1QYAzYYDIacMKj3eH+KvhBkURypxwSLTE9qEI/h+jbxzZI8McIBoN8w9O2coPJEAMDYscJ8P/EJ2VqNvWlTdTLvqTsWAND5umx+663Caw7ZCUfocvMFAz2vKWa+ZAYhA774YpHHHy+SozmZFB1b9d0p66pvkG7R2irPZY0Ss46O8wGkccIAcMEnRY4742n58MADhRebQR+alKhs3w64Ya22PDDEcaDsy2S7QOoX5HwGCT+ZHY8Rzrqr2xwrVnDW/jcqNTgbE4OrkCl5PtNt20TSd/q2/mmMz6hEm3cfzvKZkoHy72zUM9pvaJ5mmQz6eZxS0nGqzDl8qTCY/cnH0n1efVUk6e0554ik9UaTEMDeKz8FILVIeitCFV4S722oOjYGbDAYDDnBXsAGg8GQE0rmro/NYwb1F0BtsRlqFjQ1yXjw+c+LCbZsmcjVq48KdhKby7k52ka2zmjS5ID2wE4480wAqenIeSGaGDQXw8D6uDRrOSJO86WOGaQfmru0uEa1y0TPpAbZac+YqYUHo6sAwHMo1C1NM5rOifsCaVD8pz8tyQTTbtSTU+lqWu6ePCPZZ4OGaB04AHjf+32WEv0J72N4JC1d9pOsIlHsW1o7J2nLZxHMQQZJL1tUMsySvo0gbRnir+F8MieG+KyzTOFyqXsdJ1oB6X1w0i1RRurnEhnOknFWLPa1sc9qx9yycWOyC9VBh+TJPM+bb4rUdwMAVF9yCQCgsXFcwfXz8H2FoVkihsFgMFQoij5GMhCdIV5kAPU1OhaRTgHA0qUidVj82tcYciObOdCtXn1Cssvxx8swdOed8n3GY//IH0QmMVPpcffroNoeFTIhslIMKwFxKU5KBqoDAdNgjJ/S19oxOuPJjAHO1gFo15F/+XKRjz3GyaH1ukua93rLLQsAAGvWCE2cO1fKJV56qcjpC4X5tm1Idslc6aScQDbJPkzWxhDKsPwhJ7/4G+d6ptSpRaY3u2zTlGSftP+RSs/oRQLz5wsNpyXDa4lDpcpVlzGo2+rYjIspfpC4k8SrxvGANMn0b35SaMryOA8+KPLFF0WS1oYK09jBk08eV3BJJONxmnm8+1BgDNhgMBhyQsnHzST19dFHRX73u+mPHLHUT9iwME0PBFLGStYLpEzgI+cpw7j+P0VeemnPk0dOGx6PI1o4YBLlXMgk9qnFIXO0PkIfMG7XtGG1NvZqynA1fWBkwIHP7ej3zAIQ+i0ZH/iKylBxwo4XL5bjdXcfUXA4sohQn+Wg46w1w7iNeqSe40L+YcgXfzvtNJET3npZPrS2AQB2zr0QQKHVxX44ceJ8AD2LvTDdGEi7NRkvfb/0OVO/YepuuYD6Cv8MyS7H8aYZDkZFnijhpesDK4DuW+qA5LhxroRF8t7TMNb0GX7iPs2jv/pqAMDuX/4SAFAfKkz/YOrrpBhSR4fw0r4WFShWHzYGbDAYDDlhUO/vPhZvLQhSB4D6dp3h/fa3AQAHH0uDpUfRb6v+Rw5KK1aIzFpmKGGt9O2QwV0hRXnCajMHq6ToS1xOMJ4lLke2S/TFHMkwmNzC6ITamoPpTurzOqjM9w3dXKdKbiJdDswBuukZ9A+wUs+xKtO0ZWjExMSJwTo96Wl7lL8EyrtwTAjqedZ0mb9Y2yr9KVzmhvc3rks1y86rFgbZWxBkklhxVPlll4lkIalJjUEyER/GCs1/Zo1EVeI4DS/avr062aVcltridWSlxic3r3raM1nS05mHEU4VUYf0s/O1wdwNzlV0d+9Kd9KIkgP/Kn31L1S59fxjCd4Te3UZqV3bC4/LHI4weYwwH7DBYDBUOIb0Hg9HgZjVJN91+OpW5rslaFOrcXtN6ryZMldm2efPF/8PC5qEPs1PfUo/3KwprwzSjGMEAYzSUXb/fpmRD1No4+uvJMS65i3TjXYwGFdHnXuuSNXPyfE6UNdfDwD4SddHkn1YOnHXLqYny/MYP36mHCMINIlTbntDueo6y8Igw03c4qqz9naxAML7T5gvK66rg3jb0cLoaKWE8w2MiddHk/iUk4iVB4Pq+K+o3510khVjSCWV2Y0encawlgsDJkL/N0N16+qEsdcrE41rNoWLMdAy5r7d3RrLm8RP058bliyQSj3790upgsT85UuAZkdwbh6f18C+EVv1xYQxYIPBYMgJJeMlCWu9VWbf2/RrmIBEd9Ceb30LAFCrzqJ5OmN5222yjE04gs5q+VfZ9hspYFLDqdGHHhIZZMNwrZE5Oq3c3CwMhqSC5QYrZWHD3orD0GfFkTwsuj5FZ5Vxww2FB1G9/XalMKclD6T7pBPELBYjVgbDMcPjx5ECJBgsEtNXxlk56hjoGV/ND2SxU5oCH+09Wj2dweu6BBPrw7CPMUMOSBl0bZdkJ76x6Vg9rp4gqxp5PCFCx6o+gKxIg3LUb+wX3tolBZ8Y0kvmGzLgTlbRSqJwaJlxKaI30BMSIZGEEy+JMuwCffLvhlmdcdxvKfuwMWCDwWDICfYCNhgMhpxQNCMlDl6v7tAq/2prcD4jnD9qoDnAOA+6EdSmnka7LeT599wDAEgCrbhvPBMVQm3mmtli6tEMClfnIMIVhcsVNH/itdsYjhO6CK64Qork1HfL8zjYKDq4/375XdVZUCwmTXMV90W8MkZo7vI37sNHxYmlcjSDD4W4WBNvOFk3b+njaWPGS72uZrK6eraqW4gTbuEzYXjTMy/Is0hW2DhHJqamhGm4fMh0S7B/qz9kDwrXOawU0K1I91k8N9zZGa5wTNcDVw7ZHUm6hBYke3zpS9JZZ3VpvWE+zE9qAetgEcRN6v7obbXxrD5sYWgGg8FQ4SgaPyGrTBzWnLFgqUmy27CW4UwJa0roE6kV9yWtZtEeAN0tLYUXzlkfHuujH02PzyBvXRGD8yQkFXFaJ1DeDDhOQeYoTJVyMQqu+AGkQeyNynzJvsh4GeoXjuinnioyXrOPMgzDChMMwjZ8pFkrjJRTIkZfEyxkZW9sEp4ypUYZWFpPEvjpT0UytVspHHU3pVFWfdnWmTLVf/93kclElOqQXbjAiouLx3AnZcn8OWvNuXJBlo6pW/ZHynS9vZAbMsmHK4UwEUgnmLVk53XXzUz2+OZVz8mHe+4TSWv62msBAK91pGF7zOXi6yYusFRKGAM2GAyGnFA0BhwvAVZVJe/2GRronyBYiymp3UeaFFe/IGVlTA+AGsaQkSV85SsAgD0LJZGA4SwAsH9d4bXx8Dwd2e7bQfx2JfgsmXvCoi00GHgfNDYA4Ic/1GE9XCsPAFmFc6JrMlYg1Q8LzNDNzscR6oilL8moqWO2IYvIe32y/iBm+iSb9K1PuUT7ZeAwp5u4kSnIWvZzSlIcqhkA8FJbWo6SeqSeaVHMaH9CPtx+e3pR8aJyugr1y5PfDwBoXVfYDKiMRQXIgBllR0lL2rmxSVvvmQLPbdJ3x4wRRkxy+/1b/5jsg5u0CBU7pCasPN0qzJd5MwDw0ksi+UoJLeIQYSSg+YANBoOhwlF0Bkyprlq0tIjva3TzFwAAW/+Q7rP0/+pFJGUWZRaYzO6aS5vlQzgrTIfnhz8MAPjZi1I68Z6Py2aSaiBlX0wljGPZs5ZMKWeQkfE+pjVyGSbhYRdfLOmvYRz/6tWcKWbaJm9W0os5S58VyE9GTZ9yRMIApLVhYlclj9cbmyhnxAkv1MNr7eJ7nBpUl6ELnKXVTya1ohNYmVeYTq95RgnmVK2VD4vUiR86dGmaqKnx3Ml/DiBNnMnyy4cRF+WKeDkt3gf7NgsWAcCyZeLb5XNh/+Ncxde+pg1/8IOeJ9LEmIMLxGJYcpNsDktXsq/Gksjqw5aIYTAYDBWOonuJ4po4ZA+M91u9Om3b2ko6zJg/WXqopkYiG+bOFd/PrHCNHaUS6zcJG/nd72Qz3cT0rwFpeT8y4P7E95Uz4tn5JLZRb2RenTCpk74xK9mnrU10uWyZSI7u9JvxEOGoP0HXhCSTJhuhjyz0F3PhynAbkFofYSnC+D7KQf9ZxXhYQCdeNYd6/8Kttyb7TNYC32v5XXNoa9h4iaQqTwlWpZ1Cmsdpd4ZF0JkfxKiyIvtrzcLgHlhUeP081BGF1UDLClk6judl4pKxYVw6+xKtCAacUKVzujXW99VX052U+W47TfS2SjPGaamF/ZL9PV7aazgsZGPABoPBkBPsBWwwGAw5oWSrIhOk/HRBtLaGDRg2wmAecUHEAepoSk3qF6UIWmKixCsdhzHsnIygjB39WaZFOZjFIcL5mDj1eO95MulW/fhvZcMddwAAJgT21X9oWNRz7RIGRb3F1adCcBL0jDNExvoaV7U7aftGh7iCmNTCkDUG1pebPvsDhidS93RrpVXn0vXKvv6znwEAzv+v/xUAsEl3qtPo/gZG+Wv1PgDpkg7xEuKcbP7yl5OmP3lI9Numz4leijjRJaw3XAkhf3HFOX6nC2LjxrTjjx8vHY8TvgzDTBbyXqO+MnU7AMDD3eJ6eEYXTeefBJ9hOCFPFyXdONRfRuG0osMYsMFgMOSEovETjmAM2SATihM00ur1AMA1nGT45uQb25KdhckVdLyTydGBTjkzzUZMnPZkC9GcVUUgvNY4G1Xnf/Cxj8nKu9Wai3wwCMcZddNNAIA5mhAz57xmAGltZOomrFc7Y7Iy3MS84AUIFXit49ikLa+Fz5ussa+U7nLSf9aqyFmsEkgnJb/xjdSKa79OwsLufEvk5Af/TX7gmoVR3V4APZWuVO4nD0gY5pN/mzYlOWboFVlZX8kW5aTf3sBrjGoLBdZWqq+4Lm9cx3vvZaL7cB05NfyS48bFfsJ5TlrIMRvvS4+WiGEwGAwVjqKPlXEoBwd7jjIbN04MWmu8D6YCSNkrR0X6a0I/KEceRqbF/pkwYo2+orgKf+wjqwTGAPRcfYKj+n1ab+QzynZHMf4OSGOqWEBGlTlHm8y5TJUb0ofH1IGrK4owyv85XbkhZIYxsyBK6TcrJrJCpGI9ky3xXlesSNO6f/CDV1Ry4TDJCBo7VljZrl1ZqeDaIb8njHfiRJG02MJECj7KsWMLrylmwFlMvpzB62eIGftUVvnO+O81LpLDpIpwPoOhkzwPXfE8bpgYE9f+ohUf67EUejUGbDAYDDmh6D7guAgLRxemSu7fH4YeyGwyRyO6HOPZxzDInzP0nDAm8yK7LVivSy9my9ZRBcclk6kEppB1jRzVqbeEkVJRYR4nFfPiiyJJE+hoZ/2/MHzkkksAALsbxTIJffDhrkCqf7IHzlRzRplsopzLJRK9+YBDg0LaHZV8XrlS+vC+fYzokcLsu3YxsqdaZeo3T8srHlVwHv6NsAgSkDJfton/ziqh8E6I+DpZ+pH3yUoD4UrE7MLsUxdcIJIrUr9cJRE+YTo2/zbiCB7qMSsCKk6LHg4YAzYYDIacUPRxkyPc2LGF20mwQjZB3xrdlE89VXgMjmjhqMl9GGe6bZtIjmx79lenjaOygr35UCuNPVDG6adk+pNCBxfDRRh7Spaszrb1rbJPsvwOgPaocH1cxjNkD3GReFo+vfnRKgHxDH3MjBlxAwDveY/IHTuE4W7dSimdl2wq9GnGM//xPElojPAZV0rBqP4iWpy7x5zBpKaD6Rd2RHbS2wtLTU7TiZ9ps4PQhuglUlc3KtyloF/y8NR7XFiqlDAGbDAYDDmhZO/4eHFGyqyMHcaQcgaUo1RYVpFgeCXZcey3Cf2THNkqkYX1hbigCfWUZG4F/sbOztD3CLytM8Zd6grmqB/6z+JokTgeNqv4d4xK1nlvUTJZmVEsiUgrLlwOKutY4TYeh9E6tBqyLIyRhji2l7qor1PmG/7xs7btI48AALq/9z05Bi29mMIC6aSEdt6GhnEFTbOWyIotveHwBRsDNhgMhpxgL2CDwWDICcNuKGaZpkl9W0U8WRaafPwcT6zxGH0FpA9HYHUe6KtuaWxesT5znDATPoN48qkvvcW/MU2UyS8jAf1xA7BNGD4Voj8pwwNJK670vhsnkCR1enWybFLo9+IyN9p5a+ieYEwqwy6zsn+0M9dEdYez3At5vB+MARsMBkNOKFkYWm+jRxabCH3n4b5hNNWhMFInK7IwGFbUW2jfQCYaKp119ReDuc94PTHTb9+IV0qJI83aUBv8Kp9rdD08fE0krbpGJb4siQqklliVMut4xY1ygTFgg8FgyAnOe9//xs5tB/CHQzYcOTjBez9+OE9oOi4tDkP9Aqbj4cCgdDygF7DBYDAYigdzQRgMBkNOsBewwWAw5AR7ARsMBkNOGPQL2Dn3HefcDcH3R5xzi4Lv33bOfeEQx3i6H+dpc841Zmxf4JybN9DrzjjOr51z64Z6nFKg0nXsnHvcOfeSc26N/jv20HsNL0aAjqudc993zr3snGt1zn18sMcqFSpZx865MUH/XeOca3fO3TGYY2VhKAz4KQDzAMA5NwqysmZQShrzAPSpNO/9UF6gC3j+wcI5dzmArkM2zA8Vr2MAn/Lez9Z/fzx082FHpev4qwD+6L2fBlnh4D+HcKxSoWJ17L3vDPrvbEh0xy+GcC09TjCofwAmAdion2cC+BGA3wIYC+BIAB0AqvX3LwFYAWAtgJuDY3SpHAXgnwC0AngUwMMArtDf2gDcDOA5AC0ApgNoBrAVwGYAawDMB3AlgHUAngfwRD+uvw7AMkinXTdYPZTy3wjQ8eMA5uatxxGu440AjspbjyNZx8E1TFN9u2LpZtC5N977Lc65/c65KZDRZTmA4wCcA+AtAC3e+73OuQsBnALgfQAcgF8758733j8RHO5yVdQMyNotLwK4O/i93Xs/xzn3WQA3eu+vdc7dpQ/ldgBwzrUAuMh7v9k516DbJgFY5L3/YMYt3ALg2wD2DFYHpcYI0DEA/NA5dwDAzwHc6rUnlwsqWcf8HcAtzrkFAF4F8Dnv/bbiaKc4qGQdR7gKwM+K2YeHOgn3NEShVOry4Luub4EL9d9qyMg0HaLkEOcBuN97f9B7vxXAY9HvpPyrIMrPwlMA7nHOXQdddMt7vyVLoc652QBO8t7/sn+3mSsqUseKT3nvZ0JYx3wAf9nnneaHStVxFYDJAJ723s/R6779UDebEypVxyGuAvDTQ7QZEIaafU7fzkwIpd8I4IsAdgP4obZxAL7hvf+XIZxHM7txAL1cs/f+eufcWQA+BGCVc+693vsdvRzvHABznXNterxjnXOPe+8XDOEaS4VK1TG895tVdjrnfgJhNv9vCNdYKlSqjndALDi+dO4H8N+HcH2lRKXqWC7MufcAqPLerxrCtfVAMRjwpQB2eu8PeO93AmiAvODoVH8EwDXOuToAcM4dlzEb/hSAjzvnRjnnJkCc5odCJ4Ax/OKcO8l7/6z3/u8AbAdwfG87eu//2Xs/yXvfDBlRXy7Tly9QoTp2zlVxRto5N1rvoSyjTVChOlZTeHFwng8AWN+Pc+aBitRxgE+iyOwXGPoLuAUyo/lMtO0t7307AHjvfwvgJwCWq+/lAQTKUPwcwCZI57kXYn5Ei7v0wGIAH9PQkPkAvuWca3ESUvY0gOedc5Occw8P6Q7zR6Xq+EgAjzjn1kImPzYD+EF/b3qYUak6BoC/BXCT6vkvIayyHFHJOgaAP0cJXsBlUwvCOVfnve9yzh0D4PcAzlUfj6FIMB2XHqbj0mMk6bicKpA+pDOS1QBuqVSFljlMx6WH6bj0GDE6LhsGbDAYDIcbrBaEwWAw5AR7ARsMBkNOGJAPuKGh0Tc1NZfoUsoPW7e2oaOj3Q3nOU3HxUVjY6NvHsjigocBVq1a1e6LuEKG6bgn+qvjAb2Am5qacffdKwd/VRWGa66ZO+znNB0XF83NzVi58vDRZ3/gnCvqckGm457or47LKQqiB/paUfZwWkG2GIh1Gesv/N10azAMD8wHbDAYDDlh2LlOFtPq7i78rS92Fu8b/xZ+5+e6usJ9RirDy9ITt1HW1IikzilDnfSm/yy99abLkapjg6GYMAZsMBgMOcFewAaDwZATSm4o9jWR1qWLAW2NEglj90KW26KhofAYXRkLCzU29n0NI8VM7ssNE7t3iL701pvL4cgjRb7rXem22K0T7zNSdGwwlALGgA0GgyEnDBs/IasKGVd7u8hNmwp/i9kaJ47Cz5xYmz5d5BFHiNwRlFUm+2Pbo44S+ac/ZZ8HqAzGFlsGseR9A6lOOzpEvvOOyM2bRca6D0G9UefUzeTJaZumpsK2tDpooVSqjg2G4YAxYIPBYMgJReMivbExMiwyMH4HUgZMH/Dpp4u8/nqR07qekw9Ll6Y7tbaKJNWCUrB3v1vk/DOSpr9eOQlAytg6OwuvhSB7C1EJLI063rVLJNk/9Qqk9xrrf8OGwu/79gUPBrtV0tnLcXqs/D+2PmnJZ0bGe/LJhdtjhhxedyXo2GAoJYwBGwwGQ04oOgchuyHDIrulj5E+SCBlS3/91yLPcr+XD1++TeTq1SLffjvd6aSTRI7XOhfrdJkx5qIfny7vtHChMODa7p0AgG3HjANQ6CcOrxkoP1bWV3IFdcr7IZtta0vbsg3V9OabIi+4QOStt4r8i8vGJfscrJlcsM/jj4ukikN/MZk1WTeZ7oQJImP/cbjNYDjcYQzYYDAYckLR+F6c2krQN3jaaSInTkx/o9t2wrO/lg/33ivyN78Rec45Iq+6Kt3p4osBAGs7pgBIfY2jVip7Dhy8tffdLR+Ulk2YK5W3FiyYCiD1CTMiAMiOBig3kAHH7JMMOIyr5vOgy/ymm0R+pvkJ+XCbWhtXp372Ufv2AQBmqQN31s03AwB+cdn/AAAsW9b7tWzfLvLVV0WOHi0yZL0jPSXcYOgvjAEbDAZDTig6B6EPkMyX3887T+S4revTxkvVx/vCC3o1ejmXXAIA2H3PLwCkrA0A7vycSDI6RkzMnfs+AMAHZ29JGz/4YOFxtWh0rTK7qrG1AFKWBqR+z7zRH98vmW4c6UCrIASNiHnLvikf7ntcJB8UmTCQ6oshDEqjLz/vjwCAffuOTZr+7nciGWlCphv7eZVUZ96PwXC4whiwwWAw5AR7ARsMBkNOGJILoq/au7RsKcd1q2sgXLoktlfZ+LLLAACLFsnXO+5Id/F+t55HkgFodieTfw88kDZevlwkY66ybHOU98RbVmEdTrrFNX6PO04k3TJA4nVB7UP/Jh+eeUak+iS2LPwMgDTUDAjSuvU8n/60yPqONwAAp5yStn3rrcJr47XQRUQvhk3CGQw9YQzYYDAYcsKQOEjIYMh8KcnOksItpKrMvghBCsqZOqVtrUpmQ+La1ibMlwyLAf+XN2va8n1BjNTMmSI1dG1vlUy6VWMvgJRJMjkhvqdyQxzqR4OByS0LFoicsem36U6/XCFy1SqRl14KAHiiWZjvoi/L5jA9m2y1x6NSOjunYXeyqbtbngcTb/go47KhIcpZxwbDcMIYsMFgMOSEonMRMimynB7+1SyHKzMISGfXrAEAfO5zkjBBnyQAzJ4tMklfHv+afLhTkzgWLkwbK3XedtI8AMDopDRjNYCUSWYVGI8L9pQDYlZJK4BWxqwmCRPDfa3pTs8+K1Lp8fqzrwEALNKoM1V1QZEk+nyvuEJk/RpN2ogd/QBOP13C/+JVyeN1+EJYMR6DQWAM2GAwGHJCyRgw2RlZ5pamGQCAhuYZSdvadplVx0MPifzRj0QqRZo1+XEAwD///U3pCei4ZZIFq6zfeCMAYNsRk5KmLLze+rzIo48WGdb2AQr9lOUSEdEXO+T1xqUg0a4XH9JRTb8+eMMXAACLvyWbeZ8saK9ucgDAtdeKnFal1sXyFpF8uIHC6vUiampqM6+V5wnLURrzNRgExoANBoMhJ5SMi7Ak4saNIrdtExkyzMZGKajzdabBKg3b9OSTBceavGJF+uXAAZHqDN5zx/cBpLGvWb5bxsdqhnOPSI24gFA5ISvSJE7zTtrElXGAxPdLH++YMSLJeLnv2Wenu0yr05jth7RAz89/LlLZdFjykyETjJygLvmcs6IgDAaDwBiwwWAw5ISSMWCSMNbZ4Wx7S0tYlUWY1vTpJwAAPqMMq1EZMEeHdmZvAWgkhdP0uC9+Ub7++MdkzWnF95qaCwEAV1+tx1H3MdkZs8RCVMIMfRxhkmSZxVXRgSRAt3Hu+QCSMOCkCYvksL49AGCdHodmDAOlGW7BeG0ADy+Rp8TnG2fC9VVUqJx1bDAMB4wBGwwGQ06wF7DBYDDkhKIZgTQnadpytQnWrH3lFbb8Q7CXhJAxouwzN0rkf813vgMA4FRSfbBHYkPrJNyrN8XHPZA07e6WmacdO8R05moNdD1kTb5VgllM9w5dEPQUnH+6xpQxtgxI3BK1S6S28hRmstRolRxQCcFsGffnDNqVV4pUF9HarqlJ0yVLRHKxau7KIjyVoE+DIS8YAzYYDIacUHR+QlbJZAdOuJx6qiKCVBMAAAmFSURBVMiurrTCC6OaGPjPUpJkvg1cqiJkdNFSCwyfevTRD+iWl4JfJbWZbJwRbCyhmLVeWTmHTZFNMqQsnuBqaJCVjWexKk/YKKoPub5NEidWrhQZlqO86ipJ1b6Qelez5uVGSeleGdQ7iic2yc5ZICguUxneh8FwuMMYsMFgMOSEonEREq04xIgrH5Nt0gUJANcs1FRkrfqyUxMukqRWZgswjgxIHLgvb5JWZKzTp8tyy62tY5OmEyfKSZmSTJZG/2QYrVVJYHEiloCkrpMylRefn7Q98kiRL6lh8OZikc9rejYXog5D8pISog1qoih77tL18sJ18+JUYzJdFjhimFvob69UvRsMxYYxYIPBYMgJJSvGQzDAnzVzwmqRXIn3oDJfVkScpbPu/3G9LKPDKAkgXd2XLkUmAJAk/+pXqbMxLqxDpshryUI5Jwnw2ujL5nf6YRlxEtbioQ7oAiYj3bxZJHWhq0ABSH3maG4oONGyZeIbfvHFtC19/Tw+ZbwqcvzdYDAYAzYYDIbcUDIfMJkwJZnvlK2/T3davRoAMErp0awzzwQArL9JmO8NynbJ8ICUmXKCnn5QRkMwTBhIGSFZH32bE8VdnBkHXM4MmIijH3h/XHWopSX9rbub6y2xqr1EVdeozpnZHeqYoJ+deqLv90Aaap0waOqWkqU/efwQ5axbg2E4YQzYYDAYcoK9gA0GgyEnDMkYDE3h2CyO68Mmk3NhDBLtUxbq1VUt7rtPvra0pKvvpscVE5qLaHBVCLokTjstbcuEi7gtw9CY0BC6IsrNPM6qJsbQMpr/1C0nz7q7w3TvP0Z779Y2Eq534MAcAIWPhSFqsZuHoP7Ca2BSDcMC49R0g8HQE8aADQaDIScMie+FbJGfKcmAyYA4IbazKV0TbhxnzrjTrl0AUjb1la8I21ViLPt0yDplr0EKwpC9zsB6+fCfaZbABJ35a2yUFF2ySRblqYQ02azrYZIDQ/zI9NPVnQ8GrUnv96rkbFt1wTFbg4WUeU6GlPHZxc8USCfdmMgRJ4WQrTMZBqiMiU6DYThgDNhgMBhyQtE5CH2AoZ8QSP2tZFMAMI6N6fTVDIKPVP1fkRrx3/GNgJ7pemRTNX05yW0mrUryaIGDWpwm2AQgZWMMlQrDqioBsZXB+yALbWs7MWm7axezURiORvoqadr0i4eI/bZUMc/L8pdAqtt4nTomgLAoj7Fdg6EnjAEbDAZDTihZQXb6DxngT0YcJgnMYNYEG2kh9pXqQKRrdmdwnvfpMss1xxwjG0i9WKEmWK8sLBoDpH5JXiv9k5WCmPnylun7jZkqACxdOhMAsGsXzQB5QCeeKAyY9Y5CK4FMmlUtazeslQ8aFrF+Q6H/ODxnzHRpbRgDNhh6whiwwWAw5ISiM+CYpZFZ0UXLNGAA2FI3DQAwiRXZNah07s9+Jt9J8camJSZZwIfOy71Vki5Lv2R3ELP6TrpAciYqlZ3xeqkWMmAaAywBCqSsmHG/tERoKMyarPZF6JynRXJvK3cWqY7dhuZ5SdO44BGbUvcW8WAw9A5jwAaDwZATis5LyHjiAt1kRmHWGX20SzZJTO9W91UAQPvCrxYcI8xuG6+Mq/UekbGPOYy+iGfzR1pJRN4PZZYOZs7M3mdq23/Ih3s0TbAhY1FO0tvIrKFhkgXuUmmRJQZDHjAGbDAYDDnBXsAGg8GQE0o+NdJXURa6KxoKF15IwqCy9qGJyyg0TvJlnWekT/zEE550x2TVTyaSus3T3w8AmHaT5H3vqapP2rC4D3XJ70ma8dvp8fg8Ym8FMdLcPgZDMWEM2GAwGHJC0TniQFhnXPSFktvjVTbC38I5o4Get9IR32u8sgf1GCIua8mos02b6nv8Hus/Ztr9uSaDwXBoGAM2GAyGnOC89/1v7Nx2AH84ZMORgxO89+OH84Sm4+LiMNRnf1BUnZuOM9EvHQ/oBWwwGAyG4sFcEAaDwZAT7AVsMBgMOWHQL2Dn3HecczcE3x9xzi0Kvn/bOfeFQxzj6X6cp80512Ne3zm3wDk3L2uf/sA590nnXItzbq1zbknWOfLGCNDxJ1S/Lzjn/s9gj2MwjFQMhQE/BWAeADjnRgFoBBBUbcA8AH3+8XvvB/3HDWABzz9QOOeqAHwXwJ9572cBWAvgc0O4llKhknV8DIBvAfiA9/40AE3OuQ8M4VoMhhGHobyAnwZwjn4+DcA6AJ3OubHOuSMBnArgOQBwzn3JObdC2dDNPIBzrkvlKOfcPznnWp1zjzrnHnbOXRGc6/POueeUsU53zjUDuB7A3zjn1jjn5jvnrnTOrXPOPe+ce+IQ1+7031HOOQegHsCWIeiiVKhkHU8F8Ir3XpdAxVIAHx+SNgyGEYZBh89777c45/Y756ZAWNJyAMdBXhhvAWjx3u91zl0I4BQA74O89H7tnDvfex/+AV8OoBnADADHAngRwN3B7+3e+znOuc8CuNF7f61z7i4AXd772wHAOdcC4CLv/WbnXINumwRgkff+g9G173PO/RWAFgBvA3gFwF8PVhelQiXrGMAGAO/WF/kmAJchXorZYDjMMdRJuKchLwa+HJYH35/SNhfqv9UQtjYd8rIIcR6A+733B733WwE8Fv3+C5WrIC+RLDwF4B7n3HUAjgDkBZbxYoBzbjSAvwJwBoBJEBfEVw59u7mgInXsvd8F0fHPADwJoA2AFak0GAIMNYGUPsqZEPN4I4AvAtgN4IfaxgH4hvf+X4ZwHl27AgfQyzV77693zp0F4EMAVjnn3uu939HL8WbrPq8CgHPu3wB8eQjXV0pUqo7hvV8MYDEAOOf+B+wFbDAUoBgM+FIAO733B7z3OwE0QExkTg49AuAa51wdADjnjnPOHRsd5ykAH1c/5QTI5M+h0AlgDL84507y3j/rvf87ANsBHN/HvpsBzHDOMVPlAohJXo6oVB2D1+CcGwvgswAW9dXeYDjcMNQXcAtkZv6ZaNtb3vt2APDe/xbATwAsVx/iAwj+qBU/h/gJ1wO4F2JGv3WIcy8G8DFOEAH4lk4grYO8mJ53zk1yzj0c7+i93wLgZgBPOOfWQhjx1wdw38OJitSx4rvOufWQl/9t3vuX+3fLBsPhgbJJRXbO1XnvuzR86fcAzlVfpaFIMB0bDOWFcioi+JDOrFcDuMVeDCWB6dhgKCOUDQM2GAyGww1WC8JgMBhygr2ADQaDISfYC9hgMBhygr2ADQaDISfYC9hgMBhygr2ADQaDISf8f7X0K9SfJuUYAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd2a827ccc0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance after 1000 optimization iterations\n",
    "\n",
    "After 1000 optimization iterations, the model only mis-classifies about one in ten images. As demonstrated below, some of the mis-classifications are justified because the images are very hard to determine with certainty even for humans, while others are quite obvious and should have been classified correctly by a good model. But this simple model cannot reach much better performance and more complex models are therefore needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We have already performed 10 iterations.\n",
    "optimize(num_iterations=990)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on test-set: 91.5%\n"
     ]
    }
   ],
   "source": [
    "print_accuracy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYVdXZ9/HvjaCgSMcSFEZFWlDRB7FGLIAmNowFrLGgUbFH0WhiNIrxwUQI9pJHYkTNKyoarIAiFgQhQij2jgYBBRXEBvf7x9nr7DMw5eyZU8ff57rmmlN2WTPr7HXuvaq5OyIikp1GxU6AiEg5UaEpIpKACk0RkQRUaIqIJKBCU0QkARWaIiIJqNAUEUlAhaaISAIqNEVEEmhcn53btWvnFRUVOUpKeZg1a9ZSd29f7HQUivK44VMeJ1OvQrOiooKZM2fW5xBlx8w+KHYaCkl53PApj5PR7bmISAIqNEVEElChKSKSgApNEZEEVGiKiCRQr9ZzEZHaTJ8+HYCLL74YgDPOOAOAgw46KL3NRhttVPiE1ZEiTRGRBBpcpLls2TIAPvzww2q36dSpEwAjR44EoGfPngB06dIFgB122CGfSRRg+fLlALRo0QKARo30/d1QXX/99QA8//zzAEydOhWAI444Ir3N5ZdfDsTXYinTJ1VEJIGyjzQnTJgAwL/+9S8ApkyZAsBbb71V7T5du3YF4P333wfg22+/rfT+mjVrcpxKWduRRx4JxHVZQ4YMASrXc+XS4sWLAWjTpg0AjRuX/Ue/bOy+++4AjB8/HoDvv/8egAcffDC9zXPPPQfA1VdfDcBJJ50ElGY+KdIUEUmg9IrxKrzzzjsA3HTTTQDcfvvt6fdWrVoFQJKliN94440cpk7qYqeddgJgxIgRAPTt2zev5xs1ahQQRznXXXddXs8nsXPPPReAH374AYjz4pNPPklvs3TpUgBOP/10IL5TDPt26NChMInNgiJNEZEEyiLSXLhwIRB/Q9VVt27dgPJooWvottxyy4KcZ+LEiUDcghvqrxVpFt5vfvMbALbYYgsA/vvf/6bfC3eTN998MwB/+ctfAFixYkWl10uBIk0RkQSKHmmGugyII8k999wTgAMOOACA9ddfH4CWLVsC0Lx58/Q+4Zto//33B+IocpdddgFgxx13TG/brFkzoLxGHzRUhYocnn32WSCOMENdqhTPoEGD1nkttE2EazxEmrfccgsAm2++eXrb3//+9/lOYo0UaYqIJKBCU0QkgaLdnq9cuRKA/v37p1+bM2cOEHeCDXbbbTcAXn31VSA1PX8QhkuGymUNxytt8+bNAyp3N8mnSZMmVXr+hz/8oSDnlWRC1Vno3B4+J48++igATz31VHrbYcOGAbDBBhsUMolpKmFERBIoeKT53XffAXDMMccAcXQJcOmllwLQr1+/KvetasW8jh075jiFkk/Tpk0D4Isvvqj0eq4b50LDT+jMHiKZvfbaK6fnkfy47777gPhzET43AF9++SUA7dsXZ8FQRZoiIgkULNIMXYOuueYaIJ5gI/Pb4qKLLgJgww03LFSypABC3kPclSQ47LDDADjttNNyes5HHnkEgNmzZ1c6fqtWrXJ6HsmPEGmWIkWaIiIJFCzSDC3i1157LRBPBBwmJoW4Y6s0LOeff3768dqTpeSrNfv//u//8nJcyY/QuX348OHAunckYfAKQOvWrQuXsCoo0hQRSaBgkeZLL71U6XkY3hj6V0rDE/rYPfDAA+u8F3pChAmhcyW0yn/66ac5Pa5k75xzzgHgxhtvBOJp3Q499FAgjhQz+1mGNo5XXnml0rF69OgBwB133JF+rdgTEyvSFBFJoGBF9rhx4yo9f+KJJwC48sor068dcsghQOVJNqT8hH50V111FbBun0yAhx9+GICmTZvm9NzvvfceELeaB6ecckpOzyPVC/WTZgbEo7/C5BthwvDwfqZNN90UgKFDhwLxsheahFhEpEyp0BQRSaBgt+dLliwB4pA8DHPLvD0Pg/XDOiFhTsyPPvoIgM6dOwPw05/+dJ3jz58/H4gn91ADU/GElR9nzpy5znuhM/t2221X0DS1bdu2oOf7MQsNQCGvw+cgdFgPn4/ly5evs+9RRx0FwO9+97u8p7OuFGmKiCRgSVZxXFvv3r29qmiiKmGI5NqdVnNtk002AWDvvfcG4P7778/p8c1slrv3zulBS1iSPA7Td4WJV/79738DsO2226a3Cetbh5m4w+cvTBVYlSZNmgDx5BtBmMwhs0EhNACFxsQQ7YQ1tqtqfFib8ji/QkNh5l1mWLUhXL+h61G+7hjrk8eKNEVEEihYnWYYPhnqLI499ligcvQQVp1cvXp1nc8T6ktCh+rMlSdLuZ6kIQiTZIQIMwj11xAPkwtCXt96663VHjdEjWES6mDs2LEAHHzwwenXnn766UrbhAk6sokwpTBatGgBxGUAwG233QbE12+IRkuRIk0RkQQKFmmut956AOy8884AvPnmm+tsM3nyZCCOPq+44goAZsyYkfh8oa5s1qxZifeVugn1hmsLS5IA3HTTTYmPG6aWCxNOrx2p9O4dV02FXhrBGWeckfh8kr3Q4yVzcue6TPS8Zs0aIL5uS5kiTRGRBIq+7nmm/fbbr9Lz0BIaIs3QihqGVgGceuqpAIwcORKAe++9N+/plKqF9ajPPvvsSq+HaQAhHia39vIW++67LxDfiWTq06cPAF9//TUA7dq1A2Dq1KkA3HDDDeltQytwr169AOjSpUtd/hSpRRgWPWLECKDyNVmdt99+G4DRo0cD8Zr0ENd7l0PdsyJNEZEEVGiKiCRQUrfnaxswYAAQd5YODUS33357epu33noLgClTplR5jFKaHaWhW3v4axA6sgO0adMGqNs6UGvP2N2/f38g7uqUKQyn1WoAuRUabO655x4gHpQQqs4g7jYUZjsaM2ZMpd8ffPABUPlWPFTXnHXWWUBpV6so0hQRSaBgwyjrInxTnXzyyQD885//rHWfMKvzgQceCMTfiJCbtbU1xK70ZHYzCp2iQ3Rbl4hWeVy9EEVm3j1A5a5CtTXm9O3bF4CBAwemX+vXrx8Qz9SebxpGKSJSICVdp9msWTMgHsz/1VdfAZU7rIe1YMKaMyeccAIQd4yXhq99+/ZVPpbcC3XSYXXIMElLVcIdYpidf9CgQQDsueee+Uxi3inSFBFJoKQjzSB0iJ4wYQIA//jHP9LvTZs2DYgjyzC1lIjkXmgzePzxx4uckuJRpCkikkBZRJprO/7446t8LCKSb4o0RUQSUKEpIpKACk0RkQRUaIqIJKBCU0QkARWaIiIJqNAUEUlAhaaISAL1mhrOzJYAH+QuOWWhk7v/aGaFUB43fMrjZOpVaIqI/Njo9lxEJAEVmiIiCdRYaJpZWzObHf0sMrOPM56vn69EmdlCM5sbnWd6FtsPMbMl0favmdnJ9Tz/PWY2MIvt9jOzOWY238yeqc85i6VYeRydu7GZ/cfMxmex7dUZaZtrZgfW89wvmFmvLLY72swWRHl8d33OWSzFyGMz28jMZkTnWGBml2exT8Hz2MyGRWXGHDObaGZb1nbcGmc5cvfPgF7Rwa8AVrj7n9c6qZGqG11T28kS+pm7L0+w/Vh3P8/MNgPmmdmj7r40I52N3f2HXCXOzNoANwAD3H2hmZXlRJ5FzuMLgHlAtgv5XOfuo8ysJ/CsmW3iGZXyecjjbsCFwO7uvlx5nMgqYB93X2lmTYBpZva4u9e2GFFB8xiYCdzg7qvM7GzgWuDYmnao0+25mXWOvj3GAvOBLc1secb7g83szujxpmb2kJnNjL55dq3LObPl7ouA94GO0TfX3Wb2IjAmimyuj9LxHzMbEqWxkZndbGavm9lEoF0WpzoO+H/uvjA67+I8/UlFke88NrNOQH/grqRpc/d5gAGto7uCW8xsBnCNmTU3szFROl41s4Oj821oZg9EUcWDQNMsTnUaqQtqeXRe5XGWeezua9x9ZfR0faAJkHWrc6Hy2N2fcfdV0dOXgS1q26c+dZrdgJHu3gP4uIbtRgMjopXfjgJCJuxiZrdWs48Dz5jZLDM7JUmizKwz0Al4NyOd+7n7caQugsXu3gfYGRhqZh2BI4CtgB7AScDuGccbbma/qOJUXYC2ZvZc9EE6Lkk6y0Q+83gUcBEJLqTAzHYHvnH3z6OXNgd2dfdhwOXAk1Ee7wv8xcyaAmcBy9y9O3A1sGPG8e6q5jauC9DdzF40s2lmNiBpWstA3vLYzNY3s9nAp8AEd59V1XbV7FuoPM50CvBEbWmrzyTE72QRagP0A7pavKxnazNr5u7TgerqK3d194+jW+2JZvaau79Uy3mONbO9gW+BIdHtFMAj7v5NtM0AUhfB4Oh5S2BbYC/gvujWZKGZTQkHdffLqjlfY2A7UtHSRqRuP6a5+zu1pLOc5CWPLVVf/JG7zzazfgnSc5GZnQh8BQzKeP2BjNvKAcDPzeyS6HlToCOpPB4B4O6vmtn8sLO7n1TN+RoDWwN9SX0RP2dmPdz9ywRpLnV5u47d/Tugl5m1Bh42s+7u/lot5yl0HgMQnXM74Jxa0levQnNlxuM1pELpIDMsNqBP9A/Mirt/HP1eZGaPAH2A2grNse5+Xi3pNOBMd5+cuYGZHZZt2jIsBD5296+Br6MqgO2BhlRo5iuPdwd+aWaHRMdpYWZ/d/df1bLfde4+qpZ0GjBw7S8vq2Ut7mosBJ6L6tDeMbN3gG2AV+tysBKVt+s4cPdlZjYV2B+ordAsdB5jZgeQuuvpm83fl5MuR9E3wDIz29bMGgGZhdAkYGhGAmtrzWpuZs2jxxuRiuTmRc/PNbPT65HUp4AzzaxxdLyuZtYMmAoMiuo2O5CKLGozHviZma0XpbMP8Ho90lbScpnH7j7M3bdw9wpSdcNPhwLTzEaEOqo6ego4OyMt4RZtKnBM9NoOwE+zONZ4YO9on01IFZjv1SNtJS3H1/EmZtYyerwhqUj19eh5yeSxmfUGbgIOyWw4rkku+2leTOqPeYnUN3QwFNjDUg0vC4BTo8RWVxeyOfCimc0BZgAPu/uk6L3uwGf1SONtwFvAbDObB9xCKtoeB3wILCDVMDEt7FBdnWZUUf0MMJfU7cnNWdx6lLtc5XFNtgcW1SONVwIbWarLynzgiuj1G0nVQb8G/J6MaLGG+q7HgBXR3zQJOD9hj45ylKs8/gmp6oxwHT/m7k9G75VSHv+ZVPXag5bq6vRwbScvq2GUZvYYcGiOuxxIibDU/dUT7n5AsdMi+dEQ8risCk0RkWLTMEoRkQRUaIqIJKBCU0QkARWaIiIJ1KdzO+3atfOKioocJaU8zJo1a+mPaVZv5XHDpzxOpl6FZkVFBTNnZjMCq+Ewsx/VsgDK44ZPeZyMbs9FRBJQoSkikoAKTRGRBFRoiogkoEJTRCQBFZoiIgmo0BQRSUCFpohIAio0RUQSUKEpIpJAvYZRiuTLd9/F61uNHj0agCuvvBKAtm3bAvDpp58CMHHixPS2e+65JwAffJAaJXfvvfcCcPHFFwPQqJHihEJ5//33AXjzzTcBuOeeewB4773UMktbb711ett3302tuH3ssccCcNJJqcUjN9hgg4KkNQl9gkREEiipSPOHH1JL/4RvorFjxwLw1VdfVbvP4YcfDkCPHj0AaNWqVT6TKHm2Zk1qaevzzotXY37ttdR6dTfeeCMAgwallsM+88wzAdhmm23S2y5btgyAfv1Sy6mvWrUKgCFDhgDQvv2PZvKiovnkk08A2HvvvQH48MMPAQhL64Sldl988cV19n3hhRcAWLo0tTDk7373u7ymtS4UaYqIJFC0SHPOnDkAPPXUU+nXJkyYAMTfNtkYOXIkAF27dgXgT3/6EwADBw7MSTqlML788ksATjzxRAA222yz9HshT3fddddK+4R6r3bt2qVf22uvvYA4wnzyydSqsYowC6d169YAHH/88QAMHz4cgKZNmwJw2mmnAdCxY8f0PhdddFGlY9x6a2pV4F//+tdAaeWfIk0RkQRUaIqIJFDw2/Pbb78dgLvuuguA6dOnp98Lt2SnnHIKAJdeeikAG2+8caVjLF68OP34oYceAuLuKEcffTQARx55JAB33313bv8AyYtwGx0aEW666ab0e5tvvnmV+/zsZz8D4Jlnnkm/FhoTw+1dz549c59YqVGzZs0AuPDCCwH46KOPADjooIMAOOKIIwBYtGhRep+1b89D9co333yT38TWgSJNEZEEChZpPvfccwD89re/BeLOy5kRRYgw119//RqPlVnxH7oahQ7PQ4cOBeII9Nxzz01v+z//8z91/wMkr9544w0gbsipLrrMNG/ePAAOPfTQ9GvNmzcHKjcySHG0bNkSgDFjxlT5/n//+9/049AdqbrnpUSRpohIAgWLNA8++GAAVqxYAcAVV1wBwBlnnJGT44fOy6EOM9SVLlmyJCfHl/wIdVlTp04F4Pnnn89635tvvhmAr7/+Ov1aqCvffvvtc5VEyZPJkyenH4cO7yHCDHeioW6zlCjSFBFJoGCRZhgKGb5RQt1TrjRp0gQozQH+sq77778fgPvuuw+IJ3WoqT77888/B+JeFaEnxm9+85v0NqFlVkrfggULqn1vu+22A6BLly6FSk7WFGmKiCRQsEgz9JsLwx3DN0muvPPOOwDMnDkTgA4dOgDxVGFSGkI/yjBV29VXXw3AhhtuWGm7MHEHxP0w//jHPwLxBBCjRo0C4JxzzsljiiXXJk2aBMQ9XKpSyncMijRFRBIoWKQZBt7nS5jgNLSkhv6aua47lfqZO3cuAN9//z0QjxJZuHAhEE9cG6YFhPguJdR3PvroowDsv//++U+w5FyYpOeLL76odps99tijUMlJTJGmiEgCKjRFRBIoqZnb6+Laa68F4gk7wozf11xzTdHSJNULcyp+9tlnAPTv3x+I51etqKgA4iF4mfuExkPdlpen0O1wxowZQNz9MNP5558PQJ8+fQqXsIQUaYqIJFCWkWbm6oOho3O3bt2AuKPzeuutV/iESa26d+8OxDPuh9UiQ3eis846C4D//d//Te/z+uuvA+o+Vu7+9re/ATUPlQ1rApXyqqGlmzIRkRJUVpFmqL/861//mn4t1G/dcccdQNypXUpbWEky/A7CGuchryFeGygz+pTy8+CDD1b7XuiSGNYXKmWKNEVEEijpSDMsazFs2DAgrv8aMGBAeptHHnkEUB1muQtLIoQlEjIjjjDcLkzKIuVp7VVmW7VqlX58+eWXFzo5daZIU0QkgZKONMOg/fANtcMOOwBw6qmnprcJfb+WL18OxP38kghTjoVoJ5xH8m/16tUA7LvvvkDcC+Lpp59Ob5O5BrqUnzBBRxD6Z4aJySG75U1KhSJNEZEEVGiKiCRQUrfn4VY7dFB/+eWXK70fhtoddthh6dfat28PxPMvhlmNwiw6VQndkg444AAAbrnlFiBejySkQ/LvhhtuAOKVCcMaUlL+Vq5cCcDZZ59d5fuh0a/cKNIUEUmgpCLNEFneeeedQDykKpuGmRYtWgDwxBNPAPGqduHbDuI1tYcPHw7Al19+CUDjxo3X2Vbya8qUKUCcx5dcckkRUyP5EO7Ywpr2DYUiTRGRBEoq0txnn32AeGXC0MG5bdu2WR8jTDVWk9DVIUw/FqIc1WXmX1iXPqx3PnDgQECRZkMW7vqqe15uFGmKiCRQUpFmqFvs3LlzXs+TOcEtwKabblrpt+TPddddB8SrT4aJVkLeS8Mxbtw4IO7MHiLMQw45BIAePXoUJ2H1pEhTRCQBfb1LUZx88skANGvWrMgpkXwJy5OEobFhpdGwbn253l0o0hQRSaA8i3opK4sWLUo//u1vfwvA4MGDi5UcKZC+ffsCMHnyZCCePLxLly5FS1MuKNIUEUlAhaaISAK6PZe8y5wP85hjjiliSqQYwlyZmes+lTNFmiIiCajQFBFJQIWmiEgCVp/B82a2BPggd8kpC53cvX2xE1EoyuOGT3mcTL0KTRGRHxvdnouIJKBCU0QkARWaIiIJ1FhomllbM5sd/Swys48znq+fjwSZWSczm2JmC8xsvpmdlcU+Q8xsSZSu18zs5Hqm4R4zG1jLNpdk/C/mm9kPZtaypn1KUTHyODrvBdH/bb6ZVb1cYeXtC57HGdvuZmars92+1BTpOu6RcY7ZZvZVbddyka7jn5rZNDP71szOy+a4NY4IcvfPgF7Rwa8AVrj7n9c6qZFqUFqTzQmz8D1wnrvPNrMWwKtm9rS7v1nLfmPd/Twz2wyYZ2aPuvvSjHQ2dvcfcpRG3P1a4Nro2IcBZ7j7F7k6fqEUI4/NrBfwK6A38APwtJlNcPf3atm1oHkcjglcA0zM5XELqRh57O4LMs7ZBPgYGJ/FroXO46XA2cAR2e5Qp9tzM+scRYJjgfnAlma2POP9wWZ2Z/R4UzN7yMxmmtkMM9u1pmO7+yfuPjt6/CXwOtAh27S5+yLgfaCjmV1tZneb2YvAGDNrbGbXR+n4j5kNidLYyMxuNrPXzWwi0C7RPwSOBu5LuE9Jy2ceA92Bl919lbt/D0wFDqtln7QC5/F5wP2kLq4GJc95nKk/8Jq7L8x2h0Llsbt/6u4zSX15Z6U+Y8+7ASe4+8zo27g6o4ER7v6ymVUAE4CeZrYLcJK7n17djma2NdATeCXbRJlZZ6AT8G5GOvdy92/M7Exgsbv3MbMNgJfN7GlgV2AroAfwE2ABcGt0vOHAi+7+eDXnaw70A07NNo1lJF95PBf4g5m1Ab4Ffg68mG2iCpXHZtYROBDYD/hZtukrM3m/joHBJAwqCn0dJ1GfQvOdqISuTT+gq0XrhACtzayZu08Hple3U3Rr/iBwtruvyOI8x5rZ3qQuwiHuvjw65yPu/k20zQCgu5mFyRxbAtsCewH3RbcmC81sSjiou19Wy3kPBZ4rx1vzLOQlj919npldD0wCVgCvAquzOE+h83gUMMzd12T8bQ1Nvq/jpqS+eC7IMj3Fuo6zVp9Cc2XG4zVA5qeqacZjA/q4+3fZHthSldMPAXe5+6NZ7jbW3auqyM1MpwFnuvvktc6X9a1hFQYD/6jH/qUsb3ns7rcDtwOY2Qjg7Sx2K3Qe9wYeiC7adsAAM1vt7v+qw7FKVd7yOHIgMD2zXrIWxbqOs5aTLkdRyb7MzLY1s0ZUrp+aBAwNTyzVCFAtS31CxwCz3X30Wu+da2Y13QbU5ingzHAbYmZdzawZqTq1QVGdSAegbzYHM7PWwO5AQ7qIqpTLPI622ST6XQEcQqresKTy2N07unuFu1eQasQ4rYEVmJXkOo8j69T3l1Ie10Uu+2leTOqPeQnIrPAdCuwRVdguIKr7M7NdzOzWKo7Tl9Q/ur/F3RX2j97rDnxWjzTeBrwFzDazecAtpKLtccCHpOpA7gKmhR3MbLiZ/aKa4x0OPOHuq+qRpnKSqzwGGB9tOx44PWr0g9LL4x+bnOWxmW0M7MO6reYlk8dmtoWZLQTOAa4ws4VmtmFNJy+rsedm9hhwaK67lUjpUB43fOWex2VVaIqIFJuGUYqIJKBCU0QkARWaIiIJ1Gs1ynbt2nlFRUWOklIeZs2atfTHNKu38rjhUx4nU69Cs6KigpkzsxlM0HCY2Y9qWQDlccOnPE5Gt+ciIgmo0BQRSUCFpohIAio0RUQSUKEpIpKACk0RkQRUaIqIJKBCU0QkARWaIiIJ1GtEkEiu/PBDamrFsAbNeuutV8zkiFRLkaaISAKKNKWoJkyYAMDxxx8PQLt2qaWqL7300vQ2v/rVrwBo1Ejf8aVm2bJlAIwdOzb92rXXXgvAxx9/XOU+AwcOBOJ8zXytHOhTKCKSQN4izTPOOAOAXXbZBYATTzwxX6eSMtazZ08ATj75ZADGjRsHwCmnnJLe5q677gLgzjvvBKBLly6FTKJUYdWq1FqChx2WWrDyueeeW2ebffbZB4Dtt98egK5duwLw8MMPA3Dcccelt73nnnuA8og4FWmKiCRQr4XVevfu7dXNwxdaQTfZZBMAJk6cCMTfOsUwb948IP5Wu/jiiwFo3bp11scws1nu3jv3qStNNeVxPj399NPpx4MGDQLiFvbXX38dgA4dOuTl3Mrj2v31r38F4LzzzgNgq622Sr8XIsxbb02t7NukSZNK+65ZswaAY445Jv1aiD7vv/9+II5g86U+eaxIU0QkARWaIiIJ5K0hqGXLlgAsWbIEiMPuzp07A7Dhhhvm69QAfP755wDce++96df++Mc/ArB06VIAFi1aBMCYMWPymhZJbsCAAenHoQHoqKOOAmDu3LlA/m7PpXajR4+u9PzJJ59MP66toS50HQsNfADfffcdEFeZ7bHHHkBcvVdKFGmKiCSQt0gzNLYccsghQNzh9e233wbgkksuAWCzzTZL7/OTn/wk8Xk+/PBDAGbMmAHA448/DsRdIN57771q933xxRcTn08K7/DDDwfiCObVV18F4IADDihamqSy6dOnpx9n2yWsWbNm6cdXX301APvttx8Ahx56KADTpk3LVRJzRpGmiEgCeYs0QxQQfj/11FNA3Hk5DJ9bf/310/uExyHi/PbbbwE48sgjKx07DN0CuPvuuwH46quvEqexHDrSNjRhaN0///lPIL4jCN2+jjjiCAD23HPP9D4LFy4EYOXKlUD+u6NI7UKXo1DPPGzYsPR72223HQC9evXK+ng9evQA4JZbbgHg9NNPB+JO9JlRabEp0hQRSSBvkWbjxqlDhzrG3//+9wDcdtttQNyC/c0336yzb3gvCPUdNQnRxwsvvADErfZVpelPf/oTAKeeemqtx5W6C9H/Aw88kH7t/PPPByAMqmjbti0Aq1evBuDvf/87ULmuu2PHjkA8JLdbt275TLZk4aCDDgLgyiuvBODyyy9Pv/fzn/8ciK/10K6xtjDYBOI2jnCth8/H3/72NwDOOuusnKW9vhRpiogkULCp4a666ioAfvGLXwBxnVaILACaNm0KxNHHggULUolsXDmZJ510UvpxaG0Lfb7WrtvMHML17LPPArD77rvX50+RLIX/d6jPhnjYXZiQI0SRoZ9e2DYzOgn9aVu0aAH9eUqFAAAJJElEQVTABx98AECnTp3ylnbJzkUXXQRA9+7d068NHjwYiO/+jj76aCDuJ7311lsDsHz58vQ+jz32GBBP6nHaaacBcMEFFwCVh1y2adMmx39FMoo0RUQSKPgkxLvttlul36NGjVpnm5deegmI+2Cu/c3Sr1+/9OORI0cClaOZTOHbDRRhFkqoVw4TC2dOUBvqwtYWek6sWLFinffCZBBTp04F4rrNRx55pNJzKZ7MfH355ZeB+O4y5H/Irx133BGo3EMimD9/PhC3O4Q6zjDJRylQpCkikoAKTRGRBPI2n2Y+ZaZ57733BuD555+vtE1YayYM24S4IaE+NNdi7XbddVcgHpwQbqsBNt544yr3eeKJJ4C4gWjTTTdNvxcmgwhD9c4991wgbiAKwyohN92RlMe5Ea7T1157DYjXBFq8eDEQV79lCvPwhkakhx56CIDx48entwlDLOtD82mKiBRIWa5GGb65YN0Is1WrVkD8DZWL6FKSCVO2hRm8q4suIY42wkCD0O0sRJ4QR52hG1IYcte/f3+g8nDYMON76MokxROixpBfr7zyChAPPAnDYwEuu+wyIM73zC5MUPmaz0WkWR+KNEVEEijLSLOqYZUhmgldjKrqziCFEYbEhYlWMjuhH3zwwUA8YcuFF14IxNFkGPSQOYxybWEi67DuVIg4IZ68eNasWQBstNFG9flTJA/at29f6TfEg1RCpBm6q4XPSWa9ePh8FYsiTRGRBMoq0gxTwoXp5TKFb59SGtj/Y7XzzjsDcMIJJwCVh0T+8pe/BOJhc5tvvjkAzzzzDJBsIuq1I06AnXbaCYhXsAyThZTS1GJSu7AczhZbbAHE082VAkWaIiIJlFWkOWLECCBe/zpTWKxJSkeYmDazPuqtt94C4jWxQ+RZn14OIeKEOLIMS2SEPqNhOZQNNtigzueR4gkLNZYClTQiIgmo0BQRSaAsbs///e9/A/DnP/+52m1q6kAtxdG8eXOg8prYoRolc22oXNp///2BuCN1aBgKs/CE2bBUnVPaPv30UwAmT54MlFYXQn1yREQSKItIc5tttgHiGZ9DY0KmJCvfSWGFoZGFFIbhhTVmQmfpG2+8EYBzzjmn4GmS7L377rtAvIZYuIMoBYo0RUQSKItIM3R0Db8zhS4kPXv2LGiapDyEtWXCJBFhNcwtt9wyvY3WUS8911xzTaXnmflVbIo0RUQSKItIc/bs2QDMmTNnnfdClKAp4KQmYXht6PweVsUERZqlKFzrIcIspUEJijRFRBIoi0izJpnrIYtUZ7311gPiIZ2ltLqhrCsMmwwTuZRSP2xFmiIiCZRFpBn6Z4b1refOnZt+L0wdJZKNMBJII4JKS5j6LaxxH/plZk7GUir0yRERSUCFpohIAmVxe962bVsgnu07dFQG6Nq1a1HSJCK5E9Z2CsMnS5kiTRGRBMoi0gzatGlT6beISKEp0hQRScDcve47my0BPshdcspCJ3dvX/tmDYPyuOFTHidTr0JTROTHRrfnIiIJqNAUEUmgxkLTzNqa2ezoZ5GZfZzxPC8rY5lZj4xzzDazr8zsrFr2GWJmS6LtXzOzk+uZhnvMbGCW2+5mZquz3b7UFCmPO5nZFDNbYGbza8vfaJ+C57GZtTGzR83sP2Y23cx61OecxVKMPI7Oe0GUv/PN7Owsti9GHv8yyt/ZZvaKme1e64HdPasf4ArgwipeN6BRtsdJ8gM0ARYDW9Sy3RBgVPR4M2Ap0G6tbRonOO89wMAstmsMPAs8mc32pf5TqDwGfgL0ih63AN4BupRaHgMjgcuixz8FJhY7j8ooj3sBc4Bm0XX8LLBVCeZxc+K2nZ2AebUdt06352bWOYoSxgLzgS3NbHnG+4PN7M7o8aZm9pCZzTSzGWa2a4JT9Qdec/eF2e7g7ouA94GOZna1md1tZi8CY8yssZldH6XjP2Y2JEpjIzO72cxeN7OJQLssT3cecD+pzG1Q8pnH7v6Ju8+OHn8JvA50yDZtBczjHsAz0TnnA13MrG226Sx1eb6OuwMvu/sqd/8emApkPdtzofLY3Vd4VGICGwG1tozXp06zGzDS3XsAH9ew3WhghLv3Bo4CQibsYma31nKOwcB9SRJlZp2BTkAYj9UN2M/djwNOAxa7ex9gZ2ComXUEjgC2InWRnATsnnG84Wb2iyrO0xE4ELgjSfrKTN7z2My2BnoCr2SbqELlMalI6ZfRNrsBW0Q/DUm+8ngu0Deq4tgI+DmQ9UI/BcxjzOwIM3sDGE8q2q1RfUYEvePuM7PYrh/Q1czC89Zm1szdpwPTq9vJzJqSKpQuyDI9x5rZ3sC3wBB3Xx6d8xF3/ybaZgDQ3cwGR89bAtsCewH3ufsaYKGZTQkHdffLqjnfKGCYu6/J+NsamnzncQvgQeBsd1+RxXkKncfDgdFmNptUAToHWJ1FOstJXvLY3eeZ2fXAJGAF8CrZ/e8Knce4+zhgnJntA1wVHb9a9Sk0V2Y8XkOqTiTIXOjagD7u/l3C4x8ITHf3bG99x7r7eVW8nplOA85098mZG5hZXRaJ6Q08EGVoO2CAma1293/V4VilKm95bKkGiIeAu9z90Sx3K2geu/sXwK+i/RuRul18L+lxSlze8tjdbwduBzCzEcDbWexW6Os4zd2fNbO/m1krd19e3XY56XIUlezLzGzb6MOVmfhJwNDwxMx6ZXnYo1nr1tzMzjWz0+uR1KeAM82scXS8rmbWjFR9y6CoTqQD0Le2A7l7R3evcPcKUmH9aQ2swKwkl3lsqW+aMcBsdx+91nslk8dm1srMmkRPfw1McveVNe1TznJ9HZvZJtHvCuAQUvX/pZbHnaPPI2bWm1SjULUFJuS2n+bFpP6Yl4DMhpuhwB5Rhe0C4NQogdXWd5nZxsA+pAqjTN2Bz+qRxtuAt4DZZjYPuIVUtD0O+BBYANwFTMtIS7V1IT9CucrjvqS+FPtb3PVl/+i9Usrj7YAFUX3XfmRfVVTOcnYdA+OjbccDp0eNflBaeXwUMC+qghkNDKrt5GU1jNLMHgMOdfcfip0WyQ/lccNX7nlcVoWmiEixaRiliEgCKjRFRBJQoSkikoAKTRGRBFRoiogkoEJTRCQBFZoiIgn8f+zqJWNhd48fAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd2580646d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_example_errors()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model has now been trained for 1000 optimization iterations, with each iteration using 100 images from the training-set. Because of the great variety of the images, the weights have now become difficult to interpret and we may doubt whether the model truly understands how digits are composed from lines, or whether the model has just memorized many different variations of pixels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAD5CAYAAAAZf+9zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJztvX14VtWZL/xbMYQYYwwQDGKgKSKmCIoUBS1yoSLaaqfa2qnO6czx9NU5tsc5tTP1nZ5rPB2d9rx1Xuvb6RlnajtereccO3WOtraDUz+PpdbP+gWCGJXaFAJCCRAgYoCY9f5x37+9117ZCR95kv08cP+uK9fKs5+99se917PX7/5cznsPg8FgMIw+qoq+AIPBYDhSYS9gg8FgKAj2AjYYDIaCYC9gg8FgKAj2AjYYDIaCYC9gg8FgKAj2AjYYDIaCYC9gg8FgKAj2AjYYDIaCUH0wO0+Y0OSnTGkdoUspP6xf34GtW7vcaJ7TZDyyONLkCwArV77U5b2fOFrnMxkfOA7qBTxlSiueeOLFgz1HxeL88+eN+jlNxiOLI02+ADBhgvvdaJ7PZHzgOKgX8GihWq+qr2/gd/v2ZT+PGTPy13MkoDpnJOTJ3zBwDA4Fjs88+dbWSrt9+4Efx3B4wWzABoPBUBBGnAEfCFt4/31pjzoq22fPHml7e9N9Y1ZGFpEHsg62PD5xuLKKPJlTxgTlFrcAUNW3FwCwFzUAUvm/917+sYCBsiUOJxlTruH9x/c9bpy0NdX98g+FF1Lgri79TraNGXN85vgc9yHibWPHSnv00QP3PRw0l6HGMMcqRZo3LimD+LdfbmPXGLDBYDAUBHsBGwwGQ0EYMRNErK7lqUWhaSHvM1UNamzhcaiCdXdL29MzcF+isTHbTpok7THHDNy3klXmPBWWMqV8+LmlRVqqaFu2pH2ajxUh1/TJzpu6GwAA77wz8JzHHps9DlFfn/1cyXIleK91fTvTjStWSMuB19kp7a5d0vJhUOAAME8jP/RhjG8T4b28YTyAdEwD6XOjPPmb4OfmZmnzTBGVAN4Pf9dsKesQdbVq1qGMVTgN3GHt2nRnHoj2CeLkk6WdPj3Z1N8ocu/oyL82mihGYgwbAzYYDIaCUHIGPJgDLXaIAUBTU7YPsWlT9nsyVyCd+cnGWlulJVMYyilHkGFwkgzZWiUxtdgJsWOHtJQFMJAxUZZTW5RNKINrCGkXBbNgQaYPtY7m9zem+7JfV3f2BPXC+NYpew6dHwfyjMoBvM663m3yT4/Kpb093emnP5WWD4OdKJdZs7Rv8FDI4Dh49UfCXcJdCRJt/p5OOil72gkT0n0rYQzzuuuqxeGLXr1psljKCEgFwu9Wr5aW45SyXrgw7dPWJi3HI2ms0tzds85Kdl12n7R8Z8WPcl5OqHqpZGwM2GAwGApCyRkwmQ4nrTgcJLQFczanTZYzEInB1LVPyD8hO9OZcSptapv1wEqXZ8yZley6U61DDdW7ZddddQCA117LXiOZNpBOmJXA0niNvA+SBrIlIGW+xx2X7dvTI3PvaXnGeX0Ab3eJ/Ej4lizR73sDleFFzXh67jlpTz01c+LqajlGaJunvI86CiinNWFjTSwZAxyoZF6hwTx2VNCQyPFJdS40MPJ/Djbtu+jKKwEAb7YtSnbl74Ymy1WrpN28WdqJE6NrRerbKEcmzFuu6vq9/LNaWS0HWahdEBy8NHhzIPKmdbzubTw+6VLzVPTuiPrU9e5O9u3okPfCypXZ051wgrR8P/G9FGK4MjYGbDAYDAWhZAyY7GGwIHA6hbduTbeRnXFmITOqg85OHXp5c+aknWhTe+opaWkAJasIqFYDaYNSw2adfsfMEfvPgw/K12H0Rej5LAd2FrIyeropY5IrTvK8j1Bh4D68L4qEZO69k0QWKwMH8uN3ShtHStCs1tSU+J3RwI2kCYq9k6bK+VdnjwFkvcpuVEsdDUQoX2pvZGmJ150CZRu66MmsKCyVw25l/nVd62T744+nfWhMP+OMzAnX9U0GAHQEJJDPLQyiAIDXX5eWpDDU4vKie4pE+OyrNqn/gAOQjJdqWx7N5EuDTo4zz5SWv3k9Vo36LACkPxI6i1ZnB+LeWXOTXb/0JWnvvTe7KxFHEAGlk7ExYIPBYCgIJbcBx7G8RF4KIMlsTY94mWviYMDYkwmk0xG3cRYkFQj3JX3giXRqG9/9NgDglFOmAciy8nJO44xDGknIyHLZhuGQcXQIzZckGhRfXhAElQnuS6Xj0kvTfXe3zAAA7FK7Ge34j98jLcdDqMTwUe3bV7yWEcbPUr5JnG+nCuWVV6TNy41Xz/vuWoklJeOta6nP7ht66FUN2bhJ+A8DKchyw2fB4UxPfEwQaQtmeGs5Ia8AUTKoOJi408UXS6s32t86LelS1cPnoU4ODmayZwouHPjc5+yzpaVQVVWoefGZdF99HgsXHh/uguXL808DGAM2GAyGisewGHCe/YyMK/Yo58XpJvG33Uq5OPXE6W0v5tQW5fRUnY2CwPr1A09A2kD6p+3Es2WW3bBh4OHLBaGXdbDswtj2G7J4koZYFB/6kLRkWEFiUCLKBx6QlgVmaOYNoyxi1k1GzfOS+VKZie+taBtwKKtE1rEgGWqQQ+f31ivzvft7soHjcfFiAED/dNEQqh5/dMC5m1rlO5Jj2nVDDYOHow+F8uXzy7NPlgviCCgA6YCjVkoqOXu2tDrIQjLb0qL2dB2Yu+uFqdZdoQOSg42ROEBKWzmwKTj+AGiHD/pP0+faskSey/33Zy5pRLRjY8AGg8FQEOwFbDAYDAVhWCaIUD2O65XyM5k+g5vDPqT0Naofr2mX+WBmi14WVcFA5VvXJUHTU69RnfpXv5L2d7oiSBjxT88E9e8pU6RVVWVai6RBtrfX5N5b0erxYIjD0OKU6lDd53fUxNhy30ULNdTq5pvTTsuWAQA+88MfAgBe7ZsJINXiaP0Jt8WhcHHqc65DpsyQmMSYCqxjqX/JUgDpvYXq8Wn1mqZM1ZYCVo8NaytnoOOvRh/gaep4Oq1FTkCzBpCaGN54Q1pq6jSb5dVzLhfk1gKnJ4smiDibSAdsT0/6m2TyyfLlYnq4+275vHixvAuuvVZMBnM77k3PQycf7Wls43g+IA0RfOstAEDNjTfqJco1jGRyljFgg8FgKAgl5yXxhEbGO9Ts8WanzGRJiE1Ptmbiq2vrkn0TZ0/TaQCARWf3ZU904onpgUlpGFlNIz2deupNqq1NC3OUcxha7HwjqyQzi5Mtwn1iR0LCSH/0IwDAzq99LelDgjdXvRDVV3wVQOrrCOuk8P+YWZM1kHCEiQK8j3JLlaUct4+VJJLmVhEoNQ0qW+++m/Z5/HFhq/v2CUt+X4cWNQM6LG+6aWnSp1afxemnSNvMhA99gF2b0uNTnpQfi+7E8g7lW25yzYBaRZtoVfSLU6QP3y1t6OhlmN6WLYwXXQMAaG8/HQBw+eWaGBRWJGJSBlVwCoUPJvRaUngaCvd2Z01mszFgg8FgOAxRsjC0OFyGEwxNPZypQ1sgJ6O4UE9/k9h6aMb57GfTPuvXaxEPSGzUrbdKSuGNN0pb9eKv0505hZH58oQMVVNqMylgjOVoSyOYwhuz2njNNmZsAqlMGZXDUKeELKhx8cngPEmpnSjqn2w2zHWJM3Hjovd8BGHJz8HW4CoaSWVNdSP8+DkZh0xR5TDaty/MKmKV+jXaMkKfRdtFn/jEJ9ICO+edJ8ztoovk844dwoMoy/MXpIVi3t6Uan9AWkiK45T2/sn1aZH4bX0NKFvohVPW9CewvfNOvlTCmpwyYCZOlEG7ePG5AFKSmxSJuvvpAefBZZdJGyd+hNkuGjL4/HuiVb/bld2VYzeMXCsVjAEbDAZDQSh5FATTekmemPaLXp2RgqItjY1V2srnqs51mX2ffVYYyPr1vwvOulxboVzLl18AALj6atnaHFRP3rxFjt8cB2OTDuqsOGnhHyR9yjGgnaC82dLOztRe2lbDtG/K9j4tOk0RXHih7qBawJq0Cz7Jf5TW8lnSLhc6kmMbb7z8U57mU66gPDk8br1V2vXrNSMF27X9SNCLjFdrF4Le+95oe/pjoQsituvSv3H+nHQQTpokDDi2v5OlK3kD2juSPvVtp2XupyhwnIaa8jYtEcv7+M1vpH3kEe4hg/eMM8YlfebPl/byy6Xl87nmGmmrvvMP8s+mwHjObJZYXdR28+mpTf5pJc6rI5LM5zOS5T2NARsMBkNBKDkviVNe8ZRO67S5BAx1PKdx0k41WPYvPh8AcPvt3PO3wRloAxa7EIuRkNm99lo6p5AB/MESvRjOgpzK9Hyh3Tc3drHMwGvkTM3boe03tLempJ+2SqEemzZpST+lImF0ZGJB1BPwMdFOFx4/9L4Dqc05Zl955SjLDbxGMqz16xnuQVYr2tbs2R9I+qTESiTI4d3dLSEOJGIaYgoA+NnPpI3LsBJrNqVxwHFdd/5U2KemV22/Qfw7i1uND431BSJkjvyt0V/By6Z91TnZIbx0amvUEAaUraXKR3svgHWX/WcAwNTuV2VDFDbydGAufuyx7GHiuPo8Jm9LEhkMBkOFo2QMmDMzZ7K6F9Wv/sIL0rLuX0iF4moyOu3R27xlC21hoQ2YkCmUsxRnr9CGm5DteIFEGqAqwTAZYLDL5czMew9Xv0lXeKH9UgxaFAUe1kIkwfGOJ/1QN/MdN+muD0sbLlJI+zMZTbwGYrlGlQyl6dDm98EPCv085RRpr7pKtofecLJYMqK5czSml9RO07bmXndd0ueMM0THoGyoWZBNh7Wn+H8cv52w5njtLyAZ79uQMukikLcMGYvcn3yycD/eBws+kaiGMqac4sUFEkrMFUqD2qKU5Zu9Yg+fruO9arUw4rA8KscCn2FcfpL3kVdS15YkMhgMhgqFvYANBoOhIJRMB6eaQVUCy5XHs5YqeX6Yxxov06Aq70/v4g7MR3wHAyFVSagmsKU2AgDNezSsLY6opn5y+ulD3FH5gTKO19+jIzJeGQNIc06ck9RPBv8n4UtflB0y5YhU137+Fdl6221Zfb2lJdW76Phjy+XSqELGadLlAqqToUOQ5oRrr5WWa4XVdOk6ZjpudvakvIVDKzEP0SRAOxq/COwK09XJTNnwvLRanHJKek3PPy9t6tzL9knsQqHdieO7YB9cnsmsXznf+HopUjRvnowxio0y4G0BwFe+Ii1v65JL9AvaK9gpWDKGvw0+Bh5/rl7TtE3pihjTJvZmjtd/maSi8/mwDR3LpXIkGwM2GAyGglAyBhyTzAELlsWrGQPBTC1T9V5dISAtbM+yMGnYD/BBAMDJJwtbZpry1Ela9i+cpjZ0ZS+KFv8rrpBWaUU5J1+EiFfEoIjj1ShCBswFZJmoQoUkCfPRZ9AUpmYq5fhv13MD0zSEmtXXpww4TrQg82DBGjopQhnH4XNFII/B1KyV+xz/Ww17pEyoYmhMWUfvjKTPad3qbI7jxLjkNgUUeH0oo7peCRdbNEkL1CzU1TO6tyX7TpkijrTJ7U/Ihkn64Lr1PPTghQ+d6k1O5cXRRBzOBQTru2mBrCVLzgGQ/mzz1jWk8hCvYwhIksp81ZyfX5WmbS//++y+SYGqp5RahwMyOnCVqtNcgaNOv+/uS52apUpLNgZsMBgMBaHkxXgSAspYHm5g++EPp53ILDqZHCAf07AVGpTTeWLMGJntWD+ci6nu7hVbUl043fL/2Par0+HeRkl17g1quIcFb8q1IHtcGJwaA7cH2d4J8WLBEprLGrreznSaSuMtgI3VYgNbtoxxNyKntrZabdPjh+IGBtr9yjWxhew7c730T9DwSgFTiMqIT0NQ8IkMNM48Yfgl82cDDaPmqSey+yqD47jv7E6ZVhJuRvsmS6vyWnme0MiunYqOsuT9VKE/3cgYL11pukqvdbquncfxGSZi7NolyVccj8uWyb3Ons33QzZdGwCuV+3tHzRLuepHsrhAIqdw4QYyYGotzLlXtXFdz/jM/ZQSxoANBoOhIJRsjoy9mDO4dgqNgQxTCG2NpGw6VU9t3Km7MhmW7uB0WZcrr5SWLG9gScvUn9+iNmWepkEZHJd84bWGMxsZ8HvvAf3BxF1OoKxJ0LhkC1dc4iq7QDq58x6TKJHNepA//mNpAypLknXhhSKMxkYpKEMzfuilP0FrzUTZ5AniFZXz9ikSmWuhsFgBnGCR/3vukTapf4h0XJM1xVWJSMsCo+HmUyUKonnHm7LhF78AANSoFjKNdRYB9FfreP6g+D6SHNrbbgMA8NfUqMvoAEhUlOoykXN/wPOq8qI2AMz6vPxWWfozLPh0wgmigb3zDhOy5J2yahV/oPK+aGtLnQp/82W1NV+vIRSxHyjM/2YN3bg0gmrKK5ZDj59/f8OBMWCDwWAoCMNiwHmpeYkpao4GLsYUlbmrQEo/uCjeTZLz+r+/Lgacf1wssyInKCAlKTHjYrt9e7ov92G5v4ZxMgtyoiOTzItRLcdFOSlvmgNpMycLvUBqxWTYZly0ndmae7VkYU2LGpCpWgCYp7K8447s+Smv8PjxYpz8TBcAiUZeBnrZgYOLjDdc2wlIBcDxCgzMDebgOu88acmEqZ4AaP5dYEMG0pU29Vibt6daXPMYjYhQlsxzU4SN7BvSM30Yfdno7sIQPu+auFrTL38JAKg6+2wAwFe+IgsrUAsDUoXjoYckGoqvlHOlLnsyHjPrbXIgUi7UUDhQw1zkOLha1cR13dnC9iNhUzcGbDAYDAXBXsAGg8FQEIZFqvOC2RP1NDJkJ6pZULFoQCFULjurKsEXNJZk8wfOQoyXXpKW6gg1m7zoEpoYNqtKxn24PVQtym1F2bwwLkaM0SnGuHtqVTMbN6Y78yapCnNV2npxbDDUZm+grvaoaWNGr9ZSpa2DAg0X0dM5PE5miVdsLjokajBkzE9URcMxCqS2n3i8AmmOcJwfrss17GycmtkMAOMb1XnEwUvdWcOgAiNd+iPj0hGKapo4BiyMBuytFdW5r+AVMXLBcRjbodTx2bBYnsE54YOZJS+Vlhap2cdoMT4uOpYD32W69DTlEi/XHpqXGEqo9YS3tYh57p23sl1H4j1hDNhgMBgKQslTkZN4dAZS0wgeL4QFpLMg94k9axrC0xwYzOmgINOmsZ5dwokzThJgwZi4gEq5rtAQg+SBK+DOvHhTdockNznYHq0E8MRqYb5pGVmRJ/1OQOoTQk+0RASLJgVUYDIL1PRlg+Ep+zCsjyg3LSNBvJIul5DesEFaxveFXkg6d8iAyYj1WHns6cmnhPcsWqDMl4OYcYVhRgGdo1R3yJr5W/nIRwZcU9FrwQ0J/pZ5H4xpjMNVw9RqfWfMaZWPJKzcletRho47Fvmp4Y+Gz+634eo6Ci1N8Ey3FKzqeTH7dd4zLBWMARsMBkNBKNk7nbMEJ7aNTVpEBGqP5EyULtGQgrNgXOGC4T5B8gbZcLPaI9va5Dwk2KEtkrbeODSKYVvlUBTmYJCYzXiTXOqYN3DqqdKGIUnKHh5dLoyA7IEtnxtTN4E0eWIGaSxPTNtoUgsxXXk6iqsfsIpAuWoZoSmSttOaWqWQZ5whLY3ulEfoaCBrJQOlQHVfjsuQnbF7fb08k7l9+jyZUx4+Pz4oggySz0DVldCGH6/wUFYgbaWmEFdz0vG6uyUteFTXJxpfVZ8kZF1/vdwrZctDhhpvTaem2vN5sKV6FzzD/jZhvr3Ls5fCV9ZI+i+MARsMBkNBGNa7PZxh48D7ZIJpmgwAmDy9NrsDkNKC2GhFY21Mq4GEHfQ3CvPtVAJCYh0eit1pBo0ZQbkVCc9DeM20o1YfK3bcujOzKxsnTC3w8K5plzmWcqH4v/hFaelcD9chC9d8A5DaH9WOtq4znbf5nMl0YyJI5luWbAxZBpympk/OfFfdKp/JhDoCLWvOzUsBpPdL5SRmvuFYI7PiM9k9T6J86rhuXDiIU7osLZ+tMuHdvfosyjW5BVEiBgcg74caRJTRU5esGIB0fGvfGn0QCxbIc6la8TIAYO+kuUmXzfskYoLm+7ltuoKy2pZ3t6X7drRnr5OXxufEd1te4tlwYQzYYDAYCkLJrBtkOMw0pmeSk/mTa4Wxjh2bltrb0yiz1FgGPqoJk6a3mh5JwyTbBVJG0aGxgHFBnZBgcyYbzItZ1rayIZCQhXnCvqCMlUShO6h5T+WBjCzODKdswjROmh37+iSGtbVVW3VMkxEAA1lCjEqSbZwlS0c8tas45hxIi8fwmXD8UWmgRpBXjIj7JpEjbednzgsAbRrGWgdhcIw2eU8DUmivD30f5Sbz8NrqOOACPwKAdIDef3/2MzCwhKT+sKs+oAs1KM2tCfo0a/RIc6sIe/MueYcsf0HeOVyYAEjHcFxHKR7TI6ExGwM2GAyGgjBi/j0GNHC252ead4F0QgtnfCCtuFddLbNWXlwjZ6vjjpOWDC5kwHG8Lxkxr6XcmML+wOuNlySKCxKFTJ+zOVs60WObZQg+lziTkAhtYTFLqDSZhoiDcMhaeY8s6hS6JKjphQwXSGWWx4D5HZ8XnwEXUOUxgbTU6JgxdZm+HMssPlUpct/WK/dRPV1ssPVzpK1iGtvayKkDpC8IDvg4bpqLPISsmi8NFdQe/Ui3Sfie4GEHK841krI1BmwwGAwFwV7ABoPBUBBKboIgXaeDi+obaX1o/M7U7wwvKlphN69+bOxEis0NIeIkgEpR1wYD7ydW/2mOCVVYypix/XGRHKpfYRB7HLseyzSUZ6XLMgTvhS1NEjQ9UMMdKhzpQIoPxeM5XrowD+UeznewYEhlIotaDfWboyGAsxYl+/JdwjygwcxnYbo7nxHXe4zfF+G+ByL/kYIxYIPBYCgII+aEi2dqfj6YUI68Uowx+xiKGVRqmNn+sL/7yfs+nt0pG7K7A5HR4SrPwTDYfQ51/5WQ3FMOGEyGedou943DBIm898RgWmK5wRiwwWAwFATnvT/wnZ3bAuB3+93x8MEHvPcT979b6WAyHlkcgfIFTMajgUOS8UG9gA0Gg8FQOpgJwmAwGAqCvYANBoOhIBzyC9g59y3n3A3B50ecc3cFn293zv35fo7xzAGcp8M5N2A1ROfcYufcOQd73UH/DzvnVjnn1jrn/rtzzh3qsUYKh4GM/5tzbr1zrmwXyalkGTvn6pxz/+aca3fOveacu/VQjjPSqGQZa/+HnXMrVcZ3OudKtrzAcBjw0wDOAQDnXBWAJiT1zAD9bkihee8PWSgAFvP8h4jvALgWwMn6d/EwjjVSqHQZLwMwcEnr8kKly/ib3vs2AGcA+Ihz7qPDONZIodJl/Ife+9MBzAIwEcCnh3GsLLz3h/QHYDKA9fr/bAD/A8CjAMYBGAugG0CNfn8jgBcAvArgluAYPdpWAfhHAO0AHgPwcwBX6HcdAG4B8DKAVQDaALQC2ARgA4AVAM5VoawGsBLAk/u59hMAtAefrwLw3UOVxUj9VbKMo/voKVqWh7uM9RzfBnBt0TI9XGUMYAyEVHymVLI55EQM7/1G51yfc24qZHZ5FsCJAM4GsAPAKu/9XufcUgjDPAuAA/CvzrlF3vsng8N9UgU1E8DxAF4H8P3g+y7v/Vzn3BcAfNl7f41z7k59KN8EAOfcKgAXee83OOcaddtkAHd57z8WXf6JAIKlZ9Gp28oKFS7jisDhImPd9+OQl3BZ4XCQsXPuEb2uhwDcXwKxABi+E+4ZiEAp1GeDz1pUEkv17xXIzNQGEXKIhQDu8973e+83AfhF9P1PtH0JIvw8PA3gbufctQCOAuTBV+qLIYDJeORR0TJ2zlUD+BGA/+69f3vIOy0OFS1j7/1FEM15LIDzh7rRg8FwU5Fp25kNofTrAfwFgJ0AfqD7OADf8N5/dxjnYXmZ9zHINXvvr3POzQdwCYCXnHMf9t5vHeR4GwCEpYBadFs5olJlXEmodBl/D8Bb3vu/G8a1jTQqXcbw3vc6534G4BMQ88ewUQoGfCmAbd7797332wA0QlQLGtUfAfA551w9ADjnTnTOHR8d52kAn3LOVTnnmiFG8/1hF4Bj+cE5d5L3/nnv/VcBbAEwZbCO3vt3AOx0zi3Q6Ic/AfCzAzhnEahIGVcYKlbGzrmvAzgOwA1D7VcGqEgZO+fqnXMn6P/VkJd2+2D7HyyG+wJeBfFoPhdt2+G97wIA7/2jAP4ZwLNqe7kfgTAUP4bYYdcAuAeifuzA0FgG4HLn3Arn3LkAbnMSVrYa8kBXOucmO+d+Pkj/LwC4C8BaAL+B2HbKERUrY+fc/+uc6wRQ55zrdM7dfMB3PbqoSBk751oA/BXEHvqyHuOag7nxUURFyhjAMRBb9KsQJ97vAdx5oDe9P5RNKrJzrt573+OcmwDg1wA+ojYeQ4lgMh55mIxHHoeTjEesHOUh4EH1SNYA+FqlCrTMYTIeeZiMRx6HjYzLhgEbDAbDkQarBWEwGAwFwV7ABoPBUBDsBWwwGAwF4aCccBMmNPkpU1pH6FLKD+vXd2Dr1q5RrZJmMh5ZHGnyBYCVK1/q8qO4IobJ+MBxUC/gKVNa8cQTLx7UCbhgHhfTDBfdi5efJw5kaW9+x6XDDwYHuqjk+efPO/iDDxOHIuNKxmjLeDjyjZc2B9KxGy/GWdW3N9Npb1/VgD55C1CG5wkR7ztY3zxMmOBGdXmgkR7DeYtwAsUuFnuoMjYThMFgMBSEUYsD5oxNdhv+39WV3WfTpuz3TQNKLKcsoblZ2mOPzW4HUlYyGJM+EpZXj9lCrIlQxnmaCeVHLYPyCuUZLxF+1CClqitZ1rzfhvp++ae7O/muhv+vXSttfb20FMyWLbJfIICalrAMSQD2DR8Gt/FhNDYCAPobGzKXEj7nSpY1kcdyOXYHQ973g43HoTCa8jMGbDAYDAXBXsAGg8FQEEbdBBGqrDQ10ARBdWr16mw7aVLah5oYtbhZs7KfQ3PFccdJe/TR2Wt57z1pqeZUqso2mDMiVMViEwPb2AE6YULah/J46y1pKT/KuqojLTnbwBOoaowx8oA27xufuZZKlTEQONjy7DWDCZSd3n134AHVLIEzzpCWP4qnnpJ2U5BZS6E+xx6nAAAgAElEQVRTvvpjqNLP41tbAQDbUHeAd1PeiJ32QCp2imkwkYcYzPzIz+F2/k9zxWi+F4wBGwwGQ0EYMQYch+Ucc4y0IQPmzEX/RczSpk+XNpyt+J1O/AlB4PlCJtc8UZ0mkZevgWxCO23rrqx5iDP0Di3CR5nyNjuDxZaoVfCWKdPY4dnWlvaZ27QOADB/nFIPOoLa9WDhA2FHnrSjAwDQXN2ZOeHe6pShxY67cgXlvH27tM3j9L4DNWtnrZSrbVAVbGO33GeiGMxaBCB9NkA6Vqf2rEF2Z31IgZMvYcXsxHPPk/C9/raZeoyDubPyQW9v/udwjMQaMr9jy+3hsSguDl22fA+NG5fuS02Z2+J3FzESjLiy3jwGg8FwGKHkDDgOYaqrVRaqlGvixJpk34iIJkzj9NOl3aCLBIV2XYZEfVQX327oXpe9gM0Be3hXpz1OocrOkqlz8WI5/6RpSZd4Ri4nxGF7vJ0VK6TlbVErABKilNjRaSun1sFncGKwJOkzv5kKADhnQaRB5NHlFzXgng+ejG3XLmk/9CEAQE1wUeOVFZeb5kH50hZIDYPjc1uPjF2yKSDUNoT53nuvfKKIKLI8P0b3dGGvrfOkTTSzEPGDC08OoKpnp/7XkGwbKoGp3BDfModNyEJ5yxyz9E2sXCltu65PEWp+PM6ePdljxOIEgFNOkfbMM6Wllshry3sspWLD5fULMBgMhiMIJZ8rOXPFqZikAlUBxZzUOgNAyuTIeEm4SK448wHAnDnSPq3rqO7bJ2yNiRif+tTUZN8pypwnk0GTnTG8QllZHQ3KAPo0ZfRgUj1HC7R1xdEjvNYFC6RdsiTt09C3LXuQHtl5+nSxXdZt0oiGX6apo3sW/CEAYE27yGIm5Uc5vfJKejx68nkxpB58QN/WVdKvvz7tc42smlNdPxluVCttHBgYuDCY953DBwDeeUfa3/wmuy/b53QBnpAB8zlxOHKct7TMBQAcd/rcZN8TTpA2ZmH8WVHcYbAF9y1HJsxronzGN6qWpYM6N5ajXjpNmy6dlrbJ5zcvl9/6PffIbqH2yqHKZ8bIJ2JiULWBQSnUtKNXVnKteRFWw2XCxoANBoOhIIzcHEm6RuMLUyjrU1vV8se1XS4tZ6v7788eYvv2d5I+v/0tY1AlB/nkk8Vgc8EFsnVDsLg8WcGkxWLjraLhh1ObXlNoiyy32OBwVqc8yKooL87MtOM2rH057USKRhdvaCAGsLFWZNN4aWoHX71c2osv1g3LlfLRoBzSh9iYzO+UJq5RejHzgQfSPldfjXJBGEsdBh8AqTYXaxp5fgJ+R/HGxwrZE232FF3Mzmi3BFImp+6KAen5tDWHGhsfRblocaE9t65aNOO+PvUFxYOawg7p+zvp7x9A8uOcoUL4m0tF6HvnnJXsUtO3W/6hgZgvGRp8Z89Oj8eCSU2TM5fA58NLDO+DDHi4MAZsMBgMBaFkDDgOZeSsslutOpyMH7ov7UOWGTMKHuOii6S9885wB6a1iQ2TtrULL5Q29Obz+GQCNQsXyj90cyq96OpI+5RqZhtJ9Ebhubw/2tLnfyh4rBQChaqMoK/trEyf9evTLiQLZACzZp0PAGhRxWFGaJDkxbDVA76tB+Qa5C2kfQAalGJUN00tKxswGc5g5SLzyFlcACZmS2SwoYeecqVItm9nJIO69ZG66GfP/mDmWnhuMmA+P/pGgIExr0UjE1e7tgMAMJ5jKHZoMIwkHGOk/9RgaYSPCiDVPPZY2ueNN6TluIyZ9pVXpvvqQ+romZw5DVs+p1Bb2l9hoAOFMWCDwWAoCPYCNhgMhoIwLBNESMlDFRYAenulGEvk70rCxYCBFJ9q1Xe+Iy3VqzvvTNVX4CMAgFtuEWfeZz8rW6luhc4z9qf61tkrKkZPtbTL75TttEiE/x93HOA9CkdYz5SqXGwmodzOPls3vBjE7dGBwZgnbRs0LOwc9bT97a/SAKCf/Sx7fD5Dnvfss2ck3zGkquae78s/+jA1CBFMkwmD4RpUHezrKw8ZE1TvqbUyxIvPgH7M0GRGbThOKop9k+EYS00PqiaDiylkzWtA+huLyw3HWcuh069cnG+UZ1Xv7nQjsyhiLyVtNTfdJG0Ytxf+D6SeTv7ACcYChif/1a+kpc2AXsvQsade0BknMG9fjt/bK8+Bv688x7yFoRkMBkOFYlgMOAyXIeKs1dg5R3s6ANR1vgkAaLlBGNV48iQ9yO23k2m9kfS58cbLAQD/8T/K5+ajxYGxra8hcx4gZSe8FrIIbj/ppMHvbcwYlJWDCEgndZKFuEzk1E2/ln/Cyi/PP589CL1BjF5/+GEAwF/efHOyy+zZoiGQND+u4YLUWLZuTQ9HmV5yyecAAM1XXAEAaNP2Y+oYSTld+YMMkvKl3EnEQoY5f760TJggKeO4Z1IMfUsA8MILFCA9OVpQB0Kxp0xJvWeMluI1kElT84iz68sSIdtlbOhrr0mrN9S/ZCmANIFr8/a0ZAFUC2h+V0NQ9QFsmyd9xrc/I9vDNO07Rb1dqyrEdKreVFGYxwykLwi+nJQRbzr6EgCpbPNSkocLY8AGg8FQEA6JAdMuFTIB2sdIvjjRkDXMbNFQmweWpZ1++UsAwHiyMtpndCZbtuxvdMc0NZMZrc23fEH+UabVXnt+5hDA4GXrCE6KoYkpqnVSGAYrtg4AJ58sLUPumlc+mt0hpEOM5aO6Qllr/muXGu+bkqwLoLPrkwBSdsUSn9QcHnggiKnC77Vl6JSwt1tukWv66leeQOZgQELZy8EGHNrwKKK45CnR0Kv3GmRVzB+nKkCt0KOxpwjXXzRHx7v+ELZsCRgd5EfSpqUkY6YdnlezthPG29CzUf5R1vb8BPlthOIdai3F0UTyW9qUQ89JJzV+rgqDF+1KFgDgTSprZo7FvHnnAABqWM0LSGIpp69aJZ85iNkpTEjib4QPQtWZar3suLQlcGhrzeXBGLDBYDAUhGHZgMOA9LgoMu2FMyYpE1CbTBIIDSRTSv9DDwEAtmnbpBTvhz8UBvzlLy9Nukzt09nwF7+Q9hKx0zQqix1fvzfZd+FCmUW5cjJjs+Pg9ZApcJZ7773i2VkMkgZe4/jaKN2SOzDhBMBOTbhouF+jFMiANUW4iXSLacYAapdL+6Mfib2utlZYLevuhIkCb71FKvCktsJC/vqvJVrlxY+LZnLHHcF9MDC/TLz1gyEpFBM7E8Jahr/9rbQagtL87L/JZ1UJd88RdvZG6sZAW5ukw/7X/5o9XJz6CqREraFd7ftqs6cGM//rXwcAvDvxtKQPf5dDaVGjgSQBKqSOB1KBHdnIibchafIruqV9UV8lt94qrfci3IsvviTp89BdmhLP9w7HPVUJqhbhhVJ9V6GPDWpOAWYDNhgMhsMKh8SA82LfOOsmzLdR7WX3/lRaMt8wIFKnesaIMiGzSQ25ZAKM9QWQ0tdzz5X2b/8WADDzsjeyFwDgtMh92azMcNIksdMNSJ9G+cRQDhVzmDCzFcp86WpnXcPgJpJylJRFaCQHEs/vOqRlPBn1QEyZktk1JMvYtEnsZTffLLb4LVvo2Rf6xXERsjoW1QfKT8vIIE5fpWYRDhLaDSm0yOHQ2yYM+CtfSbssXSxaGgu8U4HJsz3zEiaTffNann1WWn3mjXNSBkyUKl32UJGUnBwqpjdaPfPNTolHD0kzb5mKyDe+oUs54TZthbk+8kjqX3r7H0Tzm3adbmDNSR3MOxvT8Z4s6hDle4c1p2KUSrswBmwwGAwFoWTFeGhHndyts9ODOm0x7YczHad7IGFjTeedJy2NYLo9x6SZblRb2O/Vi388mW9I37iNDFHpREuLMGCahcJVwHmZtbXlFwfMWbdf582+WeIBp+mq6Urxqtd0vJl2okxJIz7zGWkZj6m0NkwM4rM87zyx/cYLG4ZhxrSjMzpl7FixCW/YIC0fV1xyESi/WGuydd5f/xzJ5qwiayOlC+2Y1Mji1QP+/b/PHCssko922adelyTikOVpQo1sZptqO7/QVFMOUKY9qoMjJJZxJFLR2N2b8rw6amDRuj/PrxLmy8ikcBEG/p/6EV7QlgNR7LoMZgj7TFooTLju/v8pG26/HQDQEEZMcH0zhhXpi2Ga/jY2e3lfjIQ8jQEbDAZDQbAXsMFgMBSEYZkgQkcRfRGA6k9xpZ3XX5c2jBin7v+JT0jLmBvV16pV0wv9drhOvBkb1fQwmR4iDUfLxPuo927nJElppopJ9YSXFDo9eLhyBJ0q1ISp9dK6QNPBZz+bFsup69I0TYbfcOlXFcbmYyS057u3pufhI+Ij5HHphAtNNlSXIysPF0NO1LZ0fKT3US6rjhC0LNA0RTlMY1WeuCgvkHoUY0+aCiBxRj8YhF8quhrFBMFHQzlnVN3QZAekS79QiHqecizGQ2SuR29yb7WYHHh7HAv8zGi7sD/NXb/6lY5hUFCXAkjHHAAsXaKmGwrmi1+Ulg+Z2UxA+gOi4DmYk3i+7KGANBHDivEYDAZDhWJYDDj0RZAV7dsnhVzmL1RqRDrxUw1HCz0MpJ6f/jQAYFtTtijPHzBA7YYbki4dWtwlSVQkLeNxw0r3kdWcjJEzGVNPw3AdfhdHyxSFMNwlXrGB8ud9sb377rTP9OkSBjVpkrTTl8t2ij72Icm+2fNlwgCRXXWE2kl8TfFCBJWA+NoZuTTtg7IqRRIHFWpxVEfo+eJ3seMurNeqIZSUObsunReFDALobxGWXEVB8xrYSdWUroBg86dQdCIGETLgnVwhR2Wcl+YLZHNdeKupWKTw1uzZ4lC+7DLZGq4Kkrxv6F2+6ipp8/KzyXSZrqxBAZv3iROW74fwPkrlkDMGbDAYDAWhZGForLPMojwvt8tMN1eL5SRxSOGyxWpr2VgvzLdX2dJ46D+aZolgrafWOFaHdmONml6TlPYDOKn26Qwah+fkpRaGaZzlliTA667rFab0sRbRLj72ZaEG/7JeWC5TXIGB9uGYaVCcIePnI2P438xqDWtTFhaGFdV0Smp4jV5cZ7SuVmOkCAGlK2RSCoQsMc5IpTuh9SKRazOpcZhOr536tOgLE2gbWMuTtCxwNOxslaSJHiWzSR0kOicCG3OPJgy8956m1TOMS+ngq+1hkR9BuYSf5YGaMscho8FiLS4MdaRY2Pc//AcZXAw7+8xHNYUrVLc6dMCrcGlzrtkUJV0EF7Ot+vjMYapVQ+YYHomSn8aADQaDoSAMiwGHkwgDqDmjcbbYt0/sNeMu/E8AgBktaZGNzbtkVrrru/KZJGFam05/nO3DE5FafUSKvfzcSfRDuxKOkMnxeLwWms+iGPAMI+OyO+WWJACktsk6UgIWVdeCI5/R5Vzev/mrSR8Gr5MVxcyUDt9MsosiKfbTJZ3Xdcp8PbU+WGCIwlV6MqNNT0T/gJ6otnbgXF8OWkaeF5uyoZgps2b+w7AFIBlU1cp0G+jG/9a3pP3Sl6QNyh82dInW8Ec8THuHtLQfX311sm+tDv2GWklf7lcm190rLZliqM3xWRedikyEQSNxLR6uIkSlgppSyDbjFG2KMnl2ce1bALubRHNgkgsV8DPPlO207wPASt0nliVNwiy2n4nGKhGMARsMBkNBKFk5Ss5KJAD8jiuP0H7T0ZEu/sh9aXOhnfLBJkkfXLJE2j+9+e30RDoNvrlW5o7O5bKZs1doa6Tjmew8XsCQ5wsXCi232NQwfpbXPZ4XzpYrkqpA/6jn/076/NHdQdk9YIDbeW+9eHprelJWu7tWtj36lDyrWbOENUxlmnloC4srGvEiI8PZ2LENSRcys3JL9+aYpVg5psiW1k0XX8XUKwM1i34KvandKo86+joon1eC2oakfaR2rJdKmTF2HkCNsu7+pmwBqcROGS0kGh42LHpULuD18hrj6BkqAXlxzWSgZMe0I/dXix28qn1N0mfVLvEF0RTPUO7/83+yfYGB0VFk2gywigvzA6nvwOKADQaDoUJhL2CDwWAoCMMyQYTOK6oJVN9oXqBKQa0q0K6C5A3h8xdeKHyeWvIDD0i7YMG0pE9jZ/Y8VBO4b6iKMduQml1sRKfDrbpkwXilR15a6d4WkUcNPWf0FrCAKfUuIHXU8eb50D7+8czx2zvHJ10oj6VzNI2WDyovfo+6HL9jyCDzQlWFPvbY1ARBdbDcwFugiKjq6vJiibrZdHF6L0l1L3XG1elB9r4gFbtqNAW2P9Cpqy68UP6hM48HpmkitDtF+i8dg/GjCB8JTQ/lYk7Lq4QXr/5BWefte+qp0rJMAH/j9LlVdck43dmShqC+dr+0fN/wPcShHOZjMFeDP5/eKEmEobXh+65UsjUGbDAYDAWhZMV4QscKMPiKrGGYGGft1lY5EMkEnR+cecI0WQatc0YjQWCmYYiYCcSrI5PplTMDDhMFeJ010HXvSP/Z6pp6GeEzzowM+MEHpVVh1Gmc02nVQYUdPojurFdq81GSZHFs4MCoW62rHsfUjFCGHG6OQ5HKBZQvx13sE+P2cFGRmRyQHFS6c68y4L26vT5ksvwR0PlGgVCTCffVi+J4J0Ok5sJrDB1u5cJ8iTAcbrBCQRQJFaehGD3FQzbb3S0OyrCmNf3SYcRgiDDMjcya52S0YZySPxIwBmwwGAwFoeTcjyyCpkGyBc4iYU0SgrbaeGbjMcJZk+QuWW9KkwXG9yhtDkvpcyrTncdrsHyvptKWc8pmHigHht1QTrS3L/6oJKVUzZ6ddtKbfLtHWMI0UgLaiUmxQ8rBqZ9r9jWKzfl9JoKseCbdlw+JFVGovnC7DojwGcahSOUCMi3ePkMrSW45tLjQLgC0tYlWcOkVfw4AqL9ato/nyiMaxb9XVy8BgJpeSZ3dXd2QOS4RamR9HdKS+XIhk5FYoXekENpO47Uj42JOcZEoYKBypVnfCTjkwmQiHp8hpnxv5FXFpUy5D68hTtsfCRgDNhgMhoJQMgbMWY6zRuyp5PYgIzMxgdEGw2MweYMB8KFJjDPljCZNHFirYRGcJkNaRdqghqWNm2S+iW2+5RiwTuTZ8yhbypReYiahTA2nbq5vtUkNmVQdaCSj+hFWWY/cwGQLJMuTw4eo++5tFIYNbWv6NI1Zhd2bk1pabqCsOS7D9QGBdIiF7In+CQabULyTJok2QvN87fLwTMJ84xwWssHQhB8XgIntkpWmxXFokqE2cC30pC6q0Nm2hecnfeLoJR6Dmh+H46I5O9Od9HjTLpYvt3XLb5+KH90lQFofjM8h9l+NZGKLMWCDwWAoCCVjwDFT42zC2YMzdZhiyHBHbov3ofc0LE3HOstzdMXarm5pF12m02AYMqHUmWXmjlIGF0c/lJvXeDDwOhlHmzC2ibL8SlImsjYwELICEdUJTve8ebr4w7WYeAK1Jc/Fy/J5j9KxzoAiKCWrqe7PXmy1PEyu4BzGd5a7vHl9jBMnwyJrC9Pd+T9ZWqhIAClLC5kqWWy85FNeWjF/R+zPfeKY5XKWaXht/D9hvhyfUeJATWAYr6FA9EEs0sG0qC1aQvneQLWi6qDCHa8Ca2qS9wVX5gJSJTAuDh/LfCRgDNhgMBgKwoi928l8aTdk9ENYjz1M2AIGLoYY249DkAmTLS9YIHNJa+tZyT6tkbmTmS6c0cqpMPjBgCyCxID2LcopLJheN2+e/MPoB1I0MgQeJFzPJaR4wIAalrv70iLg7F4dFUipgjBijoNyZmj7A21/eUXtYy9+bN8m8w9jYWPWOhi7DbcNNlYrTa7JvVFO8Y+cAgsXI42N7/drmhtDrNg370URGXTP0bjt1tbjk208LGVMzZyHG8nMQmPABoPBUBDsBWwwGAwFYcRMEFT7CQZEh76eeLVTagtxSnLo2KDGwtC0WG0L1Tc6MKKcgLIrVnKwoFmHKhPNAHlrVo0dKzV9ea/VXGNMZbB1q7R9q9M+zc1TM8d4V4/bFzlLgYEqcqqCV/7cPtj4iMd2iOGsQjGUSaxSx2qMZOVpDcVDo7T7VKbjrpX18rjWIICB9p1LL5WWAzEuLB7+H8eQqdNvcphP3jo5cziiVDV/h0Ll/0oMBoOhQlFyBjxYOBpnE9YbAYCTTso/Rswi8gL3DyQ05HBxXMSIr/9g7id2ioXPYzAMxfiOJByInCt9bBWFuGAWGtMStAOKZrXOzHyuoaM5jFcdLI+YK8H0pdyzMXq/jGaikDFgg8FgKAjOH8SytM65LQB+N3KXU3b4gPd+4mie0GQ8sjgC5QuYjEcDhyTjg3oBGwwGg6F0MBOEwWAwFAR7ARsMBkNBsBewwWAwFIRDfgE7577lnLsh+PyIc+6u4PPtzrk/388xnhnqe92nwzk3YIU559xi59w5B3vdOcf5V+fc6v3vOfqodBk755Y7595wzq3Qv+P332t0cRjIuMY59z3n3JvOuXbn3KcO9VgjhUqWsXPu2GD8rnDOdTnn/u5QjpWH4TDgpwGcAwDOuSoATQBODb4/B8CQQvPeD+cFupjnP1Q45z4JICd/rGxQ8TIG8O+893P07/fDPNZIoNJl/FcAfu+9nwFgJoBfDuNYI4WKlbH3flcwfudAojt+MoxrGXCCQ/oDMBnAev1/NoD/AeBRAOMAjAXQDaBGv78RwAsAXgVwS3CMHm2rAPwjgHYAjwH4OYAr9LsOALcAeBnAKgBtAFoBbAKwAcAKAOcC+DSA1QBWAnjyAK6/HsBTkEG7+lDlMJJ/h4GMlwOYV7QcD3MZrwdwTNFyPJxlHFzDDJW3K5VsDjkTznu/0TnX55ybCpldngVwIoCzAewAsMp7v9c5txTAyQDOAuAA/KtzbpH3/sngcJ9UQc0EcDyA1wF8P/i+y3s/1zn3BQBf9t5f45y7Ux/KNwHAObcKwEXe+w3OuUbdNhnAXd77j+XcwtcA3A5g96HKYKRxGMgYAH7gnHsfwI8BfN3rSC4XVLKM+T2ArznnFgP4DYDrvfebSyOd0qCSZRzhSgD/UsoxPFwn3DMQgVKozwafn9Z9lurfK5CZqQ0i5BALAdznve/33m8C8Ivoe1L+lyDCz8PTAO52zl0L4ChAHnyeQJ1zcwCc5L1/4MBus1BUpIwV/857PxvCOs4F8MdD3mlxqFQZVwNoAfCM936uXvc393ezBaFSZRziSgA/2s8+B4Xh1oKgbWc2hNKvB/AXAHYC+IHu4wB8w3v/3WGcZ4+272OQa/beX+ecmw/gEgAvOec+7L3fOsjxzgYwzznXocc73jm33Hu/eBjXOFKoVBnDe79B213OuX+GMJv/OYxrHClUqoy3QjQ4vnTuA/B/DeP6RhKVKmO5MOdOB1DtvX9pGNc2AKVgwJcC2Oa9f997vw1AI+QFR6P6IwA+55yrBwDn3Ik53vCnAXzKOVflnGuGGM33h10AjuUH59xJ3vvnvfdfBbAFwJTBOnrvv+O9n+y9b4XMqG+W6csXqFAZO+eq6ZF2zo3ReyjLaBNUqIxVFV4WnOcCAGsO4JxFoCJlHOAqlJj9AsN/Aa+CeDSfi7bt8N53AYD3/lEA/wzgWbW93I9AGIofA+iEDJ57IOrHjv2cexmAyzU05FwAtznnVjkJKXsGwErn3GTn3M+HdYfFo1JlPBbAI865VyHOjw0A/ulAb3qUUakyBoC/BHCzyvmPIayyHFHJMgaAP8QIvIDLphaEc67ee9/jnJsA4NcAPqI2HkOJYDIeeZiMRx6Hk4xHcMHlg8aD6pGsAfC1ShVomcNkPPIwGY88DhsZlw0DNhgMhiMNVgvCYDAYCoK9gA0Gg6EgHJQNeMKEJj9lSusIXUr5Yf36Dmzd2uVG85wm49KiqanJt3IJbQMA4KWXXuryJVwhw2Q8EAcq44N6AU+Z0oonnnjx0K+qwnD++fNG/Zwm49KitbUVL7545MjzQOCcK+lyQSbjgThQGZdTFMSg4IrKIWz12eEhlqnJ02AYfZgN2GAwGApCoQw4ZmHvv5/93NeXbfNQPcQd8Lujjsr//khgfXnaA5DKmrLdsyd/P2CgjAeTJ3BkyNRgKBWMARsMBkNBsBewwWAwFIQRN0HEKnCo6lL97e2VdtcuaWP1+J130j5HHy1tfb20jVqSurZW2u7udN8mXR3quOOy10A1+XBz7uXdTyxLtj092TY08/B5UKZjx0p7rJZFCU0S/J/7EpUsR4NhtGAM2GAwGArCiDFgsqiYcYVMKWZaZLOTa7fJP6SzXSvSTrVKfZsmAQDerj8tc57w+GRnK1dKu0lLdjBmfF5OCCqvqRIwmHYR3gPlzm1dXdJSFmvXStvRkfbhPpRfS0u2nT493ZdaBuU+SR5L8izzHHbGjg0GgTFgg8FgKAglY8BkY7Q5knGRRZGhbtmS9iETIhs7f5auWv6j+6QljQrj0zo7pX3rLQDAtD2Py+drrpG2Z+Aq81uOm5w5D4k128mT+pN9d+1K56RyLRRHWZPxkrHyfkIRcBvZ66WXSkvm29Ym7f33p3347B57jJ9F/sceK3R21qx0XzJgPipqF6fqouMnn5zdL4QxYcORDmPABoPBUBBKxoBj5kswgmGzLpQd2g/JisnCEipH2kSqFVK6+5Qdf/Sj0irF3tbXAAAY39me7qsXc0Lr5Mw1nrNAGG9/zvxDVknPf9HIi2zgNZLFktmv1hXXwogGypvPpap3NwBgRr1Q4xlrHwYALGrqTDtduUTaL0ehE/qgvv/w5GRXMmwqJrwGXtO770pLRgwAJ5yQ/l+uWobBMBowBmwwGAwFYVgMOI+d0ebLlrGjtA02v78x2XdbrTCp8dU7AQAbMRMA0L5J2t4V2WMBwNILLwQArGuaCwD46U9l+/XXS/tm41nJvmR/U3vk+FOrlUl/Q1bBrpozRz6fe27Sp7FRmHRfH+BGtRBlFkMxX2oVcQQDP9PeCwALFkhLLaO/ug4AUPWDb8mGW24BALQHJ/z9X/81AEADGjCD2sZ11wEAPrdkTrn2rH8AAA94SURBVLLvtvqpAIB77pHPZMBUWmgbDm3AE7VIX7loGQZDUTAGbDAYDAWhZDZgxoGyresT1jm5UU9BI+Se1EA5vv0Z+UfpUV+tsE9GQ+yuPx4A8M1vpufpmi4M90qN4b3sMmmrXvw1AGBGGNxLGsZapWx36CrWpI5nnJH2qZdrqK4ulgEzQiA0f5P5bt0qLU2zNJVTxFdfnfZhxmBnYOIFgNb/8lcAgKozzwQAtN11V/JdGw27S9QWfOWVAIDNY4XthjHDtPHynPxuU7RM4lAFlQyGIxXGgA0Gg6Eg2AvYYDAYCkLJEzHoUEt0UOrQ9KStX592op6t+ulUaB/NZ131WzFBrAgykZcvl5aqrmrHifNtRsfb6c60h8RZALwWdejRGSjXkv5bZIgU5RmG9cWJK3Ss0ZdI68uMpm1Jn53V4wEADT3q/KQtYpLYLXYvXAoAqAttHfz/4osBAC93Hp85XxhGxut76ilp6QCk6YOiZ4oykIp/zJhizTwGQ9EwBmwwGAwFoWQMmM4Y7IjyfUmNmpsHdnrtNWlJk0jl7r4bADBf04v/8i+nJV3o7HlY8gcSBqYRUkBPjreHjC5mwkrTmP+Rt0vRyCsTydAuMt6GdnFAjie17EppcwNviGoE6bMebEX1IgDArCWfTPo895y0S7Tr3PYn9WLk+Uya1JDsW9MlzHrGYrmo1lYJc6NMSbh5zSH27bNEDMORDWPABoPBUBBKxoATpkZjX7umBNNYyxCwJO8YAyvC0GbLDIobbgAAzA/yl2+//f8DkDIrnpeMa/KsINf5oYekJRunAZl0TGn0jDBuSw+4racm5y5HD3mp3SztqKZZ1D3ww+zOlGeY4XDeedJSYO+9BwBY0yTMt7tDNtNmCwBLG4VR4w6lwsx2UQ2lJsz0+Kd/klaTWWZqRszOOadl7idv7T6zARuOdBgDNhgMhoJQMgZMBtWviQxVpGvz50tLIzGD+wHg9dcBABvPuARAyman0sBLxhrQs4/N05KVc2TnnzwnEQyJ7TbMAHjpJWlJj8nOuUYRmVwYARBSwTJAWGA+SXJZoQksvD9eP43CYR4zZUkZ3HQTAOAuTW65+WZpq274z2kfGoFDpgukduTHH0+3MSuEF6f71LZlC+WHTL7MRGwwFAZjwAaDwVAQhsWAw4LasRnyuHmXZPbJKxbecrZENzysJsbPLdYYXrK1CROkDXORWfVl8WK5gWphwMnSNxODgFO1ISfsjBdHCqaMeHdfau/dFRSML9JDz/vJW8IJr7wiLe26cS5yGHSrsc78bk23yCuJoOh4NTo40go+f//30rJYEW3nrLgDJM8Bp5wCANh9xZ8AAH6qBd7JgI85Ju8uDYYjG8aADQaDoSCUPAqC5JVBECRLJJ8h0WJwwxVX6AbafAkyrnC9nKi+4cJW+XjvvbyOdE5Zu1bs0SeeKJ/HjZMylySIPH8Yo0rmWZSH3nsx4VJzyFtkNKntefTR2c6MtQ5T1fQmd/eKXDo1LJs2883NYqvdfP33ki6ndT0h/yxbJi1twQy/4PJPwXf9c6Q86L13y2YGv1DWLEsK5C/UaTAciTAGbDAYDAXBXsAGg8FQEEq2IgZj/6kya4RZYorg9jCyibV86ZjbfMnnAADNDHOiZy8MLdNA/2c6xJlEkwdNGzwfkOYPMM9j4cLs9dMXFyYJcFtRK/Y6N/i5k2SXD31I2rPPBgDsrZeCOzV9st5bkvQSdNq0SRyNND1QXt/+trShDNpuOl+Ox53YqsPt7b6pA67t4TulDeUPZP2BBB+rrYpsONJhDNhgMBgKQslXRWa4Wbwa7pQp0pL1AkBd+8sAgHUt4sBpHrdXviDjVe9c/01fTfqQYXFl4zfXyhxyxx3ZrkDKZrdoaBmjtk4/XVoSuzLOw8jF7kkSvkdZ8L7a2qQQzoyAdr7dKcx3Wq2Wo+yQJIs1bWnxHSBbhKimV0uKfvzj0qrANlYL83384XTfuBwlNR1up7/QYDAMhDFgg8FgKAglD0Mjm2TLAPwLLpA2SaMFkrKTU1lL8utqSIwSJciqgZT1zZwuJ1y+XBjeSSfJ9tDOq7kBeOMNacnGGKVF1hcev5xBmZJtUhaMPpvRqhpEb8qAp0HZLEtUaip4ox6L9nHmcgBIbMh77/he5rwd7dk+wMBwM2YrM7SPTDgsq2lhaAaDwBiwwWAwFISSMWCyJNpgmeRAtpTYd0M3OTsxvZghEqRjl14KILuiL731P3kw69X/sz+TNrThknWR8fLwPAbZWZjsUM7sjOKiPBgBQrb5/CsikzFj0tRqJqE010pnFku6W23mfD5zZ+1N+uyFREE8+GD2GGS74cLTsRbBx0sZU+sIZWzRDwaDwBiwwWAwFIRhMWBGPgApAyLzYdXJxCFP2ha627kEkaa4/rxT0mJpx2Us8dNPDzy3kuOE5ZLJVfXsTHfS/OdpTUq/6vXcSsfWdDZkrjm8p3JkaYOxc4qW6d6hvZX24sbGbNx0vKjpihUpa6Y8qIjQhs6+YSw3n3v8HMiaeYxQM2H8eDnK2GAYTRgDNhgMhoJgL2CDwWAoCCV3wsUpxzXLH5V/qAtrKjGA1AShZolVD2X7EqHfjis4/K//Je3JJ0tLZ09dmIlBLxt1Z+rB2lJ9Dq0i5eaEC6+Hxc8Y2hcv6MEQsLywvS1bWIbuPW3F5NDZKQcLQ8t4PIqLIqUYQxMHs565L52vfB68xty6xgbDEQ5jwAaDwVAQhsWAQ3bGZdboEGI77a235B/SptAbo7To1bWSQst0ZTK5P7lY1n874abjB5xT69AkjIsMa2/rjGTf6unyf1X3tsxO67rqMscK2Vk5g9dJOREULRcQCUVMJ2hHR63uW5s5Vh4bjZlvWmtZ2lBD4f9cGIMsmccf3ygp4zt7bK43GGLYr8JgMBgKQslswEx2YHgTF7eY9vnPyz8sMRlUvnl1kzBbstjTWpSpPqTG4Hs2AwCmstYlAFx1FQBg4sLxmcORcYeFdXhcllXcXTs+c80TJ0obhqGVc2gUmSmZaGyb5bWTjQKp3ZiMdPt2aTdvzh4zBGXJZ0mWzISPsNolzfhMg57RFBmQSzfEDIbDDsaADQaDoSCUfFXkceOkJQN+6il5xy9YsBQA8KezNiZ9yKgSG+zjGinxb/8mLakXV+kFgL/7OwBAlX7XoCUrG3iwxoDSdWapW7xoMJljuUU+DAbKiYyd0RuMOCAb/YNL+5M+zzwn8idr5jPjvoyYCINHwkWPgVT8PA+TYABgRqPY6XGv1qjctUtarsasakgYOWEwGATGgA0Gg6EglLwcZVyE5bXXpGU86qOtk5M+LIxehf5sp0sukZZBvqGR9tlnpd2wQVpSN7bBEse75y0CkNoyeRimwlYC8w21jLh4EMttks02sPTkirVJn5NO0mL3e9YBAHbPk6Lqq1bJ95RNGAny2c9KSxv6zEbVWpL1n4IL/CddOXnHDmlJy3VAMPohXL6qnO3sBsNowhiwwWAwFISSMWAGKpDEMluLJJYENcxqo62xsVHmge5uKcZTWyttmzrUwyLrNUrLdqJB+2SvIyTLR6k5Mm/xzRCVwsjiuOXxtboIJ2lsjkG3md9p6EKdVt+Zv0FY8vw5TQP6YFKrtA8ul5aB2XE4BDDQIM1alZpa954uB1UpMjYYRhPGgA0Gg6Eg2AvYYDAYCkLJTBBUMWl6oJrMBA0iTH2l446aM/uEDhsAeOWV9P89e8T0EIewDWZeAFLzSHytlQZeN00q/dWSUl1F9Z9CCXOFaVqgEOhIY8uiy1zGOjwOvXu0AeWZIAi1Pe2tl2SX3p7sNRsMhoEwBmwwGAwFoeR5omQ8gzGfvNRXFpFhn5gBH8r5D2eE6dYAgGotVpT3NKdL2Nm+D2Q3j5kj676R1IaaCbUKaij8js8uDN8b8My2Z7cbDIbBYQzYYDAYCoLz3h/4zs5tAfC7kbucssMHvPcTR/OEJuPS4giU54GgpDI3GefigGR8UC9gg8FgMJQOZoIwGAyGgmAvYIPBYCgIh/wCds59yzl3Q/D5EefcXcHn251zf76fYzxzAOfpcM415Wxf7Jw752CvO+h/lXNulXPuVefcw3nnKBqHgYw/o/J9zTn3t4d6HIPhcMVwGPDTAM4BAOdcFYAmAKcG358DYMgfv/f+kH/cABbz/AcL51w1gG8DOM97fxqAVwFcP3SvQlDJMp4A4DYAF3jvTwUwyTl3wTCuxWA47DCcF/AzAHRpTJwKYDWAXc65cc65sQA+BOBlAHDO3eice0HZ0C08gHOuR9sq59w/OufanXOPOed+7py7IjjXnznnXlbG2uacawVwHYAvOedWOOfOdc592jm32jm30jn35H6u3enfMc45B6ABwMahuxSCSpbxNABvee+1HA8eB/CpYUnDYDjMcMiJGN77jc65PufcVAhLehbAiZAXxg4Aq7z3e51zSwGcDOAsyEvvX51zi7z34Q/4kwBaAcwEcDyA1wF8P/i+y3s/1zn3BQBf9t5f45y7E0CP9/6bAOCcWwXgIu/9Budco26bDOAu7/3Homvf55z7PIBVAN4F8BaA/3SoshgpVLKMAawFcIq+yDsBXAagpiSCMRgOEwzXCfcM5MXAl8OzweendZ+l+vcKhK21QV4WIRYCuM973++93wTgF9H3P9H2JchLJA9PA7jbOXctgKMAeYHlvBjgnBsD4PMAzgAwGWKC+C/7v91CUJEy9t5vh8j4XwD8CkAHgPf3e7cGwxGE4aYi00Y5G6IerwfwFwB2AviB7uMAfMN7/91hnGePtu9jkGv23l/nnJsP4BIALznnPuy93zrI8eZon98AgHPufwP4yjCubyRRqTKG934ZgGUA4Jz7U9gL2GDIoBQM+FIA27z373vvtwFohKjIdA49AuBzzrl6AHDOneicOz46ztMAPqV2ymaI82d/2AXgWH5wzp3kvX/ee/9VAFsATBmi7wYAM51zzFS5EKKSlyMqVcbgNTjnxgH4AoC7htrfYDjSMNwX8CqIZ/65aNsO730XAHjvHwXwzwCeVRvi/Qh+1IofQ+yEawDcA1Gjd+zn3MsAXE4HEYDb1IG0GvJiWumcm+yc+3nc0Xu/EcAtAJ50zr0KYcT/z0Hc92iiImWs+LZzbg3k5X+r9/7NA7tlg+HIQNmkIjvn6r33PRq+9GsAH1FbpaFEMBkbDOWFkpejHAYeVM96DYCv2YthRGAyNhjKCGXDgA0Gg+FIg9WCMBgMhoJgL2CDwWAoCPYCNhgMhoJgL2CDwWAoCPYCNhgMhoJgL2CDwWAoCP8/omE7OeYNa6kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd216ee2cf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_weights()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also print and plot the so-called confusion matrix which lets us see more details about the mis-classifications. For example, it shows that images actually depicting a 5 have sometimes been mis-classified as all other possible digits, but mostly as 6 or 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 956    0    3    1    1    4   11    3    1    0]\n",
      " [   0 1114    2    2    1    2    4    2    8    0]\n",
      " [   6    8  925   23   11    3   13   12   26    5]\n",
      " [   3    1   19  928    0   34    2   10    5    8]\n",
      " [   1    3    4    2  918    2   11    2    6   33]\n",
      " [   8    3    7   36    8  781   15    6   20    8]\n",
      " [   9    3    5    1   14   12  912    1    1    0]\n",
      " [   2   11   24   10    6    1    0  941    1   32]\n",
      " [   8   13   11   44   11   52   13   14  797   11]\n",
      " [  11    7    2   14   50   10    0   30    4  881]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAEmCAYAAABcYEo9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAHVlJREFUeJzt3X+QXlWd5/H3pzv8hjGYjhQmwWQHBoeiSsDeiDJSSpAVZIDZmnGhVgQWN7uz6IDOroOz1jLrbtXqjqWOzhS1mQQNDj8NUlIOww8BZbQkGkLkV1AiAkkmkLT8EEEMSX/3j3tam6bTfZ/ue5773Pt8XtStfu59bp9zL935PqfPPed7FBGYmVn3DdR9AWZm/coB2MysJg7AZmY1cQA2M6uJA7CZWU0cgM3MauIAbGZWEwdgM7OaOACbmdVkTt0XMJ72OSgGDhjKWscxS+ZlLd96R+45nspcfhs88cTjjIyMVPq/avB33hSx61elz49f7bg1It5b5TVUpacC8MABQ+x38mVZ6/je1ednLd96x+ho3hA8MOAQPJ0T3jZceZmx62X2efPZpc9/+b4v5W3VzUJPBWAzs2kJUDs+/ByAzax51I7HVw7AZtY8bgGbmdVBbgGbmdXGLWAzsxoIt4DNzOqh1rSAs36MSHqvpB9L2iTp0px1mVkf0UD5rYdluzpJg8DfAacCRwHnSDoqV31m1kek8lsPy/nxsBTYFBGPRcRO4FrgzIz1mVlfkFvAJSwANo/b35KOvYqk5ZLWSVoXv34h4+WYWSuMzYRrQQu49odwEbECWAEw+PolufOnmFkb9HjLtqycAXgrsGjc/sJ0zMxsFtozESPnXfwQOELSEkl7A2cDN2Wsz8z6xYDKbz0sWws4InZJ+jBwKzAIXBERD+Wqz8z6hCdilBMRNwM356zDzPpQjz9cK6v2h3BmZp1pTx+wA7CZNY9bwGZmNXEL2MysBg2YYFFWOz5GzKy/DAyW36Yh6QpJ2yU9OO7Y6yXdLunR9PXgdFySvpgSjN0v6bhx33NeOv9RSeeVuo0Z3LqZWY0qzwXxFWDisvWXAndExBHAHWkfiuRiR6RtOXA5FAEbuAx4G0UenMvGgvZUHIDNrHkqzAUREXcDz0w4fCawOr1eDZw17viVUbgHmCvpUODfALdHxDMR8SxwO68N6q/RU33AxyyZx/euPj9rHQf/6w9nLf/ZH/5t1vKtvIEenwVVRkTe9ChqYl9q5xMxhiStG7e/IuWgmcohEbEtvX4KOCS93lOSsVLJxybqqQBsZja9jscBj0TE8Exri4iQlOWT0F0QZtY8+dNRPp26Fkhft6fje0oyNqPkYw7AZtY8+ROy3wSMjWQ4D/jGuOMfTKMhjgeeT10VtwKnSDo4PXw7JR2bkrsgzKx5Kuy7lnQN8C6KvuItFKMZPg1cL+lC4Ang/en0m4HTgE3AS8AFABHxjKT/RZEFEuBTETHxwd5rOACbWbOo2lwQEXHOHt5aNsm5AVy0h3KuAK7opG4HYDNrniaO3piEA7CZNU4jh89NwgHYzBqlWJOzHQE42yiIyeZXm5nNmjrceljOYWhfocRUPDOzzgip/NbLcq4Jd7ekxbnKN7P+1euBtaza+4AlLafIKsSiww6r+WrMrAnaEoBrnwkXESsiYjgihucPza/7csysAdwFYWZWhwY8XCvLAdjMGkX0fsu2rJzD0K4Bvg8cKWlLmlNtZjZr7oKYxhTzq83MZqXXA2tZ7oIws8ZxADYzq4MfwpmZ1cctYDOzGrRpFIQDsJk1jlqw4jU4AJtZ08hdEFkEUKz4kc8zP/hS1vIPveCqrOUDbF6Zd4TfQBd+uQe60IIZHc37u9QNo5n/PQxmTkaQ6+odgM3MauIAbGZWAz+EMzOrUzvirwOwmTWMH8KZmdXHAdjMrCYOwGZmdWlH/HUANrPmaUsLOGdC9kWS7pL0sKSHJF2cqy4z6x+dJGPv9UCdswW8C/jziFgv6SDgXkm3R8TDGes0sz7Q64G1rJwrYmwDtqXXL0jaCCwAHIDNbFbaEoC7siy9pMXAscDaSd5bLmmdpHUjIzu6cTlm1nTqYOth2QOwpAOBG4BLIuIXE9+PiBURMRwRw0ND83Nfjpm1gPuAS5C0F0XwvSoivp6zLjPrE54JNz0V/4dWARsj4nO56jGz/iKgJfE3axfECcC5wEmSNqTttIz1mVlf8DC0aUXEd+n5LnAza6Kq46qkjwIfosgh/wBwAXAocC0wD7gXODcidkraB7gSeCvwc+DfRcTjM6m3K6MgzMyqVGULWNIC4M+A4Yg4GhgEzgY+A3w+Ig4HngUuTN9yIfBsOv75dN6MOACbWbOoaAGX3UqaA+wnaQ6wP8UchpOANen91cBZ6fWZaZ/0/jLNsK/DAdjMGkUUawqW3YChsbkGaVs+vryI2Ap8FniSIvA+T9Hl8FxE7EqnbaGYSEb6ujl97650/ryZ3IuT8ZhZ43TY3hyJiOE9l6WDKVq1S4DngK8B753N9ZXlAGxmzaLKV9U+GfhZROwAkPR1ilFccyXNSa3chcDWdP5WYBGwJXVZvI7iYVzH3AVhZo1SjAOudBjak8DxkvZPfbnLKHLW3AX8cTrnPOAb6fVNaZ/0/p0RETO5F7eAzaxhqh3fGxFrJa0B1lNkcbwPWAH8I3CtpP+djq1K37IK+KqkTcAzFCMmZqSnAvDYJ1uTbV55TvY6Fl54ddbyn/rKB7KWD7Bz12j2Ovaek/cPvNHRGTV6OjJY7Z/arVF1mIiIy4DLJhx+DFg6ybkvA39SRb09FYDNzMpoekNtjAOwmTVLZ+N7e5oDsJk1Shu6Ksc4AJtZ47Qk/joAm1nzuAVsZlaTlsRfB2AzaxiviGFmVo82rYiRc0mifYG7gX1SPWvSYGczs1no/ZUuysrZAv41cFJE/DItzvldSf8UEfdkrNPM+kBL4m/WJYkC+GXa3Stt+edumlnrtaUFnHWyvKRBSRuA7cDtEbF2knOWjyVK3jGyI+flmFkb5FkRoxZZA3BE7I6IYyhyaS6VdPQk56yIiOGIGJ4/ND/n5ZhZC2RIR1mbruQDjojnKHJrdiXLvJm1mwPwNCTNlzQ3vd4PeA/wSK76zKx/tKULIucoiEOB1ZIGKQL99RHxzYz1mVmf6PWWbVk5R0HcDxybq3wz61MNaNmW5ZlwZtYo8kQMM7P6tCT+OgCbWfMMtCQCOwCbWaNIMNCSxUodgM2scVoSfx2Azax5/BDOJjVnMP/kwqe+8oGs5S+48Jqs5QNsXXVO9jpGR/PmfurGn8FFTqt8Xtmdt/xcpbck/joAm1mziGIoWhs4AJtZ47gP2MysDg1IslOWA7CZNU5L4q8DsJk1i/BEDDOz2rQk/joAm1nzuA/YzKwGTUi0Xlb2AJwSsq8DtkbE6bnrM7P2a0sfcDfWhLsY2NiFesysT6iDrZflXpZ+IfA+YGXOesysv3hRznK+AHwcGM1cj5n1iWIYWvmtVJnSXElrJD0iaaOkt0t6vaTbJT2avh6czpWkL0raJOl+ScfN9F5yrop8OrA9Iu6d5rzlktZJWrdjZEeuyzGztuig9dtBC/hvgFsi4s3AWyi6TS8F7oiII4A70j7AqcARaVsOXD7TW8nZAj4BOEPS48C1wEmS/mHiSRGxIiKGI2J4/tD8jJdjZm1R5bL0kl4HnAisAoiInRHxHHAmsDqdtho4K70+E7gyCvcAcyUdOpP7KB2AJe3TScER8YmIWBgRi4GzgTsjIm8eRTPrCx22gIfG/spO2/IJxS0BdgBflnSfpJWSDgAOiYht6ZyngEPS6wXA5nHfvyUd69i0AVjSUkkPAI+m/bdI+tJMKjMzm60Z9AGPjP2VnbYVE4qcAxwHXB4RxwIv8tvuBgCiSMxceXrjMi3gLwKnAz9PF/Ij4N2dVBIR3/YYYDOrSsV9wFuALRGxNu2voQjIT491LaSv29P7W4FF475/YTrWsTIBeCAinphwbPdMKjMzq0KV44Aj4ilgs6Qj06FlwMPATcB56dh5wDfS65uAD6bREMcDz4/rquhImZlwmyUtBSLNavsI8JOZVGZmNltSlplwHwGukrQ38BhwAUUD9XpJFwJPAO9P594MnAZsAl5K585ImQD8pxTdEIcBTwPfSsfMzGpRdfyNiA3A8CRvLZvk3AAuqqLeaQNwRGynGMVgZtYTen2GW1nTBmBJf88kT/8iYuJQDjOz7IQYbMmicGW6IL417vW+wB/x6jFwZmbd00/pKCPiuvH7kr4KfDfbFZmZTaNvuiAmsYTfzgipVABF/3Y+uX9wua8fYOeuvLmNtqzM3+W/9FPfmv6kWfr+J1/z/KRS3fhZ7x7NW8deg3n/PeQqvRt5dLuhTB/ws/y2D3gAeIYJs0TMzLpF9EkLWMVdvoXfzvIYjW587JuZTaElz+CmbsmnYHtzROxOm4OvmdWu6nzAdSnTlbJB0rHZr8TMrIQizWQ7VsTYYxeEpDkRsQs4FvihpJ9SZAkSReN4xlngzcxmo9dbtmVN1Qf8A4qMQGd06VrMzErp8YZtaVMFYAFExE+7dC1mZtMq8gG3IwJPFYDnS/rYnt6MiM9luB4zs2n1wzjgQeBAZjGWOq0H9wJF/uBdETFZtiEzs460pAE8ZQDeFhGfqqCOd0fESAXlmJkhqS+6INpxh2bWOi2Jv1N2pVQxkT6A2yTdO8lKpABIWj62WunIyI4KqjSztmvLRIw9toAj4pkKyv+DiNgq6Q3A7ZIeiYi7J9SzAlgBcNxbhz3Tzsym1KZREFkfJkbE1vR1O3AjsDRnfWbWH6TyWy/LFoAlHSDpoLHXwCnAg7nqM7M+0UH3Q2O7ICpwCHBjmos9B7g6Im7JWJ+Z9Qm1ZIxAtgAcEY9RpLI0M6tM0Qdc91VUI2cL2MwsCwdgM7Oa9HqaybIcgM2sUdwFYWZWF8FgSyKwA7CZNYpbwGZmNWpJF7ADsJk1jRjwOOA8cq+73IZPzn32Gsxa/u7R/Ck57vlkFbmepnbY8uuylr9l5dlZyweYM9jsX9gcVy/a8e8YejAAm5lNqQFTjMtyADazxmlLNjQHYDNrFHdBmJnVyC1gM7OatCT+tmZ1ZzPrE6IIXGW30uVKg5Luk/TNtL9E0lpJmyRdJ2nvdHyftL8pvb94pvfiAGxmzaIiGU/ZrQMXAxvH7X8G+HxEHA48C1yYjl8IPJuOfz6dNyMOwGbWOOpgK1WetBB4H7Ay7Qs4CViTTlkNnJVen5n2Se8v0wzTs2UNwJLmSloj6RFJGyW9PWd9ZtZ+Y4tylt1K+gLwcWA07c8DnouIXWl/C7AgvV4AbAZI7z+fzu9Y7hbw3wC3RMSbKVbH2DjN+WZm0+qwBTwkad24bfmrypJOB7ZHxL1du4Ek2ygISa8DTgTOB4iIncDOXPWZWf/o8A/+kYgYnuL9E4AzJJ0G7Av8DkXjca6kOamVuxDYms7fCiwCtkiaA7wO+Hlnd1DI2QJeAuwAvpyeLK5MqyO/iqTlY59MIyM7Ml6OmbVD+QdwZbpmI+ITEbEwIhYDZwN3RsS/B+4C/jiddh7wjfT6prRPev/OiJllsckZgOcAxwGXR8SxwIvApRNPiogVETEcEcNDQ/MzXo6ZtUGuYWiT+AvgY5I2UfTxrkrHVwHz0vGPMUlcKyvnRIwtwJaIWJv21zCLCzUzG5NrTbiI+Dbw7fT6MWDpJOe8DPxJFfVlawFHxFPAZklHpkPLgIdz1Wdm/aPqYWh1yT0V+SPAVWkGyWPABZnrM7O2k1dFLiUiNgBTPX00M+vIWB9wGzgZj5k1jlvAZmY1aUf4dQA2s4YRMOgWsJlZPVoSfx2AzaxphFrSCeEAbGaN4xZwBgIGMq83PTo6oynb5cuf2ZTwjuRekrsbS3534yn2lpVnZy3/DedembV8gO1f/WDW8nfuGp3+pFnIUXoxDK0dEbinArCZ2bTkFrCZWW0cgM3MauKHcGZmNSiWJKr7KqrhAGxmjeMWsJlZTdwHbGZWE7eAzcxq0KY+4GxpNSUdKWnDuO0Xki7JVZ+Z9Qt19F8vy9YCjogfA8cASBqkWMr5xlz1mVmf8ESMji0DfhoRT3SpPjNrsZbE364F4LOBayZ7Q9JyYDnAosMO69LlmFlTFX3A7QjB2ZdWSgtyngF8bbL3I2JFRAxHxPD8ofm5L8fMWsCrIpd3KrA+Ip7uQl1m1g96PbKW1I0AfA576H4wM5uJXh/dUFbWLghJBwDvAb6esx4z6y9S+a2XZW0BR8SLwLycdZhZ/+nxuFqaZ8KZWfO0JAI7AJtZoxSjG9oRgR2AzaxZGtC3W5YDsJk1jgOwmVktej/JTlkOwGbWOG4Bm5nVoAlTjMvqqQAcQETUfRmz0o0kITt3jWYtf+852VOE8PIru7PXsU/m+9i2+tys5QMc9z9uy1r+ur96T9bys/1raEkEzv8vzcysYlUmZJe0SNJdkh6W9JCki9Px10u6XdKj6evB6bgkfVHSJkn3SzpupvfhAGxmjVPxVORdwJ9HxFHA8cBFko4CLgXuiIgjgDvSPhQJxo5I23Lg8pnehwOwmTVOlekoI2JbRKxPr18ANgILgDOB1em01cBZ6fWZwJVRuAeYK+nQmdyHA7CZNUsn0bfDvmJJi4FjgbXAIRGxLb31FHBIer0A2Dzu27akYx3rqYdwZmZldDgOeEjSunH7KyJixWvKlA4EbgAuiYhfaFz/RUSEpMpHCDgAm1mjiI7HAY9ExPCUZUp7UQTfqyJiLH3u05IOjYhtqYthezq+FVg07tsXpmMdcxeEmTVOlT0QKpq6q4CNEfG5cW/dBJyXXp8HfGPc8Q+m0RDHA8+P66roiFvAZtY81Y4DPgE4F3hA0oZ07C+BTwPXS7oQeAJ4f3rvZuA0YBPwEnDBTCvOGoAlfRT4EMUciweACyLi5Zx1mln7VZkLIiK+y55D+rJJzg/goirqztYFIWkB8GfAcEQcDQxSLE9vZjYrXpKofPn7SXoF2B/4l8z1mVkf6PG4Wlq2FnBEbAU+CzwJbKPoqH7NxHZJyyWtk7RuZGRHrssxszbJNA6423J2QRxMMWNkCfBG4ABJH5h4XkSsiIjhiBgeGpqf63LMrCXGliSqKhdEnXIOQzsZ+FlE7IiIVyiWpn9HxvrMrB900P/b633AOQPwk8DxkvZP4+yWUcyxNjOblZb0QOR7CBcRayWtAdZTZBu6D3jN9D8zs471emQtKesoiIi4DLgsZx1m1m96v2+3LM+EM7PG6fW+3bIcgM2sUZrQt1uWA7CZNY5a0gR2ADazxmlJ/HUANrPmaUn8dQA2s4ZpwASLsnouAEfli368Wu4fXDf6pvaakzeP/q93jWYtH2CfzPcAMDCQ92fxShf+P63/1ClZy19y0Q1Zyx958tlMJbcjAvdcADYzm8oMliTqWQ7AZtY4LYm/DsBm1jxuAZuZ1cRTkc3M6tKO+OsAbGbN05L46wBsZs3ShETrZTkAm1njtKUPOOtoeEkXS3pQ0kOSLslZl5n1kZYsiZFzUc6jgf8ILAXeApwu6fBc9ZlZ/2hJ/M3aAv59YG1EvBQRu4DvAP82Y31m1ie8KOf0HgTeKWmepP2B04BFGeszs77QyaL0vR2Bcy7KuVHSZ4DbgBeBDcDuiedJWg4sB1h02GG5LsfMWqJNuSCyPoSLiFUR8daIOBF4FvjJJOesiIjhiBgeGpqf83LMzHpK1mFokt4QEdslHUbR/3t8zvrMrD+0pQWcexzwDZLmAa8AF0XEc5nrM7M+0Ot9u2VlDcAR8c6c5ZtZH2rA6IayPBPOzBqlCeN7y3IANrPmaUkEdgA2s8YZaEkfhAOwmTVOO8Jv5nHAZmZZVJwMQtJ7Jf1Y0iZJl+a45Mk4AJtZ41Q5FVnSIPB3wKnAUcA5ko7KfAuAA7CZNczYVOQKk/EsBTZFxGMRsRO4Fjgz4y38Rk/1Ad+3/t6RA/YZeKKDbxkCRnJdTxfKb0sdvof+qaPT8t9U9QWsX3/vrfvtpaEOvmVfSevG7a+IiBXj9hcAm8ftbwHeNptrLKunAnBEdJQMQtK6iBjOdT25y29LHb6H/qmjG/cwnYh4b531V8ldEGbW77by6lS5C9Ox7ByAzazf/RA4QtISSXsDZwM3daPinuqCmIEV05/S0+W3pQ7fQ//U0Y176KqI2CXpw8CtwCBwRUQ81I26FRHdqMfMzCZwF4SZWU0cgM3MatLIAJx72qCkKyRtl/Rg1WWPq2ORpLskPSzpIUkXV1z+vpJ+IOlHqfz/WWX5E+oalHSfpG9mKPtxSQ9I2jBhLGeVdcyVtEbSI5I2Snp7xeUfma5/bPuFpEsqruOj6ef8oKRrJO1bZfmpjotT+Q9Vff19KyIatVF0kv8U+FfA3sCPgKMqruNE4DjgwYz3cShwXHp9EMV6eZXdB8WEoQPT672AtcDxme7lY8DVwDczlP04MJT5d2o18KH0em9gbsa6BoGngDdVWOYC4GfAfmn/euD8iq/7aIqVzveneHj/LeDwnD+Xftia2ALOPm0wIu4GnqmyzEnq2BYR69PrF4CNFP+Qqio/IuKXaXevtFX+xFXSQuB9wMqqy+4GSa+j+MBdBRAROyPv0lnLgJ9GRCczPsuYA+wnaQ5FkPyXisv/fWBtRLwUEbuA71Cs82iz0MQAPNm0wcoCVx0kLQaOpWilVlnuoKQNwHbg9oiotPzkC8DHgdEMZUPxoXGbpHslLc9Q/hJgB/Dl1I2yUtIBGeoZczZwTZUFRsRW4LPAk8A24PmIuK3KOihav++UNE/S/sBpvHrygs1AEwNwq0g6ELgBuCQiflFl2RGxOyKOoZjZs1TS0VWWL+l0YHtE3FtluRP8QUQcR5Gp6iJJJ1Zc/hyK7qbLI+JY4EUgSzrCNMj/DOBrFZd7MMVfgUuANwIHSPpAlXVExEbgM8BtwC3ABmB3lXX0oyYG4NqmDVZN0l4UwfeqiPh6rnrSn9R3AVXPoT8BOEPS4xRdQSdJ+ocqK0itOyJiO3AjRRdUlbYAW8b9dbCGIiDncCqwPiKerrjck4GfRcSOiHgF+DrwjorrICJWRcRbI+JE4FmK5xY2C00MwLVNG6ySJFH0O26MiM9lKH++pLnp9X7Ae4BHqqwjIj4REQsjYjHFz+HOiKis5SXpAEkHjb0GTqH4U7gyEfEUsFnSkenQMuDhKusY5xwq7n5IngSOl7R/+r1aRvFMoVKS3pC+HkbR/3t11XX0m8ZNRY4uTBuUdA3wLmBI0hbgsohYVWUdFK3Hc4EHUj8twF9GxM0VlX8osDolmx4Aro+IyoeJZXYIcGMRU5gDXB0Rt2So5yPAVekD/THggqorSB8g7wH+U9VlR8RaSWuA9cAu4D7yTBm+QdI84BXgoswPK/uCpyKbmdWkiV0QZmat4ABsZlYTB2Azs5o4AJuZ1cQB2MysJg7AtkeSdqfsXQ9K+lqagjrTst41li1N0hlTZbFL2cn+ywzq+CtJ/3Wm12jWbQ7ANpVfRcQxEXE0sBP4z+PfVKHj36GIuCkiPj3FKXOBjgOwWdM4AFtZ/wwcLmlxysV8JcWstEWSTpH0fUnrU0v5QPhN3uZHJK1nXOYsSedL+tv0+hBJN6a8xT+S9A7g08Dvptb3X6fz/pukH0q6f3xuY0n/XdJPJH0XOBKzBmncTDjrvpTi8FSKJCwARwDnRcQ9koaATwInR8SLkv4C+Jik/wv8PXASsAm4bg/FfxH4TkT8UZq1dyBFMpyjUyIhJJ2S6lxKkef4ppSU50WKKdDHUPwurwdyJgYyq5QDsE1lv3HTpP+ZInfFG4EnIuKedPx44Cjge2nK8N7A94E3UySIeRQgJemZLJ3kScAHocjeBjyfsnuNd0ra7kv7B1IE5IOAGyPipVRH43KCWH9zALap/GqsFTomBdkXxx+iyDV8zoTzXvV9syTg/0TE/5tQh5fFsUZzH7DN1j3ACZIOh99kMPs9isxriyX9bjrvnD18/x3An6bvHUwrVLxA0bodcyvwH8b1LS9ImbnuBs6StF/KmvaHFd+bWVYOwDYrEbEDOB+4RtL9pO6HiHiZosvhH9NDuO17KOJi4N2SHqDovz0qIn5O0aXxoKS/Tqs7XA18P523BjgoLel0HcW6gP9EkarUrDGcDc3MrCZuAZuZ1cQB2MysJg7AZmY1cQA2M6uJA7CZWU0cgM3MauIAbGZWk/8P9O0mofWW/FcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd216ee29b0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print_confusion_matrix()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are now done using TensorFlow, so we close the session to release its resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This has been commented out in case you want to modify and experiment\n",
    "# with the Notebook without having to restart it.\n",
    "# session.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises\n",
    "\n",
    "These are a few suggestions for exercises that may help improve your skills with TensorFlow. It is important to get hands-on experience with TensorFlow in order to learn how to use it properly.\n",
    "\n",
    "You may want to backup this Notebook before making any changes.\n",
    "\n",
    "* Change the learning-rate for the optimizer.\n",
    "* Change the optimizer to e.g. `AdagradOptimizer` or `AdamOptimizer`.\n",
    "* Change the batch-size to e.g. 1 or 1000.\n",
    "* How do these changes affect the performance?\n",
    "* Do you think these changes will have the same effect (if any) on other classification problems and mathematical models?\n",
    "* Do you get the exact same results if you run the Notebook multiple times without changing any parameters? Why or why not?\n",
    "* Change the function `plot_example_errors()` so it also prints the `logits` and `y_pred` values for the mis-classified examples.\n",
    "* Use `sparse_softmax_cross_entropy_with_logits` instead of `softmax_cross_entropy_with_logits`. This may require several changes to multiple places in the source-code. Discuss the advantages and disadvantages of using the two methods.\n",
    "* Remake the program yourself without looking too much at this source-code.\n",
    "* Explain to a friend how the program works."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}