exercises/exercises_35_two_sample_tests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "69778aa3-a805-4f35-b97f-91c24dae34c0",
   "metadata": {},
   "source": [
    "# Exercises for Section 3.5 Two-sample tests\n",
    "\n",
    "This notebook contains the solutions to the exercises\n",
    "from [Section 3.5 Two-sample tests]()\n",
    "in the **No Bullshit Guide to Statistics**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68eebd12-a082-4939-88db-7953dbaaea02",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Notebooks setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0ac52c9b-6788-4556-82f9-79be3dd2632d",
   "metadata": {
    "id": "VFzPCGo6I1NJ"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ecc91494-20f9-43dd-92f7-971ec22dcaa4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "13f1e218-f94c-404f-b784-bd98e4e62b70",
   "metadata": {},
   "source": [
    "### Estimator functions defined in Section 3.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a9bc7744-c676-4561-be77-b091aeba96cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "def mean(sample):\n",
    "    return sum(sample) / len(sample)\n",
    "\n",
    "def var(sample):\n",
    "    xbar = mean(sample)\n",
    "    sumsqdevs = sum([(xi-xbar)**2 for xi in sample])\n",
    "    return sumsqdevs / (len(sample)-1)\n",
    "\n",
    "def std(sample):\n",
    "    s2 = var(sample)\n",
    "    return np.sqrt(s2)\n",
    "\n",
    "def dmeans(xsample, ysample):\n",
    "    dhat = mean(xsample) - mean(ysample)\n",
    "    return dhat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "634071f3-19b9-4b3b-871d-f80b62b1b61c",
   "metadata": {},
   "source": [
    "## Exercises"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "496c8b13-90b8-4249-80d8-0e9716d3ad73",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "07aaa32d-034e-479e-874a-f4173bbf60ce",
   "metadata": {},
   "source": [
    "### E1. Permutation test for the sleep score sin the doctors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9b7a6196-e3b6-4b0e-b1c7-fe92a5558010",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6.992885375494076"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "doctors = pd.read_csv(\"../datasets/doctors.csv\")\n",
    "scoresU = doctors[doctors[\"loc\"]==\"urb\"][\"score\"]\n",
    "scoresR = doctors[doctors[\"loc\"]==\"rur\"][\"score\"]\n",
    "\n",
    "# observed difference between means\n",
    "dhat = dmeans(scoresR, scoresU)\n",
    "dhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "62b03059-42e6-4111-aa57-fefbca924fcf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0496"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from ministats import permutation_test_dmeans\n",
    "pvalue = permutation_test_dmeans(scoresR, scoresU)\n",
    "pvalue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ad6f1031-35c9-4672-a89b-3a4f61d51d1e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.05019498050194981"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# ALT. use the ttest_ind with permutations argument\n",
    "from scipy.stats import ttest_ind\n",
    "ttest_ind(scoresR, scoresU, permutations=10000).pvalue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d72ecdcf-a5ab-400a-9fe6-77f83af6c0bb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "34da1888-7673-4631-b46d-2c6776201d68",
   "metadata": {},
   "source": [
    "### E2. Sleep scores using t-test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bf2afd0-900a-4102-a185-4e228d2836d4",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e56ed56-4807-487d-9a74-8307fc2e1a91",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b905c7f7-d569-491a-8c88-dfd594cb04b5",
   "metadata": {},
   "source": [
    "### E3. Example 6T with pooled variance\n",
    "\n",
    "Redo Example 6T but this time run the two-sample $t$-test with pooled variance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "1e0fc5ae-ae1a-4fdc-98af-5f25d8563e88",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.10917234443214315"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "students = pd.read_csv(\"../datasets/students.csv\")\n",
    "scoresD = students[students[\"curriculum\"]==\"debate\"][\"score\"]\n",
    "scoresL = students[students[\"curriculum\"]==\"lecture\"][\"score\"]\n",
    "res = ttest_ind(scoresD, scoresL, equal_var=True)\n",
    "res.pvalue"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ca157f3a-68ca-4338-9dbd-221f3c9f047b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6ca25fb2-c5ef-45d3-8e91-73eb60d3d45f",
   "metadata": {},
   "outputs": [],
   "source": [
    "### "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1adaf756-e857-44fd-b80d-2ad6a5a4bacc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}