Chapter 3 - PMF, PDF and CDF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#-Probability-Mass-Function\" data-toc-modified-id=\"-Probability-Mass-Function-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Probability Mass Function</font></a></span><ul class=\"toc-item\"><li><span><a href=\"#-Cumulative-Distribution-Function\" data-toc-modified-id=\"-Cumulative-Distribution-Function-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Cumulative Distribution Function</font></a></span></li></ul></li><li><span><a href=\"#-Probability-Density-Function\" data-toc-modified-id=\"-Probability-Density-Function-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Probability Density Function</font></a></span></li><li><span><a href=\"#Independent-Random-Variables\" data-toc-modified-id=\"Independent-Random-Variables-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\">Independent Random Variables</font></a></span></li><li><span><a href=\"#-Expected-Value-and-Variances\" data-toc-modified-id=\"-Expected-Value-and-Variances-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span><font face=\"gotham\" color=\"purple\"> Expected Value and Variances</font></a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Probability Mass Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If $X$ is a discrete variable with a finite range $R_X = \\{x_1,x_2,x_3...\\}$, the **probability mass function** (PMF) of $X$ is \n",
    "\n",
    "$$\n",
    "P_X(x_k)=P(X=x_k),\\ \\text{for } k=1,2,3\\ldots\n",
    "$$\n",
    "\n",
    "which is a function which maps the possible values to the corresponding probabilities. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For instance, here is a discrete uniform distribution of a fair die"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa0AAAEeCAYAAAA5CErsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAApK0lEQVR4nO3df1xUdb4/8BcwiPFrRgll0BVhHDWMdjdCDRXQBEm0VveW7FBmtrdd9z6Wx8PNH2W7Fmq77E3qyrfr3m4ZqxQCFmVCBVg4F0WLMsoVxRENVH4I5gxMijLMfP/wwWzjoDMMuMPn+Ho+Hvwxn/M553zeSbw453w+Bw+9Xm8BERGRADzdPQAiIiJnMbSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLaIBeP311zF9+nQolUooFAr85S9/cfeQBsU777wDhUKBd955x91DAQD85S9/gUKhQGVlpbuHQm7G0KJBo1AobL5GjhyJsLAwJCcnIycnBz09PTb9U1JSrH23b99+w+Nu3rzZ2m/58uU223p/uN7sS6/X34py8d5772Ht2rXo7u7G008/jbVr12LmzJlO79/d3Q21Wg2FQoFHHnnEpTG4s/7+un5cwcHBiIiIwMyZM7FixQoUFxeju7vb3cOkIU7m7gGQ9KxduxYA0NPTg9OnT6O4uBiHDh3Cvn37+gwnmUyG7du344knnrDbZrFYkJubC5lMBpPJdMNz3n333UhJSelz2/Dhw12s5OZKS0sBAP/zP/+DmJiYfu9fXFyMtrY2eHh44NNPP0VjYyPGjRvn0lgGu/4FCxYgJiYGo0ePdmk8N/Pj74+Ojg7odDp88MEH2LlzJyZNmoTXX38dP/vZz2z2efrpp/HLX/4SY8eOHfTxkFgYWjTonnvuOZvPR48exdy5c7F7925UVVUhNjbWZvu8efNQUlKCI0eOICoqymbbvn370NDQgAULFqC4uPiG54yKirI7763W3NwMABg1apRL++fk5AAAnnnmGWzevBk7duzAH//4R5eONdj1y+VyyOXyQTvej/U1Tr1ej40bN2Lbtm1YtGgR9u7dC5VKZd0eFBSEoKCgWzIeEgtvD9ItN2XKFMyYMQMA8NVXX9ltX7p0KTw8PLBjxw67bdu3b8fw4cPx6KOP3vJxdnR0YOPGjdYrjHHjxmHBggXYs2ePTb/rn6/89Kc/td7yctapU6dQWVmJ6dOn4w9/+AMCAgLw9ttv3/RqciBqamqwZs0axMbGIiwsDKNHj8a9996LdevW4eLFi3b9b/RMKyoqCgqFAl1dXdi0aRN+/vOfIzg4GM8+++yAxqdQKJCVlYVHH30UFy9exIsvvmiz/WbPtE6dOoXf//73uPvuuzFq1CioVCqkpaWhpqZmQGOioYlXWuR24eHhmDVrFgoLC7FhwwbccccdAID29nZ89NFH+MUvfoERI0bc0jHo9XokJyfj+PHjuOeee/Db3/4WBoMBH3zwAR5//HGsWbMG69atAwDrc6u8vDycOXMGv/3tb/t9VfL3v/8dFosFGo0Gvr6++MUvfoHc3Fx8/PHHWLhw4aDXt337dhQXF2PGjBmYPXs2enp6UFNTg61bt6K8vByfffYZAgICnD7e0qVL8e233+KBBx7AiBEjMH78+EEZ57PPPovCwkJ8/PHH6OzsdDgmrVaLtLQ0dHV1Yd68eVCpVGhubsaePXuwd+9e5OXl4YEHHhiUsdHQwNCiW+7YsWM4cOAAAODee+/ts88TTzyBp556Crt370ZqaioAYOfOnbh69SqWLl0Ki+Xm73U+cuRInzP35s6d69TzphdffBHHjx9HWloaXnvtNXh4eAAAVq9ejTlz5uDll1/GvHnzEB0djVmzZmHWrFnYv38/zpw5gxUrViAsLMzhOXpdvXoVeXl58PX1xaJFiwAAaWlpyM3Nxfbt210KLUf1r1y5Eps3b4aXl5fN9pycHKxcuRJvvvkmVq5c6fT5zp49iwMHDgz6LbuIiAiMGTMG586dQ01NDWbNmnXDvgaDAU8++SS8vb2xd+9eTJ482bqtrq4ODzzwAP7jP/4D33zzDXx8fAZ1nOQ+DC0adL0/PHt6etDQ0IA9e/agq6sLCxYssN4mvN6CBQswcuRIbN++3RpaO3bswIQJEzBz5kyHU53/8Y9/4B//+Iddu1wudxha3d3dKCwshK+vLzIyMqyBBQBjxozBH/7wB6xduxY7duxAdHT0TY/ljOLiYrS3tyM1NdV6JTF9+nRMmDABn332GRoaGvoVgoDj+m80wWPZsmV44YUX8Nlnn/UrtNatW3fLnjGFhITg3LlzaG9vv2m//Px8fP/998jMzLQJLACYNGkSli5diq1bt2Lfvn2YN2/eLRkr/esxtGjQ/fWvfwUAeHh4ICAgAD/96U/xyCOPYNmyZTfcx8fHB6mpqdi6dSt0Oh1aW1uh0+mwYcMGp875q1/9Cn/7299cGu+JEydw6dIl3HfffbjzzjvttickJAAAvvnmG5eOf73eCRhpaWk27RqNBhs2bEBubm6/J2Q4qr+7uxs5OTkoKirCsWPH0NnZCbPZbN3eO6nEWffdd1+/+rvix7889OXzzz8HcG2iT19XmSdPngRw7d+XoSUdDC0adK6uC3riiSewdetW7NixA62trfD29savfvWrwR1cHzo6OgDceBZg77Tv3n4DUV9fj/3792PcuHF2a7pSU1OxadMmvP3223j22Wchkw3e/55PPvkkiouLMX78eKSkpGD06NEYNmwYAOBvf/sbrly50q/j3Yqp8L1aWloAwOGV3Pfffw8AyM3NvWm/H374YXAGRkMCQ4uGjEmTJuH+++9HXl4efvjhB6SkpCA4OPiWnzcwMBAAcP78+T63t7a22vQbiN4JGI2NjTecXNLS0oKPPvoIDz300IDPBwBff/01iouLER8fj3fffRfe3t7WbWazGdnZ2f0+pqOrIFedOnUK586dg0wms1urdb3ef499+/Y57EvSwdCiIWXp0qVYsWIFAPS52PhWmDhxInx9fVFbW4sLFy7Y/Yav1WoBYMA/GK9evYqdO3fCw8MDGo0Gnp72K04uXLiAjz76CNu3bx+00Dp16hQAYP78+TaBBVxbgnD58uVBOc9gyMzMBHBtrI5mDsbExODDDz/EwYMHGVq3EYYWDSmLFi1CQEAAPDw8rM+SbjVvb28sWbIEOTk5yMjIwJYtW6xXEs3NzXj11Vfh4eGBxx57bEDn2bNnD9rb2xEXF4f//u//7rNPd3c3pkyZ4vKEjL70TsLYv38/fvOb31jb29rasGrVqgEffzD0Li4uLCyEQqGwW6fVl8ceewxZWVl4+eWXER0djalTp9pst1gsOHToEKKjo623Qkl8DC0aUoYPH44FCxb8y8/7wgsv4ODBg9ixYwe+/fZbJCQkWNdpXbx4EWvWrBnw5IO///3vAHDTCSne3t7QaDT4r//6L+zYsQN/+tOfBnRO4Noyg+nTp2PPnj1ISkrC9OnTcf78eezduxdqtRpKpXLA5+iP3kkTZrPZ+hqngwcP4vLly9bXOEVERDg8zogRI7Bjxw489thjSEpKQlxcHCZPngxvb2+cO3cOX375Jc6ePYvvvvuOoSUhDC0iXHsjQ2lpKbZs2YIPP/wQW7duhY+PD+655x785je/GfCtut4JGEFBQQ5DeenSpdiyZQvefvttPPfccwOekOHl5YWdO3di06ZNKCsrw+uvvw6lUomlS5di1apVmDZt2oCO31+9s0u9vb3h7++P0NBQPPzww0hJSUFycrLdLcybiYuLw4EDB/Daa6/h008/xRdffAGZTIbRo0cjJiYGL7zwwqA8i6Shw0Ov19981SYREdEQwXcPEhGRMJy+71BRUYHS0lIYDAaEhoZiyZIlmDhxYp99m5qakJeXh6amJly+fBkKhQIxMTF46KGHbG511NXVobCwEE1NTVAoFJg3b96/7OE7ERGJx6nQqq6uRkFBATQaDdRqNSoqKpCdnY2MjIw+FwDKZDLcf//9GDduHHx9fXHmzBns2LEDZrMZ//Zv/wbg2syl7OxszJgxA7/+9a+h0+mQl5eHgICAQXlVDhERSY9ToVVeXo7Y2FjExcUBuPa6maNHj0Kr1WLx4sV2/UeNGmXzdoGgoCDU1dVBp9NZ27RaLRQKBTQaDQBAqVTi9OnTKCsrY2gREVGfHD7TMplMaGhoQGRkpE17ZGQk6uvrnTrJ+fPncfToUZvbiadOnbI75pQpU9DQ0HDL/qYQERGJzeGVltFohNlstps2GhgYiGPHjt1038zMTGsIzZo1y/pnGIBrf1bgrrvusjtmT08PjEZjv/6gHhER3R5cXgBisVgcvn/s6aefRldXF86cOYN3330Xn3zyCebPn2/dfv3+vX8z6Va914yIiMTm8Pagv78/PD097d5w3dnZ6XDR3siRIxEaGopp06bhl7/8Jfbs2YOenh4A1/7Oj8FgsDuml5cX/Pz8+luHsH78nE8qpFgTwLpEIsWaAOnW1R8OQ0smkyEsLAy1tbU27bW1tVCpVE6fyGw2W7+Aa3+h9Prbi7W1tQgLCxvUP8lARETS4dTi4sTERFRVVaGyshLNzc3Iz8+HwWBAfHw8AKCoqAhZWVnW/gcPHsSXX36J5uZmtLW1obq6Gu+//z6io6Otr2iJj4/HxYsXkZ+fj+bmZlRWVqKqqgpJSUm3oEwiIpICpy5pYmJiYDQaUVJSYl1cnJ6ebl2jpdfr0dbWZu3v6emJjz/+2Pp3iEaOHImEhAQkJiZa+wQHByM9PR2FhYXQarWQy+VITU3ldHciIrohvnvQzXQ6HdRqtbuHMaikWBPAukQixZoA6dbVH3z3IBERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDJmzHSsqKlBaWgqDwYDQ0FAsWbIEEydO7LNvXV0dysvL8d133+Hy5csIDg7G3LlzMXPmTJs+mzdvttt3w4YNUCqVLpRCRERS51RoVVdXo6CgABqNBmq1GhUVFcjOzkZGRgaCgoLs+p88eRJjxoxBcnIy5HI5jh49itzcXHh7e2PatGk2fTMyMuDn52f9HBAQMMCSiIhIqpwKrfLycsTGxiIuLg4AoNFocPToUWi1WixevNiuf0pKis3nhIQEHD9+HIcPH7YLrYCAAAYVERE5xWFomUwmNDQ0ICkpyaY9MjIS9fX1Tp+oq6sLI0aMsGt/6aWXYDKZoFQqkZKSgsmTJzt9TCIiur04DC2j0Qiz2YzAwECb9sDAQBw7dsypk3zzzTc4fvw41q5da22Ty+VIS0tDeHg4TCYTDh06hFdeeQWrVq264bMyANDpdE6dUySsSRysSxxSrAmQXl1qtbpf/Z2eiHE9i8UCDw8Ph/1OnjyJN998E6mpqQgPD7e2h4SEICQkxPpZpVKhvb0dZWVlNw2t/hY41Ol0OtYkCNYlDinWBEi3rv5wOOXd398fnp6e6OjosGnv7Oy0u/q6nk6nw5YtW/Dwww8jISHB4WDCw8PR2trqsB8REd2eHIaWTCZDWFgYamtrbdpra2uhUqluuN+JEyewZcsWLFy4EHPnznVqMGfOnIFcLneqLxER3X6cuj2YmJiIbdu2ITw8HBMmTIBWq4XBYEB8fDwAoKioCKdPn8YzzzwD4NoarOzsbCQkJGDatGkwGAwAAE9PT+tMwb179yIoKAihoaEwmUz4/PPPUVNTgxUrVtyKOomISAKcCq2YmBgYjUaUlJRYFxenp6db12jp9Xq0tbVZ+x84cABXr15FWVkZysrKrO1BQUHIzMwEcG1W4q5du6DX6+Ht7W09ZlRU1GDWR0REEuKh1+st7h7E7UyKD1alWBPAukQixZoA6dbVH3z3IBERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMGTOdqyoqEBpaSkMBgNCQ0OxZMkSTJw4sc++dXV1KC8vx3fffYfLly8jODgYc+fOxcyZM+36FRYWoqmpCQqFAvPmzUNCQsKACiIiIulyKrSqq6tRUFAAjUYDtVqNiooKZGdnIyMjA0FBQXb9T548iTFjxiA5ORlyuRxHjx5Fbm4uvL29MW3aNABAW1sbsrOzMWPGDPz617+GTqdDXl4eAgICEB0dPbhVEhGRJDgVWuXl5YiNjUVcXBwAQKPR4OjRo9BqtVi8eLFd/5SUFJvPCQkJOH78OA4fPmwNLa1WC4VCAY1GAwBQKpU4ffo0ysrKGFpERNQnh8+0TCYTGhoaEBkZadMeGRmJ+vp6p0/U1dUFX19f6+dTp07ZHXPKlCloaGiAyWRy+rhERHT7cBhaRqMRZrMZgYGBNu2BgYEwGAxOneSbb77B8ePHrVdqAGAwGPo8Zk9PD4xGo1PHJSKi24vTEzGuZ7FY4OHh4bDfyZMn8eabbyI1NRXh4eE2267f32Kx9Nn+YzqdzoXRDm2sSRysSxxSrAmQXl1qtbpf/R2Glr+/Pzw9PdHR0WHT3tnZaXeldD2dTofs7Gw8/PDDdrMC5XK53ZVaZ2cnvLy84Ofnd8Nj9rfAoU6n07EmQbAucUixJkC6dfWHw9uDMpkMYWFhqK2ttWmvra2FSqW64X4nTpzAli1bsHDhQsydO9due0REBI4dO2Z3zLCwMMhkLl8AEhGRhDm1uDgxMRFVVVWorKxEc3Mz8vPzYTAYEB8fDwAoKipCVlaWtX9dXR22bNmC+Ph4TJs2DQaDAQaDAZ2dndY+8fHxuHjxIvLz89Hc3IzKykpUVVUhKSlpkEskIiKpcOqSJiYmBkajESUlJdbFxenp6dY1Wnq9Hm1tbdb+Bw4cwNWrV1FWVoaysjJre1BQEDIzMwEAwcHBSE9PR2FhIbRaLeRyOVJTUzndnYiIbshDr9db3D2I25kU71FLsSaAdYlEijUB0q2rP/juQSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkPLTXbV/4CowhZM3X8HogpbsKv+B3cPacCkWBPAukQixZoA6dblCg+9Xm9x9yBuN7vqf0D6AQMu9/zzP/0dXh7IniHHIyo/N47MdVKsCWBdIpFiTYB063IVQ8sNogpbcOaHHrv2n/h54cijIW4Y0cBJsSaAdYlEijUB0q3LVbw96AZn+/gGvFm7CKRYE8C6RCLFmgDp1uUqhpYbjPXz6le7CKRYE8C6RCLFmgDp1uUqhpYbrI8OsGu7w8ujz3ZRSLEmgHWJRIo1AdKty1UMLTfo6+Gp6A9VpVgTwLpEIsWaAOnW5SqG1hAhxW9AKdYEsC6RSLEmQLp1OYOhRUREwmBoERGRMBhaREQkDJmzHSsqKlBaWgqDwYDQ0FAsWbIEEydO7LNvd3c3cnNz0djYiJaWFqhUKqxevdqmT11dHTZv3my374YNG6BUKvtZBhER3Q6cCq3q6moUFBRAo9FArVajoqIC2dnZyMjIQFBQkF1/s9kMb29vzJkzB0eOHMGlS5dueOyMjAz4+f3zoWJAwO05jZOIiBxz6vZgeXk5YmNjERcXB6VSCY1GA7lcDq1W22d/Hx8fPP7444iLi8OIESNueuyAgADI5XLrl6cn71gSEVHfHF5pmUwmNDQ0ICkpyaY9MjIS9fX1Ax7ASy+9BJPJBKVSiZSUFEyePHnAxyQiImlyeFljNBphNpsRGBho0x4YGAiDweDyieVyOdLS0rBixQqsWLECISEheOWVV3DixAmXj0lERNLm9ESM61ksFnh4eLh84pCQEISE/PMNxSqVCu3t7SgrK7vhBA8A0Ol0Lp9zaPG1+SSNuqRYE8C6RCLFmgDp1gWo1ep+9XcYWv7+/vD09ERHR4dNe2dnp93V10CFh4ejurr6pn36W+CQtf+czUdJ1CXFmgDWJRIp1gRIty4XOLw9KJPJEBYWhtraWpv22tpaqFSqQR3MmTNnIJfLB/WYREQkHU7dHkxMTMS2bdsQHh6OCRMmQKvVwmAwID4+HgBQVFSE06dP45lnnrHu09TUBJPJBKPRiCtXrqCxsREAMG7cOADA3r17ERQUhNDQUJhMJnz++eeoqanBihUrBrtGIiKSCKdCKyYmBkajESUlJdbFxenp6dY1Wnq9Hm1tbTb7ZGdn48KFC9bPGzduBAC88cYbAK7NSty1axf0ej28vb2tx4yKihqUwoiISHqcnogxe/ZszJ49u89ty5cvt2vLzMy86fGSk5ORnJzs7OmJiIj47kEiIhIHQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgyZztWVFSgtLQUBoMBoaGhWLJkCSZOnNhn3+7ubuTm5qKxsREtLS1QqVRYvXq1Xb+6ujoUFhaiqakJCoUC8+bNQ0JCgsvFEBGRtDl1pVVdXY2CggLMnz8f69evh0qlQnZ2Ni5cuNBnf7PZDG9vb8yZMwdRUVF99mlra0N2djZUKhXWr1+PBx98EPn5+fjqq69cr4aIiCTNqdAqLy9HbGws4uLioFQqodFoIJfLodVq++zv4+ODxx9/HHFxcRgxYkSffbRaLRQKBTQaDZRKJeLi4nD//fejrKzM9WqIiEjSHIaWyWRCQ0MDIiMjbdojIyNRX1/v8olPnTpld8wpU6agoaEBJpPJ5eMSEZF0OQwto9EIs9mMwMBAm/bAwEAYDAaXT2wwGPo8Zk9PD4xGo8vHJSIi6XJ6Isb1LBYLPDw8BnTy6/e3WCx9tv+YTqcb0DmHDl+bT9KoS4o1AaxLJFKsCZBuXYBare5Xf4eh5e/vD09PT3R0dNi0d3Z22l0p9YdcLre7Uuvs7ISXlxf8/PxuuF9/Cxyy9p+z+SiJuqRYE8C6RCLFmgDp1uUCh7cHZTIZwsLCUFtba9NeW1sLlUrl8okjIiJw7Ngxu2OGhYVBJnP5ApCIiCTMqdmDiYmJqKqqQmVlJZqbm5Gfnw+DwYD4+HgAQFFREbKysmz2aWpqQmNjI4xGI65cuYLGxkY0NjZat8fHx+PixYvIz89Hc3MzKisrUVVVhaSkpEEsj4iIpMSpS5qYmBgYjUaUlJRYFxenp6cjKCgIAKDX69HW1mazz/XruDZu3AgAeOONNwAAwcHBSE9PR2FhIbRaLeRyOVJTUxEdHT0ohRERkfQ4fR9u9uzZmD17dp/bli9fbteWmZnp8JiTJk3Cn/70J2eHQEREtzm+e5CIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEgZDi4iIhMHQIiIiYTC0iIhIGAwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBgMLSIiEobM2Y4VFRUoLS2FwWBAaGgolixZgokTJ96w/9mzZ7Fz506cPn0afn5+iIuLw4IFC+Dh4QEAqKurw+bNm+3227BhA5RKpQulEBGR1DkVWtXV1SgoKIBGo4FarUZFRQWys7ORkZGBoKAgu/6XL1/Gq6++CrVajeeffx4tLS3IycmBj48PkpKSbPpmZGTAz8/P+jkgIGCAJRERkVQ5dXuwvLwcsbGxiIuLg1KphEajgVwuh1ar7bP/559/jqtXr2L58uUYM2YMoqOjkZycjPLyclgsFpu+AQEBkMvl1i9PT96xJCKivjm80jKZTGhoaLC7QoqMjER9fX2f+9TX10OtVmPYsGHWtilTpmD37t1ob29HcHCwtf2ll16CyWSCUqlESkoKJk+e7GotREQkcQ5Dy2g0wmw2IzAw0KY9MDAQx44d63Ofjo4OjBgxwq5/77bg4GDI5XKkpaUhPDwcJpMJhw4dwiuvvIJVq1bd9FmZTqdzWJQYfG0+SaMuKdYEsC6RSLEmQLp1AWq1ul/9nZ6IcT2LxWKdVOGKkJAQhISEWD+rVCq0t7ejrKzspqHV3wKHrP3nbD5Koi4p1gSwLpFIsSZAunW5wOEDJH9/f3h6eqKjo8OmvbOz0+7qq1dgYKBd/97PN9oHAMLDw9Ha2upw0EREdHtyGFoymQxhYWGora21aa+trYVKpepzH5VKBZ1Oh+7ubpv+CoUCd9555w3PdebMGcjlcmfHTkREtxmnpuolJiaiqqoKlZWVaG5uRn5+PgwGA+Lj4wEARUVFyMrKsvafOnUqhg0bhpycHJw7dw6HDx/GJ598gsTEROstxb179+Lrr79Ga2srzp07h6KiItTU1GDOnDm3oEwiIpICp55pxcTEwGg0oqSkxLq4OD093bpGS6/Xo62tzdrf19cXK1euRF5eHjZt2gQ/Pz8kJiYiMTHR2sdkMmHXrl3Q6/Xw9va2HjMqKmqQSyQiIqlweiLG7NmzMXv27D63LV++3K5t7NixWLNmzQ2Pl5ycjOTkZGdPT0RExHcPEhGROBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCYGgREZEwGFpERCQMhhYREQmDoUVERMJgaBERkTAYWkREJAyGFhERCYOhRUREwmBoERGRMBhaREQkDIYWEREJg6FFRETCkDnbsaKiAqWlpTAYDAgNDcWSJUswceLEG/Y/e/Ysdu7cidOnT8PPzw9xcXFYsGABPDw8rH3q6upQWFiIpqYmKBQKzJs3DwkJCQMqiIiIpMupK63q6moUFBRg/vz5WL9+PVQqFbKzs3HhwoU++1++fBmvvvoqAgIC8PzzzyM1NRWlpaUoLy+39mlra0N2djZUKhXWr1+PBx98EPn5+fjqq68GpzIiIpIcD71eb3HU6c9//jPGjh2LpUuXWtuef/55REdHY/HixXb99+3bh/feew9ZWVkYNmwYAKC4uBharRb/+Z//CQ8PD7z77rv4+uuv8dJLL1n32759O5qamvDcc88NRm1DmiLnHADg9x7r3DwSIhLF/7P8GQCgf3KMm0fiPg5vD5pMJjQ0NCApKcmmPTIyEvX19X3uU19fD7VabQ0sAJgyZQp2796N9vZ2BAcH49SpU4iMjLTZb8qUKTh48CBMJhNkMqfvXArpn9902906DiISx0Z3D2AIcHh70Gg0wmw2IzAw0KY9MDAQBoOhz306Ojr67N+7DQAMBkOffXp6emA0Gp2vgIiIbhsuzx60WCw2kypccf3+Foulz3YiIiLAidDy9/eHp6en9QqpV2dnp92VUq/AwEC7/r2fe/eRy+V2V2qdnZ3w8vKCn5+f8xUQEdFtw2FoyWQyhIWFoba21qa9trYWKpWqz31UKhV0Oh26u7tt+isUCtx5550AgIiICBw7dszumGFhYZJ/nkVERK5x6vZgYmIiqqqqUFlZiebmZuTn58NgMCA+Ph4AUFRUhKysLGv/qVOnYtiwYcjJycG5c+dw+PBhfPLJJ0hMTLTe+ouPj8fFixeRn5+P5uZmVFZWoqqqym7CBxERUS+nprwDN19c/NZbb+HEiRPIzMy09j979izy8vJsFhcvXLjwhouL5XI5kpOT7RYX93dRswhOnDiBsrIyNDQ0QK/XY9myZZgxY4a7hzUgH330EQ4fPozW1lbIZDJERERg8eLFGDNG7Km5FRUV0Gq11jWJoaGhSElJwT333OPmkQ2ekpISfPDBB5g9ezY0Go27h+OyDz/8EHv27LFpCwwMtPmFWlR6vR5FRUU4cuQIurq6EBwcjLS0NEyaNMndQ3PJs88+2+c636ioKKSnp990X6dDyx2qq6uxbds2aDQaqNVqVFRUoKqqChkZGQgKCnL38Fx25MgR6HQ6hIWF4a233oJGoxE+tF599VVMnToV48ePh8Viwe7du3Hq1Cls2LBB6GeUNTU18PLywujRo2E2m3Hw4EGUlpbij3/8I8aOHevu4Q1YfX093njjDdxxxx1Qq9XCh1Z1dTVWrVplbfP09ERAQIAbRzVwly5dwsaNGzFhwgTMmTMH/v7+aG9vh0KhgFKpdPfwXNLZ2Qmz2Wz9bDAYsGnTJixbtgyxsbE33XdIv3uwvLwcsbGxiIuLg1KphEajgVwuh1ardffQBiQqKgqLFy9GdHS0ZGZKrly5EjNmzMCYMWMwduxYPPXUU+js7MTJkyfdPbQB+dnPfoaoqCiMGjUKISEhWLRoEXx8fG64RlEkly5dwrZt2/DEE0/A19fX3cMZFJ6enpDL5dYv0QMLAD755BPI5XI89dRTCA8PR3BwMO666y5hAwsAAgICbP6djhw5guHDhyM6OtrhvkN2xoMri5pp6Ojq6oLFYpHMD0MAMJvN+PLLL3HlypUbTkISSW5uLu69917cddddKC4udvdwBkV7eztWr14NLy8vREREYNGiRQgODnb3sAakpqYGU6ZMweuvv466ujrI5XLMmjULs2fPlsQvvRaLBfv378f06dPh4+PjsP+QDa2bLWq+ftYhDT35+fn4yU9+Iokf7mfPnkVmZia6u7vh4+OD3/3ud8LfGvy///s/nD9/Hk899ZS7hzJowsPDsWzZMiiVSnR0dKCkpASZmZnIyMiAv7+/u4fnsra2Nuzbtw+JiYl48MEHcebMGezcuRMAMGfOHDePbuBqa2vR3t6OmTNnOtV/yIbWjQzGoma6tQoKCnDy5EmsXbsWnp5D+g60U0JCQrB+/XpcunQJhw8fRk5ODlatWiXsJJOWlha8//77WLNmjaSWl0RFRdl8joiIwLp164SflWyxWDB+/Hjre17HjRuH1tZWVFRUSCK0KisrMX78eIwbN86p/kP2O9aVRc3kfgUFBaiursYzzzwj/G2ZXjKZDKNGjQIAjB8/Ht999x3Ky8uxbNky9w7MRfX19TAajXjxxRetbWazGTqdDlqtFq+99hq8vb3dN8BBMnz4cISGhuL8+fPuHsqAyOVyu+dXSqUSn376qZtGNHg6OjpQU1PTrwlAQza0fryo+b777rO219bWOvWwjv718vPz8cUXX2D16tVCPyR2xGw2w2QyuXsYLvv5z3+O8ePH27Tl5ORg9OjRmD9/vmSuvrq7u9HS0iLstPBeEyZMQEtLi01ba2ur0DOoex04cAAymQxTp051ep8h/d2ZmJiIbdu2ITw8HBMmTIBWq7VZ1Cyqrq4u629/FosF33//PRobG+Hn5yfsN+I777yDQ4cO4Xe/+x18fX2tr+jy8fHB8OHD3Tw617333nuIiorCyJEj0dXVhS+++AInTpzA73//e3cPzWW+vr52E2R8fHzg5+cn7C1PANi1axfuuecejBw5Ep2dnSguLsaVK1ccTqEe6ubOnYu//vWvKCkpQUxMDBobG/HZZ59h0aJF7h7agPROwJg6dWq/fkYM6XVagDQXF9fV1WHz5s127ffffz+WL1/uhhEN3L//+7/32b5w4UI89NBD/+LRDJ633noLdXV16OjowB133IGxY8ciKSkJd999t7uHNqhefvlljBkzRuh1Wv/7v/+LEydOwGg0IiAgABEREXj44YcRGhrq7qEN2Lfffov3338fLS0tGDlyJObMmYM5c+YI/Xz/+PHjyMrKwrp16xAeHu70fkM+tIiIiHqJP7WLiIhuGwwtIiISBkOLiIiEwdAiIiJhMLSIiEgYDC0iIhIGQ4uIiITB0CIiImEwtIiISBj/H+2JVEuNNiVvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pmf = 1/6*np.ones(6)\n",
    "diceN = np.arange(1,7)\n",
    "plt.stem(diceN, pmf,use_line_collection = True)\n",
    "plt.axis([0, 7, 0, .3])\n",
    "plt.title('PMF of A Fair Die')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Cumulative Distribution Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **cumulative distribution function** (CDF) of random variable $X$ is defined as\n",
    "$$\n",
    "F_{X}(x)=P(X \\leq x), \\text { for all } x \\in \\mathbb{R}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CDF of the die example is "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaQAAAEeCAYAAADFHWEmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAkb0lEQVR4nO3de1iUZf4/8DcHRU4D64mjIiCa5JgbQgkJYk7lqTy1KF6bLtZWtuK6lpoWm5alrawb1uVaKqGpgIkrhldARYhgSfrziMqICcqKgn6Z4Wgc5veHMTUxODMw7NwzvF/X5R9zP/czz+fTmG+e57mfwaq6uloFIiIiE7M2dQFEREQAA4mIiATBQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIurA/v37ERERgUGDBsHV1RUvv/yyqUsyiry8PLi6uuK9994zdSkAgN27d8PV1RW7d+82dSlkYramLoDMy+XLl7Ft2zbk5eXh2rVraGhogKurK0aOHIlJkyZhzpw5kEgk6vlSqRTXrl1Tv7a1tYWTkxPc3d0xcuRIyGQyPP3007C3t293rLy8PEybNu2+9Rw5cgSjRo0yXoM/O378OJ5//nm4ubnhueeeg5OTE6RSqUHvMW7cOJw9exbDhw/H999/b3ANpuzfUF35nInaMJBIb/Hx8Vi3bh1aW1sRFBSEqKgoSCQSVFVV4dixY1i+fDnWr1+PK1eutNv3pZdegouLC1QqFWpqanDlyhVkZWXh888/x9q1a/HRRx9h/PjxWo87aNAgREdHa93m5uZmzBbVsrKyoFKpsG7dOsyaNcvg/U+cOIGzZ8/CysoKly5dQkFBAUJDQztVi7H7DwoKwvHjx9GvX79O1XM/nfmcp06diuDg4G77LMl8MJBIL//617/w9ttvw8vLCzt27MAjjzzSbk5+fj6WL1+udf+XX34ZPj4+GmP19fVISEjAhg0bEBUVhfT0dK3vO3jwYLz++uvGaURPN27cAAAMHDiwU/snJiYCAJYtW4aNGzfi008/7XQgGbt/BwcHDBs2zGjv92ud+ZxdXFzg4uLSLfWQeeE9JNKprKwM69atQ69evZCcnKw1NAAgLCwM33zzjd7v6+DggJUrV2LZsmW4e/duh2FmDHfv3sUHH3yAsLAweHh4wNvbGxMnTsTOnTuhUv3y7Vm/vZ8xbdo0uLq6wtXVFaWlpXodS6lU4sCBA/D29sbrr78OX19fpKen4//+7/+6pbfLly/jrbfewvjx4+Hv74+BAwdi5MiRiI2N1biM1qaje0hTpkyBq6srrl69is2bN+PRRx+Fm5tbh2dn+tL1Od/vHtLNmzexcuVKPPzww3Bzc4OPjw9mzJiB3NzcLtVEYmIgkU67d+9GU1MTpk6dqvM+ip2dncHvv2TJEtjb2+P06dO4dOlSZ8vsUFNTE2bNmoW///3vuHv3LmJiYjB37lyUl5cjNjZWY7GCVCrFihUrMHLkSADA3LlzsWLFCqxYsULvn+JTU1NRV1eHOXPmwMbGBtHR0WhsbMSePXuM3hsAHDp0CDt27ICXlxdmzZqFP//5zxg+fDh27dqFCRMmoLy83KD3W758OTZu3IhRo0bhxRdfxOjRo41Sp6Gf8/nz5zFu3Dhs3boVvr6+eOGFFzBlyhScPHkS06dPx65du4xSF4mDl+xIp2PHjgEAIiMju+X9nZ2dMXr0aBw7dgw//PADhg8frrG9rKxM64qw4OBgTJw4Uef7b968GUePHsWECROQnJyM3r17AwDeeOMNPPXUU0hOTsZTTz2F6dOnY9SoURg1ahTKyspw7tw5REdHY9y4cQb18+mnnwIA5s2bB+BeqL333ntISkrCK6+8YtB7Abr7j4qKwqJFi9r9MJCdnY2oqChs3LgRmzZt0vt4Z8+exZEjR9pdeusqXZ/zr7W0tGD+/PlQKBQ4dOgQHnvsMfW2iooKPP7443jttdfw5JNPdvqyKomHgUQ63bx5EwDg6enZbcdwd3cHAFRVVbXbdu3aNWzYsKHd+EsvvaRXIH322WcAgHXr1qnDCLh37yIuLg5z585FUlISpk+f3snqf/HDDz/g3LlzCA0Nha+vLwDA29sbERERyMnJQX5+PsLCwgx6T139d/S5yGQyPPDAAwZdRgWAxYsXGz2M2tzvc/61rKwsXL58GYsWLdIIo7b3WLx4MVauXImDBw/ihRde6JZa6X+PgUQ6td1jsbKy6vZjaTtGWFgYMjIyOvV+bSu9Bg4ciBEjRrTbHhERAQA4ffp0p97/t9oWM7SdHbWZN28ecnJykJSUZHAg6epfpVIhNTUVe/bswblz51BdXY2Wlhb19l+HsD7GjBlj0PzO0PV3qW2Z/PXr17WeHbat5CwuLjZ+cWQyDCTSyd3dHcXFxQbfizBERUUFABh9KbJSqQTQ8Wo5BwcHSCQS9byuHuvAgQNwdHTEM888o7Ft6tSpcHFxwcGDB7F+/Xr07du3y8drs2rVKmzZsgXu7u54/PHH4eHhgT59+gAA9uzZo3Vhw/105yUwfT/nO3fuAADS09ORnp7e4by6ujrjFUcmx0AincaOHYsjR44gNzcXzz33nNHfX6lU4tSpUwDu3RcxpraHdG/duqV1e319PZRKpVECIiUlBfX19QDuXabryJ49e/CXv/yly8cDgMrKSmzduhWBgYHIzMyEs7Ozxvb9+/cb/J7ddSZsyOfc9rnt3LkTTz/9dLfUQ+JhIJFO8+bNwz//+U+kp6ejqKgIgYGBHc69e/euwSvtEhIS0NDQgNGjRxv9+RhnZ2f4+fnhypUruHjxIh544AGN7UeOHAEAo6wkS0pKAgBMnz69XTAA9/7bpKamYufOnUYLpKtXr6K1tRWRkZHtjlleXo6rV68a5TjGYMjn3BZYx44dYyD1IFz2TToNHjwYq1evRlNTE/7whz+gsLBQ67zvvvtOr0UGberr67FhwwbEx8fDzs5O6417Y/jjH/8I4N6quqamJvW4UqnE2rVrAaDLZ36FhYU4d+4cfH19kZiYiM2bN7f78/HHH+Ohhx5CcXExjh492qXjtRk8eDCAe//tf33fqLa2FkuWLEFzc7NRjtMVnfmcJ0+eDD8/PyQmJuLw4cNa55w+fVp9aY8sA8+QSC9//etf0dzcjHfffRcymQxjxozBww8/DGdnZ9y+fRvHjx9HUVFRh/cGtmzZov5KmdraWly5cgUFBQVQKpXw9PTERx991OEDt131yiuv4KuvvsJXX32F0NBQPPnkk2hqasKhQ4fw3//+F3PmzOnyCru2pd7z58+/7yWv+fPn429/+xuSkpLarR7rDDc3N8yaNQv79+/HuHHjEBkZCaVSiZycHPTp0wdSqRRnz57t8nH0ZazPuVevXvjss88wc+ZMREdHY8yYMXjooYfg6OiI8vJynDlzBnK5HEeOHDHq/TgyLQYS6e3VV1/F9OnTsW3bNhw5cgTJycmor6+Hq6srAgMDsWHDhg6f6v/3v/8NALCxsVF/6aZMJlN/6aaDg0O31d27d2+kpaVhy5YtSE1NxbZt22BtbY0RI0Zg5cqV6jOozlIoFDhw4ABsbW11fqvB7Nmz8eabbyI9PR0bNmwwyj+mmzdvxpAhQ5CWloZt27ahf//+mDRpElatWtXl3gxlzM85MDAQ+fn52LJlCw4fPoy9e/dCpVLBzc0NDzzwABYvXoyAgIDuaoVMwKq6ulqlexoREVH34j0kIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkLqRXC43dQlGZ4k9AezLnFhiT4Dl9mUIBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREKw1WdScXExsrKyUFpaiurqaixYsABhYWH33ef69evYu3cvfvzxRzg6OiI8PBxTp06FlZWVUQonIiLLotcZ0t27d+Hp6Yk5c+agd+/eOuc3NDRg06ZNcHZ2xurVqzFnzhxkZmYiOzu7ywUTEVmSfSV1kKZWIOSoPaSpFdhXUmfqkkxGrzMkqVQKqVQKAEhMTNQ5//vvv8dPP/2EmJgY9O7dG15eXrhx4ways7Mhk8l4lkREhHthFJuvQEOLCoAVrtW1IDZfAQB41t/RtMWZQLfcQyopKUFAQIDG2dSDDz6I6upqVFVVdcchiYjMztoTNT+H0S8aWlRYe6LGRBWZll5nSIZSKpX43e9+pzEmkUjU2wYMGKB1P0v8Fb7syXywL/NhKT1dr7MH0P6K0fW6ZovoMSAgwKD53RJInWVo8aKTy+XsyUywL/NhST15/78KXKtraT/uaGsxPRqiWy7ZSSQSKJVKjbG2121nSkREPV1ckHO7MXsbK63jPUG3BJK/vz/kcjmamprUY0VFRXB1dUX//v2745BERGZH28KFhDCXHrmgAdAzkBobG1FWVoaysjKoVCrcuXMHZWVluH37NgAgLS0N8fHx6vkhISHo3bs3EhMTUV5ejpMnT+LLL7/kCjsiIh16ahgBet5DKi0txcaNG9Wv09PTkZ6ejrFjxyImJgbV1dWorKxUb3dwcMDSpUuxZ88evPPOO3B0dIRMJoNMJjN+B0REZBH0CqThw4fjk08+6XB7TExMuzFvb28sX76885UREVGPwu+yIyIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIjIL+0rqIE2tQMhRe0hTK7CvpM7UJZGR2Zq6ACIiXfaV1CE2X4GGFhUAK1yra0FsvgIA8Ky/o2mLI6PhGRIRCW/tiZqfw+gXDS0qrD1RY6KKqDswkIhIeNfrWgwaJ/PEQCIi4Xk72hg0TuZJ73tIOTk5yMzMhEKhgKenJ6KiojBs2LAO5587dw6HDh1CeXk5evXqBX9/f8yePRvu7u5GKZyIeo64IGe8cKRaY8zexgpxQc6mKYi6hV5nSIWFhUhJScHkyZMRFxcHf39/JCQk4Pbt21rnV1ZW4qOPPkJAQADi4uKwdOlSNDU1ISEhwajFE1HPoG3hQkKYCxc0WBi9Aik7OxuhoaEIDw+Hh4cHoqOj4eLigtzcXK3zy8rK0NLSgpkzZ2LgwIEYPHgwJk2ahMrKStTU8CYkEXUdw8jy6Ayk5uZmlJaWIjAwUGM8MDAQJSUlWvfx8fGBjY0N8vLy0NraisbGRhQUFGDIkCFwduYpNhERtafzHlJtbS1aW1shkUg0xiUSCS5cuKB1n/79+2Pp0qXYunUr9uzZA5VKhUGDBmHJkiXGqZqIiCxOpx+MValUsLKy0rpNoVAgKSkJY8eORUhICBobG3Hw4EFs3boVy5Ytg7W19hMzuVze2XKExZ7MB/sSnYPGK/YlvoCAAIPm6wwkJycnWFtbQ6lUaozX1NS0O2tqk5OTAzs7O8yePVs9tnDhQqxYsQIlJSUdFmlo8aKTy+XsyUywLzNwtFzjJfuyPDrvIdna2sLHxwdFRUUa40VFRfD399e6z08//dTuLKjttUql0rYLERH1cHqtspPJZCgoKEBeXh5u3LiB5ORkKBQKREREAADS0tIQHx+vni+VSlFWVob09HTcvHkTpaWl+PTTT9G3b1/4+Ph0TydERGTW9LqHFBwcjNraWmRkZKgfjI2NjUW/fv0AANXV1aisrFTPHzFiBJ5//nlkZmYiKysLvXr1gp+fH5YsWQI7O7vu6YSIiMya3osaIiMjERkZqXVbTExMu7GQkBCEhIR0vjIiIupR+F12REQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBFZoH0ldZCmViDkqD2kqRXYV1Jn6pKIdLI1dQFEZFz7SuoQm69AQ4sKgBWu1bUgNl8BAHjW39G0xRHdB8+QiCzM2hM1P4fRLxpaVFh7osZEFRHph4FEZGGu17UYNE4kCgYSkYXxdrQxaJxIFAwkIgsTF+TcbszexkrrOJFIGEhEFkbbwoWEMBcuaCDhMZCIegCGEZkDBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCUHv77LLyclBZmYmFAoFPD09ERUVhWHDhnU4X6VS4euvv0Zubi6qqqrg6OiIsWPHYtasWUYpnIiILItegVRYWIiUlBRER0cjICAAOTk5SEhIwJo1a9CvXz+t+6SmpuLMmTN49tln4eXlhYaGBigUCqMWT0RElkOvQMrOzkZoaCjCw8MBANHR0Th//jxyc3Mxc+bMdvMrKiqQk5ODv//97/Dw8DBuxUREZJF0BlJzczNKS0vxxBNPaIwHBgaipKRE6z6nTp1C//79ce7cOSQkJEClUmHYsGGYPXs2JBKJcSonIiKLojOQamtr0dra2i5IJBIJLly4oHWfyspK3L59G4WFhfjTn/4EAPj888/x4YcfYuXKlbC21r6WQi6XG1q/8NiT+bCsvhw0XllGb5bYE2C5fQEBAQEGze/0L+hTqVSwsrLqcFtzczNiYmLg7u4OAIiJicGbb76Jq1evws/PT+t+hhYvOrlczp7MhMX1dbRc46VF9GaJPQGW21cn6Fz27eTkBGtrayiVSo3xmpqaDi+/ubi4wMbGRh1GAODm5gYbGxvcuXOniyUTEZEl0hlItra28PHxQVFRkcZ4UVER/P39te4zdOhQtLS04NatW+qxyspKtLS0dLgqj4iIeja9HoyVyWQoKChAXl4ebty4geTkZCgUCkRERAAA0tLSEB8fr54/YsQIDB48GElJSSgrK0NZWRmSkpLg6+sLHx+f7umEiIjMml73kIKDg1FbW4uMjAz1g7GxsbHqs53q6mpUVlaq51tbW2Px4sVITk7G+++/j969e2PEiBGIiorqcEEDERH1bHovaoiMjERkZKTWbTExMe3GXF1d8dJLL3W+MiIi6lF4ukJEREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhL1aPtK6iBNrUDIUXtIUyuwr6TO1CUR9Vid/o2xROZuX0kdYvMVaGhRAbDCtboWxOYrAADP+juatjiiHohnSNRjrT1R83MY/aKhRYW1J2pMVBFRz8ZAoh7rel2LQeNE1L0YSNRjeTvaGDRORN2LgUQ9VlyQc7sxexsrreNE1P0YSNRjaVu4kBDmwgUNRCbCQCL6FYYRkekwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiHo/SvMc3JykJmZCYVCAU9PT0RFRWHYsGE697t58ybefvttAMCHH37Y+UqJiMii6XWGVFhYiJSUFEyePBlxcXHw9/dHQkICbt++fd/9mpub8fHHH+sVXERE1LPpFUjZ2dkIDQ1FeHg4PDw8EB0dDRcXF+Tm5t53v/3798Pb2xtBQUFGKZaIiCyXzkBqbm5GaWkpAgMDNcYDAwNRUlLS4X5nzpzBmTNnMHfu3K5XSUREFk/nPaTa2lq0trZCIpFojEskEly4cEHrPtXV1di5cydefvll9OnTR+9i5HK53nPNBXsSnYPGK8vpzRL7ssSeAMvtCwgICDBovt6LGn5LpVLByspK67bt27dj/Pjx8Pf3N+g9DS1edHK5nD2J7mi5xkuL6c0S+7LEngDL7asTdAaSk5MTrK2toVQqNcZramranTW1uXjxIoqLi3Ho0CEA98JLpVLhxRdfxLx58xAeHm6E0omIyJLoDCRbW1v4+PigqKgIY8aMUY8XFRV1uFjhrbfe0nh96tQpHD58GKtWrYKrq2uXCiYiIsuk1yU7mUyG7du3w9fXF0OHDkVubi4UCgUiIiIAAGlpafjxxx+xbNkyAICXl5fG/levXoWVlVW7cSIiojZ6BVJwcDBqa2uRkZGhfjA2NjYW/fr1A3BvEUNlZWW3FkpERJZN70UNkZGRiIyM1LotJibmvvuGhYUhLCzMsMqIiKhH4XfZERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEikl30ldZCmViDkqD2kqRXYV1Jn6pKIyMJ0+rvsqOfYV1KH2HwFGlpUAKxwra4FsfkKAMCz/o6mLY6ILAbPkEintSdqfg6jXzS0qLD2RI2JKiIiS8RAIp2u17UYNE5E1BkMJNLJ29HGoHEios5gIJFOcUHO7cbsbay0jhMRdRYDiXTStnAhIcyFCxqIyKgYSNQpDCMiMjYGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRAYSEREJAQGEhERCYGBREREQmAgERGREBhIREQkBAYSEREJgYFERERCYCAREZEQbPWdmJOTg8zMTCgUCnh6eiIqKgrDhg3TOvfSpUvIzs7G1atX0dDQgAEDBmDixIl47LHHjFY4ERFZFr0CqbCwECkpKYiOjkZAQABycnKQkJCANWvWoF+/fu3mX758GV5eXnjqqafg4uKC8+fPY9euXejVqxceeeQRozdBRETmT69Ays7ORmhoKMLDwwEA0dHROH/+PHJzczFz5sx286dMmaLxevz48bh48SJOnjzJQCIiIq103kNqbm5GaWkpAgMDNcYDAwNRUlKi94EaGxvh4OBgeIVERNQj6DxDqq2tRWtrKyQSica4RCLBhQsX9DrI6dOncfHiRaxYseK+8+RyuV7vZ04spyfNHybYl+gssS9L7Amw3L6AgIAAg+brvajht1QqFaysrHTOu3z5MrZt24Y5c+bA19f3vnMNLV50crnccno6Wq7xkn0JzhL7ssSeAMvtqxN0BpKTkxOsra2hVCo1xmtqatqdNf2WXC5HQkICnnnmGYwfP75LhRIRkWXTeQ/J1tYWPj4+KCoq0hgvKiqCv79/h/sVFxfjgw8+wLRp0zBx4sSuV0pERBZNrwdjZTIZCgoKkJeXhxs3biA5ORkKhQIREREAgLS0NMTHx6vnX7p0CR988AEiIiLwyCOPQKFQQKFQoKampnu6EMy+kjpIUysQctQe0tQK7CupM3VJRETC0+seUnBwMGpra5GRkaF+MDY2Nlb9DFJ1dTUqKyvV8/Pz8/HTTz8hKysLWVlZ6vF+/fph/fr1Rm5BLPtK6hCbr0BDiwqAFa7VtSA2XwEAeNbf0bTFEREJTO9FDZGRkYiMjNS6LSYmpt3r3471FGtP1PwcRr9oaFFh7YkaBhIR0X3wu+yM7Hpdi0HjRER0DwPJyLwdbQwaJyKiexhIRhYX5NxuzN7GSus4ERH9goFkZNruEyWEufD+ERGRDgyk/wGGERGRbgwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEwEAiIiIhMJCIiEgIDCQiIhICA4mIiITAQCIiIiEwkIiISAgMJCIiEgIDiYiIhMBAIiIiITCQiIhICAwkIiISAgOJiIiEYKvvxJycHGRmZkKhUMDT0xNRUVEYNmxYh/OvX7+OvXv34scff4SjoyPCw8MxdepUWFlZGaVwIiKyLHqdIRUWFiIlJQWTJ09GXFwc/P39kZCQgNu3b2ud39DQgE2bNsHZ2RmrV6/GnDlzkJmZiezsbKMWT0RElsOqurpapWvSu+++C29vbzz33HPqsdWrVyMoKAgzZ85sN//bb7/F/v37ER8fj969ewMAvvjiC+Tm5uL999+3+LMk18RyAMBiq1UmroSIzMVm1bsAgOo/eZm4EtPRecmuubkZpaWleOKJJzTGAwMDUVJSonWfkpISBAQEqMMIAB588EEcPHgQVVVVGDBgQBfLFtsvf6GSTFoHEZmPt01dgAB0XrKrra1Fa2srJBKJxrhEIoFCodC6j1Kp1Dq/bRsREdFvdXqVnUqlsvhLb0RE9L+jM5CcnJxgbW3d7sympqam3VlQG4lE0m5+2+uO9iEiop5NZyDZ2trCx8cHRUVFGuNFRUXw9/fXuo+/vz/kcjmampo05ru6uqJ///5dLJmIiCyRXpfsZDIZCgoKkJeXhxs3biA5ORkKhQIREREAgLS0NMTHx6vnh4SEoHfv3khMTER5eTlOnjyJL7/8EjKZjJf5iIhIK70ejA0ODkZtbS0yMjLUD8bGxsaiX79+AIDq6mpUVlaq5zs4OGDp0qXYs2cP3nnnHTg6OkImk0Emk2m8r6EP25qD4uJiZGVlobS0FNXV1ViwYAHCwsJMXVaXHD58GCdPnsTNmzdha2sLPz8/zJw5E15e5r08NScnB7m5uern6Tw9PTFlyhSMGjXKxJUZT0ZGBv7zn/8gMjIS0dHRpi6n09LT03Ho0CGNMYlEovGDsLmqrq5GWloazp49i8bGRgwYMADz5s3D8OHDTV1ap6xcuVLrM6pSqRSxsbH33Vev55C6Q2FhIbZv347o6GgEBAQgJycHBQUFWLNmjTrozNHZs2chl8vh4+ODHTt2IDo62uwDadOmTQgJCcGQIUOgUqlw8OBBXLlyBWvXroWjo6Opy+u0U6dOwcbGBm5ubmhtbcWxY8eQmZmJN954A97e3qYur8tKSkrwySefwN7eHgEBAWYfSIWFhXj11VfVY9bW1nB2djZhVV1XX1+Pt99+G0OHDsWECRPg5OSEqqoquLq6wsPDw9TldUpNTQ1aW1vVrxUKBd555x0sWLAAoaGh993XZN9ll52djdDQUISHh8PDwwPR0dFwcXFBbm6uqUoyCqlUipkzZyIoKMhiLk8uXboUYWFh8PLygre3NxYuXIiamhpcvnzZ1KV1yejRoyGVSjFw4EC4u7tjxowZsLOz6/D5OnNSX1+P7du3Y/78+XBwcDB1OUZhbW0NFxcX9R9zDyMA+PLLL+Hi4oKFCxfC19cXAwYMwIgRI8w2jADA2dlZ43M6e/Ys+vTpg6CgIJ376v1ddsbUmYdtSRyNjY1QqVQW8w8dALS2tuKHH37A3bt3O1ysY0527dqFhx9+GCNGjMAXX3xh6nKMoqqqCq+99hpsbGzg5+eHGTNmmP1D9qdOncKDDz6IrVu34tKlS3BxccG4ceMQGRlpET/QqlQqHD16FI8++ijs7Ox0zjdJIN3vYdsLFy6YoiQyQHJyMgYNGmQR/3Bfv34d69evR1NTE+zs7LBo0SKzv1x35MgR3Lp1CwsXLjR1KUbj6+uLBQsWwMPDA0qlEhkZGVi/fj3WrFkDJycnU5fXaZWVlfj2228hk8kwadIkXLt2DXv37gUATJgwwcTVdV1RURGqqqrw2GOP6TXfJIHUET5sK76UlBRcvnwZK1asgLW1+f/2End3d8TFxaG+vh4nT55EYmIiXn31VbNdsFFRUYEDBw5g+fLlsLUV6n/vLpFKpRqv/fz8sGrVKhQUFLS70mJOVCoVhgwZov5O0MGDB+PmzZvIycmxiEDKy8vDkCFDMHjwYL3mm+RvbGcetiXTS0lJQWFhIZYtW2b2l0ra2NraYuDAgQCAIUOG4OrVq8jOzsaCBQtMW1gnlZSUoLa2Fm+99ZZ6rLW1FXK5HLm5ufjwww/Rq1cv0xVoJH369IGnpydu3bpl6lK6xMXFpd39Ig8PD3z99dcmqsh4lEolTp06ZdBiGpME0q8fth0zZox6vKioSK8bX/S/l5ycjOPHj+O1114z6xuuurS2tqK5udnUZXTa73//ewwZMkRjLDExEW5ubpg8ebLFnDU1NTWhoqLCbJdGtxk6dCgqKio0xm7evGnWK43b5Ofnw9bWFiEhIXrvY7K/nTKZDNu3b4evry+GDh2K3NxcjYdtzVVjY6P6pzaVSoU7d+6grKwMjo6OZvuXbPfu3fjuu++waNEiODg4qL9U187ODn369DFxdZ23f/9+SKVS9O3bF42NjTh+/DiKi4uxePFiU5fWaQ4ODu0Wm9jZ2cHR0dFsL0MCwL59+zBq1Cj07dsXNTU1+OKLL3D37l2dy4hFN3HiRGzYsAEZGRkIDg5GWVkZvvnmG8yYMcPUpXVJ22KGkJAQg/6NMNlzSIBlPhh76dIlbNy4sd342LFjERMTY4KKuu6FF17QOj5t2jQ8/fTT/+NqjGfHjh24dOkSlEol7O3t4e3tjSeeeAIjR440dWlG9Y9//ANeXl5m/RzSxx9/jOLiYtTW1sLZ2Rl+fn545pln4OnpaerSuuzMmTM4cOAAKioq0LdvX0yYMAETJkww6/vpFy9eRHx8PFatWgVfX1+99zNpIBEREbUx/2VSRERkERhIREQkBAYSEREJgYFERERCYCAREZEQGEhERCQEBhIREQmBgUREREJgIBERkRD+P9uF/rY40O6aAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pmf = 1/6*np.ones(6)\n",
    "cdf = np.cumsum(pmf)\n",
    "plt.stem(diceN, cdf ,use_line_collection = True)\n",
    "plt.axis([0, 7, 0, 1.1])\n",
    "plt.title('CDF of A Fair Die')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One particular useful formula for calculating probability within an interval is\n",
    "\n",
    "$$\n",
    "P(a<X\\leq b) = F_X(b)-F_X(a)\n",
    "$$\n",
    "\n",
    "It is commonly used for computing the probability of a range, for instance, what is the probability $P(2<X\\leq 5)$ in this case?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the formula, \n",
    "\n",
    "$$\n",
    "P(2<X\\leq5) = F_X(5) - F_X(2) = 5/6 - 2/6 = 1/2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Probability Density Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **Probability Density Function** (PDF) is concept used for continuous distribution. For instance the temperature of a room follows a continuous distribution.\n",
    "\n",
    "We denote PDF as $f_X(x)$, however any realization of $x$ must be $0$ probability, using the room temperature example, the probability that a room reaches exactly $20  ^\\circ \\text{C}$ is theoretically $0$. $f_X(x = 25)= 0$. Therefore to obtain positive probability, we must delimit range of the arguments $(x,\\ \\Delta x)$\n",
    "\n",
    "$$\n",
    "f_{X}(x)=\\lim _{\\Delta \\rightarrow 0^{+}} \\frac{P(x<X \\leq x+\\Delta)}{\\Delta}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Furthermore, use CDF formula and definition of derivative, the relation of CDF and PDF\n",
    "\n",
    "$$\n",
    "f_{X}(x)=\\lim _{\\Delta \\rightarrow 0} \\frac{F_{X}(x+\\Delta)-F_{X}(x)}{\\Delta}=\\frac{d F_{X}(x)}{d x}=F_{X}^{\\prime}(x)\n",
    "$$\n",
    "\n",
    "We will see examples in next chapter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\">Independent Random Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Independent R.V.s are similar to independent events, recall that independent events have the property\n",
    "\n",
    "$$\n",
    "p(A\\cap B) =p (A)p(B)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now consider two random variable $A$ and $B$, they are independent as long as\n",
    "\n",
    "$$\n",
    "p(X=x,Y=y)=p(X=x)p(X=y)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, independent variables have the property:\n",
    "\n",
    "$$\n",
    "p(X_1=x_1,X_2=x_2, ..., X_n = x_n)=p(X_1=x_1)p(X_2=x_2)...p(X_n=x_n)=\\prod_{i=1}^np(X_i=x_i)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Expected Value and Variances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The **expected value** of discrete and continuous random variables are\n",
    "\n",
    "$$\n",
    "\\text{Discrete:}\\qquad E(X)=\\sum_{i=1}^k x_ip_X(x_i)=\\sum_{i=1}^k x_i p(X_i = x_i)\n",
    "$$\n",
    "\n",
    "$$\\text{Continuous:}\\qquad E(X) = \\int_{-\\infty}^{\\infty}xf_X(x)dx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "They are expressing the same idea that weighting each possibilities equally, then sum up."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The variance of discrete and continuous random variables are similar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\text{Discrete:}\\qquad\\operatorname{Var}(X)=E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right]=E (X^{2})-[E (X)]^{2}\n",
    "$$\n",
    "\n",
    "\n",
    "\\begin{aligned}\n",
    "\\text{Continuous:}\\qquad \n",
    "\\operatorname{Var}(X)&= E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right]=\\int_{-\\infty}^{\\infty}\\left(x-\\mu_{X}\\right)^{2} f_{X}(x) d x \\\\\n",
    "&=E (X^{2})-[E (X)]^{2}=\\int_{-\\infty}^{\\infty} x^{2} f_{X}(x) d x-\\mu_{X}^{2}\n",
    "\\end{aligned}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And a common method for manual calculation of variance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "\\operatorname{Var}(X) &=E\\left[\\left(X-\\mu_{X}\\right)^{2}\\right] \\\\\n",
    "&=E\\left[X^{2}-2 \\mu_{X} X+\\mu_{X}^{2}\\right] \\\\\n",
    "&=E\\left[X^{2}\\right]-2 E\\left[\\mu_{X} X\\right]+E\\left[\\mu_{X}^{2}\\right]\\\\\n",
    "&=E\\left[X^{2}\\right]-2 \\mu_{X}^{2}+\\mu_{X}^{2}\\\\\n",
    "&=E\\left[X^{2}\\right]-\\mu_{X}^{2}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}