done reading chapter 3

study/001_introduction-to-probability-statistics-and-random-processes/README.md

@@ -4,11 +4,12 @@
 
 total pages=1007
 
-**Currently reading:** chapter 3, page 190
+**Currently reading:** chapter 3, page 222
 
 TODO:
 
 - ch1 end of chapter problems
 - ch2 end of chapter problems
 - 3.1.6 problems
+- 3.2.5 problems
 - ch3 end of chapter problems

study/001_introduction-to-probability-statistics-and-random-processes/ch3/notes.ipynb

@@ -464,8 +464,116 @@
    "id": "9768e347",
    "metadata": {},
    "source": [
+    "NOTE: If we have the PMF, we can find the CDF from it. In particular, if $R_X = \\{ x_1, x_2, x_3, \\dots \\}$, we can write\n",
+    "\n",
+    "$$F_X(x) = \\sum_{x_k \\leq x}P_X(x_k)$$\n",
+    "\n",
     "NOTE: For all $a \\leq b$, we have\n",
-    "$$P(a \\lt X \\leq b) = F_X(b) - F_X(a)$$"
+    "$$P(a \\lt X \\leq b) = F_X(b) - F_X(a)$$\n",
+    "\n",
+    "NOTE: To find $P(X \\lt x)$, for a discrete random variable, we can simply write\n",
+    "\n",
+    "$$P(X \\lt x) = P(X \\leq x) - P(X = x) = F_X(x) - P_X(x)$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0fd28226",
+   "metadata": {},
+   "source": [
+    "## Expectation (3.2.2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d9ad903b",
+   "metadata": {},
+   "source": [
+    "***Definition*** Let $X$ be a discrete random variable with range $R_X = \\{ x_1, x_2, x_3, \\dots \\}$ (finite or countably infinte). The *expected* value of $X$, denoted by $E[X]$ is defined as\n",
+    "\n",
+    "$$E[X] = \\sum_{x_k \\in R_X}x_kP(X=x_k) = \\sum_{x_k \\in R_X}x_kP_X(x_k)$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e4c5500d",
+   "metadata": {},
+   "source": [
+    "***Theorem*** Expectation is linear. We have\n",
+    "- $E[aX +b] = aE[X] + b, \\forall a,b \\in \\mathbb{R}$\n",
+    "- $E[X_1 + X_2 + \\dots + X_n] = E[X_1] + E[X_2] + \\dots + E[X_n]$, for any set of random variables $X_1, X_2, \\dots, X_n$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "110d9ecf",
+   "metadata": {},
+   "source": [
+    "## Functions of Random Variables (3.2.3)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1ed681ae",
+   "metadata": {},
+   "source": [
+    "Law of the unconscious statistician (LOTUS) for discrete random variables:\n",
+    "\n",
+    "$$E[g(X)] = \\sum_{x_k \\in R_X}g(x_k)P_X(x_k)$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9c9c6a44",
+   "metadata": {},
+   "source": [
+    "## Variance (3.2.4)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5d7e781f",
+   "metadata": {},
+   "source": [
+    "***Definition*** The **variance** of a random variable $X$ is defined as\n",
+    "\n",
+    "$$\\text{Var}(X) = E[(X - E[X])^2] = \\sum_{x_k \\in R_X} (x_k - E[X])^2P_X(x)$$\n",
+    "\n",
+    "Or\n",
+    "\n",
+    "$$\\text{Var}(X) = E[X^2] - (E[X])^2 = \\sum_{x_k \\in R_X}x_k^2P_X(x_k) - (E[X])^2$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1def8444",
+   "metadata": {},
+   "source": [
+    "***Definition*** The *standard deviation* of a random variable $X$ is defined as\n",
+    "\n",
+    "$$\\text{SD}(X) = \\sigma_X = \\sqrt{\\text{Var}(X)}$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "def50a00",
+   "metadata": {},
+   "source": [
+    "***Theorem*** For a random variable $X$ and real numbers $a$ and $b$,\n",
+    "\n",
+    "$$\\text{Var}(aX + b) = a^2\\text{Var}(X)$$\n",
+    "\n",
+    "$$\\text{SD}(aX + b) = |a|\\text{SD}(X)$$"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e32be7d1",
+   "metadata": {},
+   "source": [
+    "***Theorem*** If $X_1, X_2, \\dots, X_n$ are independent random variables and $X = X_1 + X_2 + \\dots + X_n$, then\n",
+    "\n",
+    "$$\\text{Var}(X) = \\text{Var}(X_1) + \\text{Var}(X_2) + \\dots + \\text{Var}(X_n) $$"
    ]
   }
  ],