State De Morgan's Law for $n$ sets --- For any sets $A_1, A_2, \dots, A_n$: $$\left(\bigcup_{i=1}^n A_i\right)^c = \bigcap_{i=1}^n A_i^c$$ $$\left(\bigcap_{i=1}^n A_i\right)^c = \bigcup_{i=1}^n A_i^c$$ The complement of a union is the intersection of complements, and vice versa. === State the Distributive Law for sets $A$, $B$, and $C$ --- $$A \cap (B \cup C) = (A \cap B) \cup (A \cap C)$$ $$A \cup (B \cap C) = (A \cup B) \cap (A \cup C)$$ === State the inclusion-exclusion principle for a finite collection of sets $A_1, A_2, A_3, \dots A_n$ where $n=2$ --- $n = 2$ case: $|A \cup B| = |A| + |B| - |A \cap B|$ --- $n = 3$ case: $|A \cup B \cup C| = |A| + |B| + |C| - |A \cap B| - |A \cap C| - |B \cap C| + |A \cap B \cap C|$ --- For a finite collection of sets $A_1, A_2, A_3, \dots A_n$, we have $\left| \bigcup_{i=1}^n A_i \right| = \sum_{i=1}^n |A_i| - \sum_{i < j} |A_i \cap A_j| + \sum_{i < j < k} |A_i \cap A_j \cap A_k| - \dots + (-1)^{n-1} |A_1 \cap A_2 \cap \dots \cap A_n|$ === For the function $$f: A \to B$$ State the following of $f$: 1. Domain 2. Co-domain 3. Range --- 1. $A$ 2. $B$ 3. Set of all possible outputs of $f$ (not necessarily $B$) === State definition for each the following: 1. Random experiment 2. Outcome 3. Sample space 4. Event --- 1. A **random experiment** is a process by which we observe something uncertain 2. An **outcome** is a result of a random experiment 3. The **sample space** $S$ is the set of all possible outcomes 4. An **event** is a subset of the sample space === State the Axioms of Probability --- **Axioms of Probability** 1. For any event $A$, $P(A) \geq 0$ 2. $P(S) = 1$ 3. If $A_1, A_2, A_3, \dots$ are disjoint events, then $P(A_1 \cup A_2 \cup A_3 \cup \dots) = P(A_1) + P(A_2) + P(A_3) + \dots$ === In a finite sample space $S$, where all outcomes are equally likely, what is the probability of any event $A$? --- $P(A) = \frac{|A|}{|S|}$ === What is $P(A^c)$ in terms of $P(A)$? --- $$P(A^c) = 1 - P(A)$$ Follows from the axioms: $A$ and $A^c$ are disjoint, and $A \cup A^c = S$, so $P(A) + P(A^c) = P(S) = 1$. === Define conditional probability $P(A|B)$ --- The probability of $A$ given $B$ (when $P(B) > 0$): $$P(A|B) = \frac{P(A \cap B)}{P(B)}$$ In the equally-likely case: $$P(A|B) = \frac{|A \cap B|}{|B|}$$ === State the chain rule of probability for $n$ events --- $$P(A_1 \cap A_2 \cap \cdots \cap A_n) = P(A_1)\,P(A_2|A_1)\,P(A_3|A_1,A_2)\cdots P(A_n|A_1,A_2,\dots,A_{n-1})$$ Each factor conditions on all previously listed events. === What conditions must hold for three events $A$, $B$, $C$ to be independent? --- All four conditions must hold: 1. $P(A \cap B) = P(A)P(B)$ 2. $P(A \cap C) = P(A)P(C)$ 3. $P(B \cap C) = P(B)P(C)$ 4. $P(A \cap B \cap C) = P(A)P(B)P(C)$ Pairwise independence alone is **not** sufficient. === What does it mean for $n$ events $A_1, A_2, \dots, A_n$ to be independent? --- Every subset of the events must satisfy the product rule. That is, for all $i < j < k < \dots$: $$P(A_i \cap A_j) = P(A_i)P(A_j)$$ $$P(A_i \cap A_j \cap A_k) = P(A_i)P(A_j)P(A_k)$$ $$\vdots$$ $$P(A_1 \cap A_2 \cap \cdots \cap A_n) = \prod_{i=1}^n P(A_i)$$ === If $A_1, A_2, \dots, A_n$ are independent, what is $P(A_1 \cup A_2 \cup \cdots \cup A_n)$? --- $$P(A_1 \cup A_2 \cup \cdots \cup A_n) = 1 - \prod_{i=1}^n (1 - P(A_i))$$ Derived by taking the complement (none of the events occur) and using independence. === What is the difference between disjoint and independent? --- - Disjoint: $A$ and $B$ cannot occur at the same time. $A \cap B = \empty$ - Independent: $A$ does not give any information about $B$ === State the Law of Total Probability using event $B$ and its complement --- $$P(A) = P(A|B)\,P(B) + P(A|B^c)\,P(B^c)$$ === State the general Law of Total Probability for a partition of the sample space --- If $B_1, B_2, B_3, \dots$ is a partition of $S$, then for any event $A$: $$P(A) = \sum_i P(A \cap B_i) = \sum_i P(A|B_i)\,P(B_i)$$ === State Bayes' Rule for two events $A$ and $B$ --- For $P(A) \neq 0$: $$P(B|A) = \frac{P(A|B)\,P(B)}{P(A)}$$ === State Bayes' Rule when $B_1, B_2, \dots$ form a partition of $S$ --- For any event $A$ with $P(A) \neq 0$: $$P(B_j|A) = \frac{P(A|B_j)\,P(B_j)}{\displaystyle\sum_i P(A|B_i)\,P(B_i)}$$ The denominator expands $P(A)$ via the Law of Total Probability. === State the meaning of conditionally independent between events $A$ and $B$ given event $C$. Note $P(C) \gt 0$. --- $$P(A \cap B \mid C) = P(A \mid C)P(B \mid C)$$ === State the multiplication rule for $P(A \cap B)$ --- $$P(A \cap B) = P(A|B)\,P(B) = P(B|A)\,P(A)$$ Rearrangement of the definition of conditional probability.