study/reading/000_Introduction to Probability, Statistics, and Random Processes/ch1/flashcards/definitions.md

State De Morgan's Law for n sets

For any sets A_1, A_2, \dots, A_n:

\left(\bigcup_{i=1}^n A_i\right)^c = \bigcap_{i=1}^n A_i^c \left(\bigcap_{i=1}^n A_i\right)^c = \bigcup_{i=1}^n A_i^c

The complement of a union is the intersection of complements, and vice versa.

===

State the Distributive Law for sets A, B, and C

A \cap (B \cup C) = (A \cap B) \cup (A \cap C) A \cup (B \cap C) = (A \cup B) \cap (A \cup C)

===

State the inclusion-exclusion principle for a finite collection of sets A_1, A_2, A_3, \dots A_n where n=2

---

n = 2 case:

|A \cup B| = |A| + |B| - |A \cap B|

---

n = 3 case:

|A \cup B \cup C| = |A| + |B| + |C| - |A \cap B| - |A \cap C| - |B \cap C| + |A \cap B \cap C|

---

For a finite collection of sets A_1, A_2, A_3, \dots A_n, we have

\left| \bigcup_{i=1}^n A_i \right| = \sum_{i=1}^n |A_i| - \sum_{i < j} |A_i \cap A_j| + \sum_{i < j < k} |A_i \cap A_j \cap A_k| - \dots + (-1)^{n-1} |A_1 \cap A_2 \cap \dots \cap A_n|

===

For the function

f: A \to B

State the following of f:

1. Domain
2. Co-domain
3. Range

---

1. A
2. B
3. Set of all possible outputs of f (not necessarily B)

===

State definition for each the following:

1. Random experiment
2. Outcome
3. Sample space
4. Event

---

1. A random experiment is a process by which we observe something uncertain
2. An outcome is a result of a random experiment
3. The sample space S is the set of all possible outcomes
4. An event is a subset of the sample space

===

State the Axioms of Probability

---

Axioms of Probability

1. For any event A, P(A) \geq 0
2. P(S) = 1
3. If A_1, A_2, A_3, \dots are disjoint events, then P(A_1 \cup A_2 \cup A_3 \cup \dots) = P(A_1) + P(A_2) + P(A_3) + \dots

===

In a finite sample space S, where all outcomes are equally likely, what is the probability of any event A?

---

P(A) = \frac{|A|}{|S|}

===

What is P(A^c) in terms of P(A)?

P(A^c) = 1 - P(A)

Follows from the axioms: A and A^c are disjoint, and A \cup A^c = S, so P(A) + P(A^c) = P(S) = 1.

===

Define conditional probability P(A|B)

The probability of A given B (when P(B) > 0):

P(A|B) = \frac{P(A \cap B)}{P(B)}

In the equally-likely case:

P(A|B) = \frac{|A \cap B|}{|B|}

===

State the chain rule of probability for n events

P(A_1 \cap A_2 \cap \cdots \cap A_n) = P(A_1)\,P(A_2|A_1)\,P(A_3|A_1,A_2)\cdots P(A_n|A_1,A_2,\dots,A_{n-1})

Each factor conditions on all previously listed events.

===

What conditions must hold for three events A, B, C to be independent?

All four conditions must hold:

1. P(A \cap B) = P(A)P(B)
2. P(A \cap C) = P(A)P(C)
3. P(B \cap C) = P(B)P(C)
4. P(A \cap B \cap C) = P(A)P(B)P(C)

Pairwise independence alone is not sufficient.

===

What does it mean for n events A_1, A_2, \dots, A_n to be independent?

Every subset of the events must satisfy the product rule. That is, for all i < j < k < \dots:

P(A_i \cap A_j) = P(A_i)P(A_j) P(A_i \cap A_j \cap A_k) = P(A_i)P(A_j)P(A_k) \vdots P(A_1 \cap A_2 \cap \cdots \cap A_n) = \prod_{i=1}^n P(A_i)

===

If A_1, A_2, \dots, A_n are independent, what is P(A_1 \cup A_2 \cup \cdots \cup A_n)?

P(A_1 \cup A_2 \cup \cdots \cup A_n) = 1 - \prod_{i=1}^n (1 - P(A_i))

Derived by taking the complement (none of the events occur) and using independence.

===

What is the difference between disjoint and independent?

---

• Disjoint: A and B cannot occur at the same time. A \cap B = \empty
• Independent: A does not give any information about B

===

State the Law of Total Probability using event B and its complement

P(A) = P(A|B)\,P(B) + P(A|B^c)\,P(B^c)

===

State the general Law of Total Probability for a partition of the sample space

If B_1, B_2, B_3, \dots is a partition of S, then for any event A:

P(A) = \sum_i P(A \cap B_i) = \sum_i P(A|B_i)\,P(B_i)

===

State Bayes' Rule for two events A and B

For P(A) \neq 0:

P(B|A) = \frac{P(A|B)\,P(B)}{P(A)}

===

State Bayes' Rule when B_1, B_2, \dots form a partition of S

For any event A with P(A) \neq 0:

P(B_j|A) = \frac{P(A|B_j)\,P(B_j)}{\displaystyle\sum_i P(A|B_i)\,P(B_i)}

The denominator expands P(A) via the Law of Total Probability.

===

State the meaning of conditionally independent between events A and B given event C. Note P(C) \gt 0.

---

P(A \cap B \mid C) = P(A \mid C)P(B \mid C)

===

State the multiplication rule for P(A \cap B)

P(A \cap B) = P(A|B)\,P(B) = P(B|A)\,P(A)

Rearrangement of the definition of conditional probability.