Probability Cheatsheet

Cheatsheet Content

1. Basic Definitions Experiment: A process with an uncertain outcome. Outcome: A single result of an experiment. Sample Space ($S$ or $\Omega$): The set of all possible outcomes. Event ($E$): A subset of the sample space. Probability of an Event ($P(E)$): A number between 0 and 1 indicating the likelihood of an event. $0 \le P(E) \le 1$ $P(S) = 1$ (The probability of the entire sample space is 1) 2. Types of Events Mutually Exclusive (Disjoint) Events: Events that cannot occur at the same time. $E_1 \cap E_2 = \emptyset$. Independent Events: The occurrence of one event does not affect the probability of the other. Complementary Event ($E^c$ or $\bar{E}$): All outcomes in $S$ that are not in $E$. $P(E^c) = 1 - P(E)$. 3. Probability Rules 3.1. Addition Rule For any two events $A$ and $B$: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$ For mutually exclusive events $A$ and $B$: $P(A \cup B) = P(A) + P(B)$ 3.2. Multiplication Rule (Conditional Probability) Conditional Probability: $P(A|B) = \frac{P(A \cap B)}{P(B)}$, provided $P(B) > 0$. General Multiplication Rule: $P(A \cap B) = P(A|B)P(B) = P(B|A)P(A)$ For independent events $A$ and $B$: $P(A \cap B) = P(A)P(B)$ 4. Bayes' Theorem For events $A$ and $B$ where $P(B) > 0$: $$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$ If $B$ can be partitioned by $A$ and $A^c$ (or $A_i$ forming a partition of $S$): $$P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A) + P(B|A^c)P(A^c)}$$ 5. Counting Principles Permutations ($P_k^n$ or $_nP_k$): Number of ways to arrange $k$ items from a set of $n$ distinct items, where order matters. $$P_k^n = \frac{n!}{(n-k)!}$$ Combinations ($C_k^n$ or $_nC_k$ or $\binom{n}{k}$): Number of ways to choose $k$ items from a set of $n$ distinct items, where order does not matter. $$\binom{n}{k} = \frac{n!}{k!(n-k)!}$$ Factorial: $n! = n \times (n-1) \times \dots \times 2 \times 1$. $0! = 1$. 6. Random Variables Random Variable ($X$): A function that assigns a numerical value to each outcome in the sample space. Discrete Random Variable: Takes on a finite or countably infinite number of values. Continuous Random Variable: Takes on any value within a given range. 7. Probability Distributions 7.1. Discrete Probability Distributions Probability Mass Function (PMF): $P(X=x)$ for discrete RVs. $\sum P(X=x) = 1$. Expected Value ($E[X]$): Mean of a discrete RV. $E[X] = \sum x \cdot P(X=x)$. Variance ($Var(X)$): $Var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$. Standard Deviation ($\sigma_X$): $\sigma_X = \sqrt{Var(X)}$. Common Discrete Distributions: Bernoulli: $X \sim Ber(p)$, $P(X=1)=p$, $P(X=0)=1-p$. $E[X]=p$, $Var(X)=p(1-p)$. Binomial: $X \sim B(n, p)$, number of successes in $n$ independent Bernoulli trials. $$P(X=k) = \binom{n}{k}p^k(1-p)^{n-k}$$ $E[X]=np$, $Var(X)=np(1-p)$. Poisson: $X \sim Pois(\lambda)$, number of events in a fixed interval. $$P(X=k) = \frac{e^{-\lambda}\lambda^k}{k!}$$ $E[X]=\lambda$, $Var(X)=\lambda$. Geometric: $X \sim Geom(p)$, number of trials until the first success. $$P(X=k) = (1-p)^{k-1}p$$ $E[X]=\frac{1}{p}$, $Var(X)=\frac{1-p}{p^2}$. 7.2. Continuous Probability Distributions Probability Density Function (PDF): $f(x)$ for continuous RVs. $\int_{-\infty}^{\infty} f(x) dx = 1$. $P(a \le X \le b) = \int_a^b f(x) dx$. Expected Value ($E[X]$): $E[X] = \int_{-\infty}^{\infty} x \cdot f(x) dx$. Variance ($Var(X)$): $Var(X) = \int_{-\infty}^{\infty} (x - E[X])^2 f(x) dx = E[X^2] - (E[X])^2$. Common Continuous Distributions: Uniform: $X \sim U(a, b)$, $f(x) = \frac{1}{b-a}$ for $a \le x \le b$, 0 otherwise. $E[X]=\frac{a+b}{2}$, $Var(X)=\frac{(b-a)^2}{12}$. Exponential: $X \sim Exp(\lambda)$, waiting time until an event. $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$. $E[X]=\frac{1}{\lambda}$, $Var(X)=\frac{1}{\lambda^2}$. Normal (Gaussian): $X \sim N(\mu, \sigma^2)$. $f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$. $E[X]=\mu$, $Var(X)=\sigma^2$. Standard Normal: $Z \sim N(0, 1)$. $Z = \frac{X-\mu}{\sigma}$. 8. Cumulative Distribution Function (CDF) For any random variable $X$: $F(x) = P(X \le x)$. For discrete RV: $F(x) = \sum_{t \le x} P(X=t)$. For continuous RV: $F(x) = \int_{-\infty}^{x} f(t) dt$. Properties: $0 \le F(x) \le 1$ $F(x)$ is non-decreasing. $\lim_{x \to -\infty} F(x) = 0$, $\lim_{x \to \infty} F(x) = 1$. 9. Joint & Marginal Distributions Joint PMF/PDF: $P(X=x, Y=y)$ or $f(x, y)$. Marginal PMF: $P(X=x) = \sum_y P(X=x, Y=y)$. Marginal PDF: $f_X(x) = \int_{-\infty}^{\infty} f(x, y) dy$. Conditional PMF/PDF: $P(Y=y|X=x) = \frac{P(X=x, Y=y)}{P(X=x)}$. $f(y|x) = \frac{f(x, y)}{f_X(x)}$. Independence: $X, Y$ are independent if $P(X=x, Y=y) = P(X=x)P(Y=y)$ (discrete) or $f(x, y) = f_X(x)f_Y(y)$ (continuous). 10. Covariance and Correlation Covariance: Measures the linear relationship between two RVs. $Cov(X, Y) = E[(X - E[X])(Y - E[Y])] = E[XY] - E[X]E[Y]$. Correlation Coefficient ($\rho_{XY}$): Normalized covariance, $-1 \le \rho_{XY} \le 1$. $$\rho_{XY} = \frac{Cov(X, Y)}{\sigma_X \sigma_Y}$$ If $X, Y$ are independent, then $Cov(X, Y) = 0$ and $\rho_{XY} = 0$. (The converse is not always true). 11. Laws of Large Numbers & Central Limit Theorem Law of Large Numbers (LLN): As the number of trials increases, the sample mean converges to the expected value. Central Limit Theorem (CLT): For a large sample size $n$, the distribution of the sample mean ($\bar{X}$) of a random variable (from any distribution with finite mean $\mu$ and variance $\sigma^2$) will be approximately normal: $$\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right)$$ The standardized sample mean $Z = \frac{\bar{X} - \mu}{\sigma/\sqrt{n}}$ approaches a standard normal distribution.