3 Distributions of Random Variables
3.1 Discrete
CDF: \(F(k) = \displaystyle\sum_{k=0}^i p(k)\)
3.1.1 Bernoulli
\(X\sim\text{Bern}(p)\)
\(\mathbb{E}[X] = p\)
\(\text{Var}[X] = p(1-p)\)
\[\begin{equation}\begin{aligned} p(x) = \begin{cases} p & x=1 \\ 1-p & x=0 \\ 0 & else \end{cases}\end{aligned}\end{equation}\]
3.1.2 Binomial
\(X\sim\text{Binom}(n,p)\)
\(\mathbb{E}[X] = np\)
\(\text{Var}[X] = np(1-p)\)
\[p(k) = \binom{n}{k} p^k (1-p)^{n-k}\]
3.1.3 Poisson
\(X\sim\text{Poisson}(\lambda\sim np)\)
\(\mathbb{E}[X] = \lambda\)
\(\text{Var}[X] = \lambda\)
\[p(k) = e^{-\lambda} \frac{\lambda^k}{k!}\]
- Approximation to binomial when \(n \rightarrow \infty\) and \(p \rightarrow 0\).
- E.g. number of misprints per page of a book.
3.1.4 Geometric
\(X\sim\text{Geom}(p)\)
\(\mathbb{E}[X] = \frac{1}{p}\)
\(\text{Var}[X] = \frac{1-p}{p^2}\)
\[\begin{equation}\begin{aligned} p(k) &= (1-p)^{k-1} \\ \\ F(k) &= 1-(1-p)^k \\ \end{aligned}\end{equation}\]
- Experiment with infinite trials; stop at first success.
- Memoryless.
- E.g. flip a coin until heads comes up.
3.1.5 Negative Binomial
TO-DO
3.1.6 Hypergeometric
TO-DO
3.2 Continuous
3.2.1 Uniform
\(X\sim\text{Unif}(a,b)\)
\(\mathbb{E}[X] = \frac{b+a}{2}\)
\(\text{Var}[X] = \frac{(b-a)^2}{12}\)
\[\begin{equation}\begin{aligned} f(x) &= \begin{cases} \frac{1}{b-a} & x \in [a,b] \\ 0 & \text{else} \\ \end{cases} \\ \\ F(x) &= \begin{cases} \frac{x-a}{b-a} & x \in [a,b] \\ 0 & \text{else} \end{cases}\end{aligned}\end{equation}\]
- Simplest continuous distribution.
- All outcomes equally likely.
- E.g. uniformly pick random point on disk of radius \(r\). \(x\) is distance to center (not Uniform). \(f(x) = \frac{2x}{r^2}\) when \(0 \leq x \leq r\).
3.2.2 General Normal
\(X\sim\text{N}(\mu,\sigma)\)
\(\mathbb{E}[X] = \mu\)
\(\text{Var}[X] = \sigma^2\)
\[\begin{equation}\begin{aligned} f(x) &= \frac{1}{\sqrt{2\pi}\sigma} e^{-(x-\mu)^2/2\sigma^2} \\ \\ F(x) &= \phi(Z=\frac{x-\mu}{\sigma}) \end{aligned}\end{equation}\]
- To find CDF, convert to standard normal, then use Z table.
- E.g. biological variables.
- E.g. exam scores.
3.2.3 Standard Normal
\(X\sim\text{N}(0,1)\)
\(\mathbb{E}[X] = 0\)
\(\text{Var}[X] = 1\)
\[\begin{equation}\begin{aligned} f(x) &= \frac{1}{\sqrt{2\pi}} e^{-(x^2)/2} \\ \\ F(x) &= \phi(x) \end{aligned}\end{equation}\]
- To find CDF, use Z table.
- Special case of the normal with no parameters.
3.2.3.1 Normal Approximation to the Binomial Distribution
When \(X\sim\text{Binom}(n,p)\), \(n \rightarrow \infty\), & \(p \rightarrow \frac{1}{2}\):
\(\mathbb{E}[X] = np = \mu\), \(\sigma = \sqrt{np(1-p)}\), \(z = \frac{x-np}{\sqrt{np(1-p)}}\)
\(F_z(a) \rightarrow \phi (a)\)
\(\mathbb{P}(a \leq z \leq b) \approx \phi (b) - \phi (a)\)
via De Maivre-Laplace Central Limit Theorem
3.2.4 Exponential
\(X\sim\text{Exp}(\lambda)\)
\(\mathbb{E}[X] = \frac{1}{\lambda}\)
\(\text{Var}[X] = \frac{1}{\lambda^2}\)
\[\begin{equation}\begin{aligned} f(x) &= \begin{cases} \lambda e^{-\lambda x} & x > 0 \\ 0 & else \\ \end{cases} \\ \\ F(x) &= \begin{cases} 1-e^{-\lambda x} & x > 0 \\ 0 & else \end{cases} \end{aligned}\end{equation}\]
- Memoryless.
- \(\lambda=\) rate.
- Continuous version of Geom(\(p\)).
3.2.4.1 Hazard & Survival
Survival: \(S_T(t) = \mathbb{P}(T>t) = 1 - \mathbb{P}(T \leq t) = 1 - F_T(t) = e^{-\int_{u=o}^t \lambda(u) du}\)
Density: \(f_T(t) = F_T'(t) = -S_T'(t)\)
Hazard: \(\lambda(t) = \frac{f_T(t)}{S_T(t)} = \frac{-S_T'(t)}{S_T(t)} = -\frac{d}{dt} \log S_T(t)\)
3.2.5 Gamma
\(X\sim\Gamma[\alpha,\lambda]\)
\(\mathbb{E}[X] = \frac{\alpha}{\lambda}\)
\(\text{Var}[X] = \frac{\alpha}{\lambda^2}\)
3.2.6 Chi Square
\(X\sim\chi^2[n]\)
\(\mathbb{E}[X] = n\)
\(\text{Var}[X] = 2n\)
- Special case of \(\Gamma\) where \(\alpha=\frac{n}{2}\) and \(\lambda=\frac{1}{2}\).
3.3 Properties
3.3.1 Density Functions
PMF: \(p(k) \quad\) PDF: \(f(x)\)
- Derivative of the distribution function.
- Nonnegative everywhere.
- Integral over its domain is \(1\): \(\int_a^b f(x)=1\)
3.3.2 Distribution Functions
CDF: \(F(x)\)
- \(\lim_{x\rightarrow -\infty} F(x) = 0\)
- \(\lim_{x\rightarrow +\infty} F(x) = 1\)
- Nondecreasing.
3.3.3 Parameters
\[\begin{equation}\begin{aligned} \text{Law of total expectation:} \quad \mathbb{E}[X] &= \sum_j \mathbb{E}(E|F_j)\mathbb{P}(F_j)\\ \\ \text{Discrete:} \quad \mathbb{E}[X] &= \mu = \sum_{i=1}^k x_i p_i \\ \\ \text{Continuous:} \quad \mathbb{E}[X] &= \mu = \int_{-\infty}^{\infty} x f(x) dx \\ \\ \text{Var}[X] &= \mathbb{E}(X^2) - \mathbb{E}(X)^2 = \sigma^2 \\ \\ \sigma &= \sqrt{\text{Var}[X]} \\ \\ Z &= \frac{x-\mu}{\sigma}, \quad Z\sim\text{N}(0,1) \end{aligned}\end{equation}\]
3.4 Distributions of Functions
\(X\) is a random variable. \(Y=g(x)\) is a function of \(X\).
3.4.1 Transformation Method
If \(Y=g(x)\) is monotonic:
\(f_Y(y)=\frac{1}{|g'(x)|}f_X(x)\)
Derive \(g'(x)\) from \(g(x)\). Integrate \(f_Y\) to find \(F_Y\).
Note: monotonic = invertible = one-to-one.
3.4.2 CDF Method
Must use when \(Y=g(x)\) is not monotonic:
\(F_Y(y)=\mathbb{P}(Y \leq y) = \mathbb{P}(g(x) \leq y) \rightarrow\) solve for \(x\) and frame in terms of \(F_X(y)\).
Differentiate \(F_Y\) to find \(f_Y\).