4 Joint Distributions

\(\mathbb{P}(x \in A, y \in B) = \int_A \int_B f(x,y) dy dx\)

\(\mathbb{E}[XY] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} xy f(x,y) dx dy\)

4.1 Marginals

\(f_X = \int f(x,y) dy\)

\(f_Y = \int f(x,y) dx\)

\(\mathbb{P}(x \in A) = \mathbb{P}(x \in A, y \in (-\infty, \infty)) = \int_A \int_{-\infty}^{\infty} f(x,y) dy dx\)

\(\mathbb{P}(y \in B) = \mathbb{P}(x \in (-\infty, \infty), y \in B) = \int_{-\infty}^{\infty} \int_B f(x,y) dy dx\)

4.2 Independence

\(f(x,y)=f_X(x)f_Y(y) \quad \forall x,y\)

\(F(x,y)=F_X(x)F_Y(y) \quad \forall x,y\)

4.2.1 Minimum & Maximum

Max: \(F_{\text{Max}}(t)=\mathbb{P}(\text{Max} \leq t) = \mathbb{P}(x \leq t, y \leq t) \rightarrow\) use independence \(\rightarrow = F_X(t)F_Y(t)\)

Min: \(1-F_{\text{Max}}\)

4.3 Sums of Independent Random Variables

4.3.1 Distributions

Convolution (CDF): \(F_{X+Y}(a) = \mathbb{P}(X+Y \leq a) = \int_{-\infty}^\infty F_X (a-y) f_Y (y) dy\)

Density (PDF): \(f_{X+Y} = \int_{-\infty}^\infty f_X (a-y) f_Y (y) dy\)

4.3.2 Uniform

4.3.3 Normal

The sum of \(n\) normal RVs \(\sum_i^n X_i\) is normally distributed with parameters:

\[\begin{aligned} \mu &= \sum_i^n \mu_i \\ \sigma^2 &= \sum_i^n \sigma_i^2 \\ \sigma &= \sqrt{\sum_i^n \sigma_i^2} \neq \sum_i^n \sqrt{\sigma_i^2} \end{aligned}\]

4.3.4 Gamma

4.3.5 Poisson

\(X_1\sim\text{Poisson}(\lambda_1)\)

\(X_2\sim\text{Poisson}(\lambda_2)\)

\(Y = X_1+Y_2\)

\(Y\sim\text{Poisson}(\lambda= \lambda_1 + \lambda_2)\)

\(\mathbb{P}(X_1+X_2=n) = \frac{1}{n!} e^{-(\lambda_1+\lambda_2)} (\lambda_1 + \lambda_2)^n\)

4.3.6 Binomial

\(X_1\sim\text{Binom}(n,p)\)

\(X_2\sim\text{Binom}(m,p)\)

\(Y = X_1+Y_2\)

\(Y\sim\text{Binom}(n+m,p)\)

\(\mathbb{P}(X_1+X_2=k) = \binom{n+m}{k} = \sum_{i=0}^n \binom{n}{i} \binom{m}{k-i}\)

4.3.7 Geometric

4.4 Conditional Joint Distributions

4.4.1 Discrete

\(P_{X|Y} = \frac{P(x,y)}{P_Y(y)} = \mathbb{P}(X=x|Y=y)\)

\(\mathbb{E}[X|Y=y] = \displaystyle\sum_x x P_{X|Y}(x|y)\)

4.4.2 Continuous

\(f_{X|Y} = \frac{f(x,y)}{f_Y(y)}\)

\(\mathbb{E}[X|Y=y] = \displaystyle\int_{-\infty}^\infty x f_{X|Y} (x|y) dx\)

\(F_{X|Y}(a,y) = \mathbb{P}(X \leq a | Y=y) = \displaystyle\int_{-\infty}^a f_{X|Y}(x|y) dx\)

4.4.3 Bayes’ Theorem (Continuous)

\(f_{X|Y} = \frac{f_{Y|X}(y|x)f_x(x)}{f_Y(y)} = \frac{f_{Y|X}(y|x)f_x(x)}{\int f_{Y|X}(y|x)f_x(x)dx}\)

4.5 Order Statistics

4.6 Transformations of Joint Distributions

4.6.1 The Jacobian

\((u,v) = G(x,y)\)

\(\text{Jac}(x,y) = \text{det} \begin{bmatrix} \frac{\partial u}{\partial x} & \frac{\partial u}{\partial y} \\ \frac{\partial v}{\partial x} & \frac{\partial v}{\partial y} \end{bmatrix} = \frac{\partial u}{\partial x} \frac{\partial v}{\partial y} - \frac{\partial u}{\partial y} \frac{\partial v}{\partial x}\)

\(f_{u,v}(u,v) = \frac{1}{|\text{Jac}(x,y)|} f_{x,y}(x,y)\)