import numpy as np
from numpy.random import default_rng
rng = default_rng()
from scipy.stats import norm
from matplotlib import pyplot as plt
import plotly.graph_objects as go

1

Show that

$$P(X = x) = F(x ^ +) - F(x ^ -)$$

Solution:

Since $F$ is right continuous, $F(x ^ +) = F(x)$

\begin{align*} F(x ^ +) - F(x ^ -) &= F(x) - F(x ^ -)\\ &= F(x) - \lim_{y \to x, y < x} F(y) \\ &= \lim_{y \to x, y < x} [ F(x) - F(y) ] \\ &= \lim_{y \to x, y < x} P(X \le x) - P(X \le y) \\ &= \lim_{y \to x, y < x} P(y < X \le x) \tag{$\{t \mid t \le y\} \subseteq \{t \mid t \le x\}$}\\ &= P(X = x) \end{align*}

2

Let $X$ be such that $P(X = 2) = P(X = 3) = 1 / 10$ and $P(X = 5) = 8 / 10$. Plot the CDF $F$. Use $F$ to find $P(2 < X \le 4.8)$ and $P(2 \le X \le 4.8)$

Solution:

$$ F(x) = \begin{cases} 0 & x < 2 \\ 1 / 10 & 2 \le x < 3 \\ 2 / 10 & 3 \le x < 5 \\ 1 & 5 \le x \end{cases} $$$$P(2 < X \le 4.8) = F(4.8) - F(2) = 2 / 10 - 1 /10 = 1 / 10$$$$ P(2 \le X \le 4.8) = F(4.8) - \lim_{y\to 2, y < 2} F(y) = 2 / 10 - 0 = 2 / 10$$

def F(x):
    if x < 2:
        return 0
    elif x < 3:
        return 1 / 10
    elif x < 5:
        return 2 / 10
    else:
        return 1

xx = np.arange(0, 10, 0.01)
plt.scatter(xx, [F(x) for x in xx], s=0.5)
plt.grid()
plt.xlabel('x')
plt.ylabel('F(x)', rotation=0)
plt.show()

3

Prove Lemma 2.15:

Let $F$ be the CDF for a random variable $X$. Then:

1. $P(X = x) = F(x) - F(x ^ -)$ where $F(x ^ -) = \lim_{y \uparrow x} F(y)$

See Problem #1.

2. $P(x < X \le y) = F(y) - F(x)$

Assume $x \le y$. Then, $\{t \mid t \le x\} \subseteq \{t \mid t \le y\}$, so:

\begin{align*} P(x < X \le y) &= P(X \le y \cap X > x) \\ &= P(X \le y \cap \overline{X \le x}) \\ &= P((X \le y) - (X \le x)) \\ &= P(X \le y) - P(X \le x) \\ &= F(y) - F(x) \end{align*}

3. $P(X > x) = 1 - F(x)$

$$P(X > x) = 1 - P(X \le x) = 1 - F(x)$$

4. If $X$ is continuous then

\begin{align*} F(b) - F(a) &= P(a < X < b) = P(a \le X < b)\\ &= P(a < X \le b) = P(a \le X \le b) \end{align*}

Note that if $X$ is continuous, then $P(X = x) = 0$ for all $x$.

\begin{align*} F(b) - F(a) &= P(a < X \le b) \tag{from 2} \\ &= P(a < X \le b) + P(X = a) \\ &= P(a < X \le b \cup X = a) \tag{disjoint events} \\ &= P(a \le X \le b) \end{align*}

Proofs for $P(a < X < b)$ and $P(a \le X < b)$ follow similar lines.

4

Let $X$ have probability density function

$$ f_X(x) = \begin{cases} 1 / 4 & 0 < x < 1 \\ 3 / 8 & 3 < x < 5 \\ 0 & \text{otherwise} \\ \end{cases} $$

(a) Find the cumulative distribution function of $X$.

$$F(x) = \int_{-\infty}^x f_X(t) \, dt$$

If $x \le 0$, $F(x) = \int_{-\infty}^x 0 \, dt = 0$.

If $0 < x \le 1$, $F(x) = \int_{0}^x 1 / 4 \, dt = x / 4$.

If $1 < x \le 3$, $F(x) = \int_{0}^1 1 / 4 \, dt + \int_{1}^x 0 \, dt = 1 / 4$

If $3 < x \le 5$, $F(x) = \int_{0}^1 1 / 4 \, dt + \int_{1}^3 0 \, dt + \int_3^x 3 / 8 \, dt = 3 x / 8 - 7 / 8$

If $5 < x$, $F(x) = \int_{0}^1 1 / 4 \, dt + \int_{1}^3 0 \, dt + \int_3^5 3 / 8 \, dt + \int_5^x 0 \, dt = 1$

So

$$ F(x) = \begin{cases} 0 & x \le 0\\ x / 4 & 0 < x \le 1\\ 1 / 4 & 1 < x \le 3\\ 3 x / 8 - 7 / 8 & 3 < x \le 5\\ 1 & 5 < x\\ \end{cases} $$

(b) Let $Y = 1 / X$. Find the probability density function $f_Y(y)$ for $Y$. Hint: Consider three cases: $\frac15 \le y \le \frac13$, $\frac13 \le y \le 1$, and $y \ge 1$.

If $y \le \frac15$, $P(Y \le y) = 0$

If $\frac15 \le y \le \frac13$, $5 \ge x \ge 3$, and $P(Y \le y) = P(X \ge \frac{1}{y}) = 1 - P(X \le \frac{1}{y}) = 15 / 8 - 3 / (8 y)$

If $\frac13 \le y \le 1$, $3 \ge x \ge 1$, and $P(Y \le y) = 1 - P(X \le \frac{1}{y}) = 1 - \frac{1}{4} = \frac34$

If $y \ge 1$, $x \le 1$, and $P(Y \le y) = 1 - P(X \le \frac{1}{y}) = 1 - \frac{1}{4y}$

So $$ F_Y(y) = \begin{cases} 0 & y \le \frac15\\ \frac{15}{8} - \frac{3}{8y} & \frac15 < y \le \frac13\\ \frac34 & \frac13 < y \le 1\\ 1 - \frac{1}{4y} & 1 < y\\ \end{cases} $$

and $f_Y(y) = F_Y'(y)$:

$$ f_Y(y) = \begin{cases} 0 & y \le \frac15\\ \frac{3}{8y^2} & \frac15 < y \le \frac13\\ 0 & \frac13 < y \le 1\\ \frac{1}{4y^2} & 1 < y\\ \end{cases} $$

def F(x):
    if x <= 0:
        return 0
    elif x <= 1:
        return x / 4
    elif x <= 3:
        return 1 / 4
    elif x <= 5:
        return 3 * x / 8 - 7 / 8
    else:
        return 1

xx = np.arange(0, 10, 0.01)
plt.scatter(xx, [F(x) for x in xx], s=0.5)
plt.grid()
plt.xlabel('x')
plt.ylabel('F(x)', rotation=0)
plt.show()

def F_Y(y):
    if y <= 1 / 5:
        return 0
    elif y <= 1 / 3:
        return 15 / 8 - 3 / (8 * y)
    elif y <= 1:
        return 3 / 4
    else:
        return 1 - 1 / (4 * y)

yy = np.arange(0, 3, 0.001)
plt.scatter(yy, [F_Y(y) for y in yy], s=0.5)
plt.grid()
plt.xlabel('y')
plt.ylabel('F_Y(y)', rotation=0)
plt.show()

5

Let $X$ and $Y$ be discrete random variables. Show that $X$ and $Y$ are independent if and only if $f_{X,Y}(x,y) = f_X(x)f_Y(y)$ for all $x$ and $y$.

Solution:

$\Rightarrow$: Suppose $X$ and $Y$ are independent. Then,

\begin{align*} f_{X,Y}(x,y) &= P(X = x, Y = y) \tag{Definition}\\ &= P(X = x) P(Y = y) \tag{Assumption}\\ &= f_X(x) f_Y(y) \tag{Definition} \end{align*}

$\Leftarrow$: Suppose $f_{X,Y}(x,y) = f_X(x)f_Y(y)$ for all $x$ and $y$. Then,

\begin{align*} P(X \in A, Y \in B) &= \sum_{x \in A, y \in B} f_{X,Y}(x,y) \tag{Definition}\\ &= \sum_{x \in A, y \in B} f_{X}(x)f_Y(y) \tag{Assumption}\\ &= \sum_{x \in A} f_X(x) \sum_{y \in B} f_Y(y) \tag{Factoring}\\ &= P(X \in A)P(Y \in B) \tag{Definition} \end{align*}

6

Let $X$ have distribution $F$ and density function $f$ and let $A$ be a subset of the real line. Let $I_A(x)$ be the indicator function for $A$:

$$ I_A(x) = \begin{cases} 1 & x \in A \\ 0 & x \notin A.\\ \end{cases} $$

Let $Y = I_A(X)$. Find an expression for the cumulative distribution of $Y$. (Hint: first find the probability mass function for $Y$.)

Solution:

Observe that $Y$ can only assume the values of $0$ or $1$. We have $P(Y = 1) = P(X \in A) = \int_A f(x) \, dx$, and thus $P(Y = 0) = 1 - \int_A f(x) \, dx$, meaning:

$$ f_Y(y) = \begin{cases} 1 - \int_A f(x) \, dx & y = 0 \\ \int_A f(x) \, dx & y = 1 \\ 0 & \text{otherwise} \end{cases} $$

and

$$ F_Y(y) = \begin{cases} 0 & y \le 0 \\ 1 - \int_A f(x) \, dx & 0 \le y \le 1 \\ 1 & 1 < y \\ \end{cases} $$

7

Let $X$ and $Y$ be independent and suppose that each has a Uniform(0,1) distribution. Let $Z = \min \{X , Y\}$. Find the density $f_Z(z)$ for $Z$. Hint: It might be easier to first find $P(Z > z)$.

Solution:

We have:

\begin{align*} P(Z > z) &= P(X > z, Y > z) \\ &= P(X > z)P(Y > z) \tag{Independence} \\ &= (1 - P(X \le z))(1 - P(Y \le z)) \end{align*}

thus: $P(Z \le z) = 1 - (1 - P(X \le z))(1 - P(Y \le z))$. Assuming $z \in [0,1]$,

\begin{align*} P(Z \le z) &= 1 - (1 - P(X \le z))(1 - P(Y \le z)) \\ &= 1 - (1 - \int_0^z dx)(1 - \int_0^z dy) \\ &= 1 - (1 - z)(1 - z) \\ &= 2z - z^2 \end{align*}

and $f_Z(z) = 2 - 2z$.

8

Let $X$ have CDF $F$. Find the CDF of $X^+ = \max \{0, X \}$.

Solution:

If $x \ge 0$, $P(X^+ \le x) = P(X \le x)$. If $x < 0$, $P(X^+ \le x) = 0$. So,

$$ F_{X^+}(x) = \begin{cases} 0 & x < 0 \\ F(x) & 0 \le x \\ \end{cases} $$

9

Let $X \sim \text{Exp}(\beta)$. Find $F(x)$ and $F^{-1}(q)$.

Solution:

If $x \ge 0$,

$F(x) = \int_{0}^x \frac{1}{\beta} e ^ {-t / \beta} \, dt = 1 - e ^ {-x / \beta}$

otherwise, $F(x) = 0$, since the exponential distribution only has support on the positive real numbers. So,

$$ F(x) = \begin{cases} 0 & x < 0 \\ 1 - e ^ {-x / \beta} & x \ge 0. \\ \end{cases} $$

Meanwhile, the quantile function, $F^{-1}(q)$ can be found by solving $q = F(x)$ for $x$ for $q \in [0,1]$, yielding $F^{-1}(q) = \log((1-q)^{-\beta})$.

10

Let $X$ and $Y$ be independent. Show that $g(X)$ is independent of $h(Y)$ where $g$ and $h$ are functions.

Solution:

Let $A$ and $B$ be two events. Let $g^{-1}(A) = \{x \mid g(x) \in A\}$, and $h^{-1}(B) = \{y \mid h(y) \in B\}$. We have

\begin{align*} P(g(X) \in A, h(Y) \in B) &= P(X \in g^{-1}(A), Y \in h^{-1}(B)) \\ &= P(X \in g^{-1}(A))P(Y \in h^{-1}(B)) \tag{Independence} \\ &= P(g(X) \in A)P(h(Y) \in B) \end{align*}

11

Suppose we toss a coin once and let $p$ be the probability of heads. Let $X$ denote the number of heads and let $Y$ denote the number of tails.

(a) Prove that $X$ and $Y$ are dependent.

Solution: Assume $p \notin \{0, 1\}$. Observe $P(X = 1, Y = 1) = 0 \ne p(1-p) = P(X=1)P(Y = 1)$. If $p \in \{0, 1\}$, they are independent.

(b) Let $N \sim \text{Poisson}(\lambda)$ and suppose we toss a coin $N$ times. Let $X$ and $Y$ be the number of heads and tails. Show that $X$ and $Y$ are independent.

Solution:

We have

\begin{align*} P(X = x, Y = y) &= P(N = x + y, X = x) \\ &= e^{-\lambda} \frac{\lambda^{x + y}}{(x + y)!}{x + y \choose x}p^x(1-p)^y \\ &= e^{-\lambda} \frac{\lambda^{x + y}}{x!y!}p^x(1-p)^y \\ &= e^{-\lambda \cdot p + -\lambda \cdot (1-p)} \frac{(\lambda \cdot p)^x}{x!} \frac{(\lambda \cdot (1-p))^y}{y!} \\ &= e^{-\lambda \cdot p} \frac{(\lambda \cdot p)^x}{x!} \cdot e^{-\lambda \cdot (1-p)} \frac{(\lambda \cdot (1-p))^y}{y!} \end{align*}

Since the joint density $P(X = x, Y = y)$ can be expressed in the form $g(x)h(y)$, by Theorem 2.33, $X$ and $Y$ are independent.

12

Prove Theorem 2.33.

Suppose that the range of $X$ and $Y$ is a (possibly infinite) rectangle. If $f(x,y) = g(x)h(y)$ for some functions $g$ and $h$ (not necessarily probability density functions), then $X$ and $Y$ are independent.

Solution:

13

Let $X \sim N(0, 1)$, and let $Y = e^X$.

(a) Find the PDF for Y. Plot it.

We have \begin{align*} F_Y(y) &= P(Y \le y) \\ &= P(e^X \le y)\\ &= P(x \le \log(y))\\ & = \Phi(\log(y)) \\ \end{align*} Thus, $f_Y(y) = F_Y(y)' = \frac{d}{dy} \Phi(\log(y)) = \frac{1}{y} \cdot \phi(\log(y))$

(b) (Computer Experiment.) Generate a vector $x = (x_1, \dots, x_{10,000})$ consisting of 10,000 random standard Normals. Let $y = (y_1, \dots, y_{10,000})$, where $y_i = e^{x_i}$. Draw a histogram of $y$ and compare it to the PDF you found in part (a).

n = 10000
x = np.random.normal(loc=0, scale=1, size=n)
y = np.exp(x)
fig = go.Figure(data=go.Histogram(x=y, 
                                  opacity=0.5, 
                                  histnorm='probability density', 
                                  xbins=dict( # bins used for histogram
                                        start=0,
                                        end=10,
                                        size=0.05
                                  ),
                                  name='Histogram'))

xx = np.arange(0.1, 40, 0.1)
yy = norm.pdf(np.log(np.abs(xx))) / xx
fig.add_trace(go.Scatter(x=xx, y=yy, name='$ \\frac{1}{y} \cdot \phi(\log(y))$'))

fig.update_layout(
    legend=dict(
        yanchor="top",
        y=0.99,
        xanchor="right",
        x=0.99
    ),
    
)
fig.update_xaxes(range=[0, 10])
fig.show()

14

Let $(X, Y)$ be uniformly distributed on the unit disk $\{(x,y) : x ^ 2 + y ^ 2 \le 1 \}$. Let $R = \sqrt{X ^ 2 + Y ^ 2}$. Find the CDF and PDF of $R$.

Solution:

Let $0 \le r \le r$. Observe that $P(R \le r)$ is equal to the proportion of unit disk also within the disk of radius $r$ centered at 0, i.e., $\pi r ^ 2 / \pi = r^2$. Thus, the CDF, $F_R(r)$, is

$$ F_R(r) = \begin{cases} 0 & r < 0 \\ r^2 & 0 \le r < 1 \\ 1 & 1 \le r \\ \end{cases} $$

and the PDF, $f_R(r) = F_R'(r)$, is

$$ F_R(r) = \begin{cases} 0 & r < 0 \\ 2r & 0 \le r < 1 \\ 0 & 1 \le r \\ \end{cases} $$

15

(A universal random number generator.) Let $X$ have a continuous, strictly increasing CDF $F$. Let $Y = F(X)$. Find the density of $Y$. This is called the probability integral transform. Now let $U \sim \text{Uniform}(0,1)$ and let $X = F ^ {-1} (U)$. Show that $X \sim F$. Now write a program that takes $\text{Uniform}(0,1)$ random variables and generates random variables from an $\text{Exponential}(\beta)$ distribution.

Solution:

Let $X$ have a continuous, strictly increasing CDF $F$. Let $Y = F(X)$. Find the density of $Y$:

Since $F$ is continuous and strictly increasing, there is a continuous and strictly increasing $F^{-1}$ such that $F(F^{-1}(t)) = F^{-1}(F(t)) = t$ for all $t$. Observe that since $F(X) \in (0,1)$, if $y \le 0$, $P(Y \le y) = P(F(X) \le y) = 0$, and if $y \ge 1$, $P(Y \le y) = 1$. Now, if $0 < y < 1$, $P(Y \le y) = P(F(X) \le y) = P(X \le F^{-1}(y)) = F(F^{-1}(y)) = y$. Thus, $Y \sim \text{Uniform}(0,1)$.

Now let $U \sim \text{Uniform}(0,1)$ and let $X = F ^ {-1} (U)$. Show that $X \sim F$:

Let $F_U(u)$ be the CDF of $U$. Observe that for $u \in (0,1)$, $F_U(u) = u$, and that $F(x) \in (0,1)$ for all $x$. Therefore, $P(X \le x) = P(F ^ {-1} (U) \le x) = P(U \le F(x)) = F_U(F(x)) = F(x)$, meaning $X \sim F$.

n = 100000
beta = 2

u = rng.uniform(0, 1, n) 
x0 = np.log(np.power((1 - u), -1 * beta)) # from Problem 9, $F^{-1}(u) = \log((1-u)^{\beta})$.
x1 = rng.exponential(beta, n) # using NumPy

fig = go.Figure() 
fig.add_trace(go.Histogram(x=x0, opacity=0.5, name='Using Transform'))
fig.add_trace(go.Histogram(x=x1, opacity=0.5, name='Using NumPy')) 
fig.show()

16

Let $X \sim \text{Poisson}(\lambda)$ and $Y \sim \text{Poisson}(\mu)$ and assume that $X$ and $Y$ are independent. Show that the distribution of $X$ given that $X + Y = n$ is $\text{Binomial}(n, \pi)$ where $\pi = \lambda / (\lambda + \mu)$.

Solution:

We have $f_X(x) = e ^ {\lambda} \frac{\lambda ^ x}{x!}$ and $f_Y(y) = e ^ {\mu} \frac{\mu ^ y}{y!}$, and, due to independence, $f_{X+Y}(z) = e ^ {\lambda + \mu} \frac{(\lambda + \mu)^z}{z!}$. Thus,

\begin{align*} P(X = x \mid X + Y = n) &= \frac{P(X = x, X + Y = n)}{P(X + Y = n)} \\ &= \frac{P(X = x, Y = n - x)}{P(X + Y = n)} \\ &= \frac{P(X = x)P(Y = n - x)}{P(X + Y = n)} \tag{independence}\\ &= \frac{e ^ {\lambda} \frac{\lambda ^ x}{x!} e ^ {\mu} \frac{\mu ^{n-x}}{(n-x)!}}{e ^ {\lambda + \mu} \frac{(\lambda + \mu)^n}{n!}} \\ &= \frac{n!}{(n-x)!x!}\left(\frac{\lambda}{\lambda + \mu}\right)^x\left(\frac{\mu}{\lambda + \mu}\right)^{n-x} \\ &= {n \choose x}\pi^x(1-\pi)^{n-x} \end{align*}

17

Let

$$ f_{X,Y}(x,y) = \begin{cases} c(x + y ^ 2) & 0 \le x \le 1 \text{ and } 0 \le y \le 1 \\ 0 & \text{otherwise } \end{cases} $$

Find $P(X < \frac12 \mid Y = \frac12)$.

Solution:

Computing the marginal PDF, $f_Y(y)$:

\begin{align*} f_Y(y) &= \int_0^1 c(x + y ^ 2) \, dx \\ &= c \left( \frac12 + y ^ 2 \right) \end{align*}

Thus we have the conditional PDF $f_{X \mid Y}(x \mid y) = \frac{f_{X, Y} (x, y)}{f_Y(y)} = \frac{ x + y ^ 2}{ \frac12 + y ^ 2}$. Thus, $f_{X \mid Y}(x \mid y= \frac12) = \frac{4x + 1}{3}$

Computing the conditional probability $P(X \le \frac12 \mid Y = \frac12) = \int_0^{1/2} \frac{4x + 1}{3} \, dx = \frac13$.

18

Let $X \sim N(3, 16)$. Solve the following using the normal Table and using a computer package.

(a) Find P(X < 7).

Since $P(X = 7) = 0$, $P(X < 7) = P(X \le 7)$, and

$$P(X \le 7) = P((X - 3) / 4 \le 1) = P(Z \le 1) = \Phi(1) \approx 84.13\%$$

(b) Find P(X > -2)

$$P(X > -2) = 1 - P(X \le -2) = 1 - P(Z \le -5/4) = 1 - \Phi(-5/4) \approx 89.44\%$$

(c) Find $x$ such that $P(X > x) = 0.05$.

$$P(X > x) = 0.05 \Rightarrow P(Z \le (x-3)/4) = 0.95 \Rightarrow x = 4 \cdot \Phi^{-1}(0.95) + 3 \approx 9.58$$

(d) Find $P(0 \le X \le 4)$

$$P(0 \le X \le 4) = P(X \le 4) - P(X \le 0) = \Phi(\frac14) - \Phi(-\frac{3}{4}) \approx 37.21\%$$

(e) Find $x$ such that $P(|X| > |x|) = 0.05$

\begin{align*} P(|X| > |x|) &= 1 - P(|X| < |x|)\\ &=1 - P(-|x| \le X \le |x|) \\ &=1 - P\left(\frac{-|x|-3}{4} \le X \le \frac{|x| - 3}{4}\right)\\ &=1 - \Phi \left( \frac{|x| - 3}{4} \right) + \Phi \left( \frac{-|x|-3}{4} \right) \end{align*}

which equals 0.05 when $x \approx 9.611$.

x = np.arange(0, 20, 0.0001) 
y = 1 - norm.cdf((x - 3) / 4) + norm.cdf((-x - 3) / 4)
x[np.argmin(np.abs(y - 0.05))]

9.611

19

Prove formula (2.12):

Suppose $X$ is a random variable with PDF $f_X$ and CDF $F_X$. Let $Y = r(X)$. When $r$ is strictly monotone increasing or strictly monotone decreasing then $r$ has an inverse $s = r^{-1}$ and in this case one can show that

$$f_Y(y) = f_X(s(y)) \left| \frac{ds(y)}{dy} \right| \tag{2.12}$$

Solution:

Assume $r$ is strictly monotone increasing. Since $r$ is strictly monotone increasing, so is $s$. Thus, $\frac{ds(y)}{dy} > 0$. We have $P(Y \le y) = P(r(X) \le y) = P(X \le s(y)) = F_X(s(y))$. Thus, $f_Y(y) = F_Y'(y) = \frac{d}{dy} P(Y \le y) = \frac{d}{dy} F_X(s(y)) = f_X(s(y)) \cdot \frac{ds(y)}{dy}$.

Now assume $r$ is strictly monotone decreasing; now, so is $s$. Thus, $\frac{ds(y)}{dy} < 0$. Then, we follow the same steps as above, except that we observe $P(r(X) \le y) = P(X \ge s(y))$ (since $a \le b \Rightarrow s(a) \ge s(b)$), meaning $f_Y(y) = \frac{d}{dy} (1 - F_X(s(y))) = -f_X(s(y)) \cdot \frac{ds(y)}{dy}$.

The result then follows.

20

Let $X, Y \sim \text{Uniform}(0,1)$ be independent. Find the pdf for $X - Y$ and $X / Y$.

Let $Z = X - Y$. Note that the pdf of $Z$, $f(z)$, has support on [-1,1]. If $z < 0$, $P(Z \le z)$ is the area of the triangle connecting $(0, -z)$, $(-z,1)$, and $(0,1)$, $\frac12 (1+z)^2$. If $z \ge 0$, $P(Z \le z)$ is the area of the region connecting $(0,0)$, $(0, z)$, $(1, z)$, $(1,1)$, and $(0,1)$, $ \frac12 + z - z^2 / 2$. Thus, the cdf of $Z$, $F(Z)$, is $$ F(Z) = \begin{cases} 0 z < -1 \\ \frac12 (1+z)^2 & -1 \le z < 0 \\ \frac12 + z - z^2 / 2 & 0 \le z < 1\\ 1 & 1 \le z \\ \end{cases} $$

and

$$ f(z) = F'(z) = \begin{cases} 0 & z < -1 \\ z + 1 & -1 \le z < 0 \\ 1 - z & 0 \le z < 1\\ 0 & 1 \le z \\ \end{cases} $$

Now let $Z = X / Y$. Note that the pdf of $Z$, $f(z)$, has support on $[0, \infty)$. Observe that $P(Z \le z)$ is the area of the portion of the unit square that is also above the line of slope $\frac1z$ and passes through the origin. If $z \le 1$, $P(Z \le z)$ is the area of the triangle connecting $(0,0)$, $(z, 1)$, $(0, 1)$, $\frac{z}{2}$. If $z > 1$, $P(Z \le z)$ is the area of the region connecting $(0,0)$, $(1, \frac1z)$, $(1,1)$, and $(0,1)$, $1 - \frac{1}{2z}$.

Thus, the cdf of $Z$, $F(Z)$, is $$ F(z) = \begin{cases} 0 & z < 0 \\ \frac{z}{2} & 0 \le z < 1 \\ 1 - \frac{1}{2z} & 1 \le z \\ \end{cases} $$

and

$$ f(z) = \begin{cases} 0 & z < 0 \\ \frac{1}{2} & 0 \le z < 1 \\ \frac{1}{2z^2} & 1 \le z \\ \end{cases} $$

21

Let $X_1, \dots, X_n \sim \text{Exp}(\beta)$ be IID. Let $Y = \text{max} \{X_1, \dots, X_n\}$. Find the PDF of $Y$. Hint: $Y \le y$ if and only if $X_i \le y$ for $i = 1, \dots, n$.

Solution:

We know a priori that the PDF is $0$ on the negative real line, since the exponential distribution has support on the positive real line. Assuming $y \ge 0$, we can calculate the CDF of $Y$, $F(y)$:

\begin{align*} F(y) = P(Y \le y) &= P(X_1 \le y \cap \dots \cap X_n \le y) \\ &= \prod_i P(X_i \le y) \tag{independence} \\ &= P(X_1 \le y) ^ n \tag{identically distributed} \\ &= (1 - e ^ {-y / \beta})^n \tag{Problem #9} \end{align*}

we then can calculate the PDF, $f(y)$, by taking the derivative:

\begin{align*} f(y) &= F'(y) \\ &= \frac{d}{dy} (1 - e ^ {-y / \beta})^n \\ &= \frac{n}{\beta}(1 - e ^ {-y / \beta})^{n - 1} e ^ {-y / \beta} \end{align*}