1

Let $X_1, \dots, X_n \sim \text{Poisson}(\lambda)$ and let $\hat{\lambda} = n^{-1} \sum_{i=1}^n X_i$. Find the bias, se, and MSE of this estimator.

Solution:

Bias: $E\left[ \frac1n \sum_{i=1}^n X_i \right] = \frac{1}{n} n E[X_1] = \lambda = E[X_i] \Rightarrow $ bias = 0

se: $V\left[ \frac1n \sum_{i=1}^n X_i \right] = \frac{1}{n^2} n V[X_1] = \lambda / n \Rightarrow $ se = $\sqrt{\lambda / n}$.

MSE: By bias-variance decomposition, MSE $ = \hat{\lambda} / n$.

2

Let $X_1, \dots, X_n \sim \text{Uniform}(0,\theta)$ and let $\hat{\theta} = \max \{X_1, \dots, X_n\}$. Find the bias, se, and MSE of this estimator.

Solution:

Observing

\begin{align*} P(\hat{\theta} \le x) &= P \left( \bigcap_{i=1}^n X_i \le x \right) \\ &= P(X_1 \le x) ^ n \\ &= (x / \theta)^n, \end{align*}

we see the PDF of $\hat{\theta}$ is $nx^{n-1}/\theta^n \mathbb{1}_{[0,\theta]}$, meaning

\begin{align*} E[\hat{\theta}] &= \frac{n}{\theta^n} \int_0^\theta x^n \, dx \\ &= \frac{n}{n+1} \theta \end{align*}

and hence the bias is $\frac{n}{n+1} \theta - \theta = -\frac{\theta}{n+1}$.

Meanwhile,

\begin{align*} E \left[\hat{\theta}^2 \right] &= \frac{n}{\theta^n} \int_0^\theta x^{n+1} \, dx \\ &= \frac{n}{n+2} \theta^2 \end{align*}

meaning $V\left[\hat{\theta}^2 \right] = E \left[\hat{\theta}^2 \right] - E[\hat{\theta}]^2 = \theta^2 \left[\frac{n}{(n+2)(n 1)^2 }\right]$, and se = $\theta \sqrt{\frac{n}{(n+2)(n +1)^2 }}$.

By the bias-variance decomposition, the MSE is $\frac{\theta^2}{(n+1)^2} + \theta^2 \left[\frac{n}{(n+2)(n + 1)^2 }\right] = \frac{\theta^2}{(n+1)^2} \left[ 1 + \frac{n}{n+2} \right]$.

3

Let $X_1, \dots, X_n \sim \text{Uniform}(0,\theta)$ and let $\hat{\theta} = 2 \bar{X}_n$. Find the bias, se, and MSE of this estimator.

Solution:

We have $E[\hat{\theta}] = E[2 \bar{X}_n] = \frac{2}{n} n E[X_1] = \theta$, so the bias is 0.

Meanwhile, $V[\hat{\theta}] = V(2 \bar{X}_n] = \frac{4}{n} V[X_1] = \frac{4}{n} \frac{\theta^2}{12} = \frac{\theta^2}{3n}$, meaning se $= \frac{\theta}{\sqrt{3n}}$.

By the bias-variance decomposition, the MSE is then $\frac{\theta^2}{3n}$.