Maximum Likelihood Estimation | ISI MStat 2017 PSB Problem 8

This problem based on Maximum Likelihood Estimation, gives a detailed solution to ISI M.Stat 2017 PSB Problem 8, with a tinge of simulation and code.

Problem

Let \(\theta>0\) be an unknown parameter, and \(X_{1}, X_{2}, \ldots, X_{n}\) be a random sample from the distribution with density.

\(
f(x) = \begin{cases}
\frac{2x}{\theta^{2}} & , 0 \leq x \leq \theta \\
0 & , \text { otherwise }
\end{cases}
\)

Find the maximum likelihood estimator of \(\theta\) and its mean squared error.

Prerequisites

Proof algorithm to find the MLE of \( \theta\) for U\((0, \theta)\)
Order Statistics \( X_{(1)}, X_{(2)}, \ldots, X_{(n)} \)
Mean Square Error

Solution

Do you remember the method of finding the MLE of \( \theta\) for U\((0, \theta)\) ? Just proceed along a similar line.

\( L(\theta) = f\left(x_{1}, \cdots, x_{n} | \theta\right) \overset{ X_{1}, X_{2}, \ldots, X_{n} \text{are iid}}{=}f\left(x_{1} | \theta\right) \cdots f\left(x_{n} | \theta\right) \\ =

\begin{cases}
\frac{ 2^n \prod_{i=1}^{\infty} X_{i}}{ \theta^{2n}} & , 0 \leq X_{(1)} \leq X_{(2)} \leq \ldots \leq X_{(n)} \leq \theta \\
0 & , \text { otherwise }
\end{cases}
\)

Let's draw the diagram.

Thus, you can see that \( L(\theta) \) is maximized at \( \theta = X_{(n)}\).

Hence, \( \hat{\theta}_{mle} = X_{(n)}\).

MSE

Now, we need to find the distribution of \( X_{(n)}\).

For, that we need to find the distribution function of \(X_i\).

Observe \( F_{X_i}(x) = \begin{cases}
\frac{x^2}{\theta^{2}} & , 0 \leq x \leq \theta \\
0 & , \text { otherwise }
\end{cases} \)

\( F_{X_{(n)}}(x) \overset{\text{Order Statistics}}{=} \begin{cases}
0 &, x \leq 0 \\
\frac{x^{2n}}{\theta^{2n}} & , 0 \leq x \leq \theta \\
1 & , \text { otherwise }
\end{cases} \)

\( f_{X_{(n)}}(x) = \begin{cases}
\frac{2n.x^{2n-1}}{\theta^{2n}} & , 0 \leq x \leq \theta \\
0 & , \text { otherwise }
\end{cases} \)

MSE(\( X_{(n)} \)) = E\(((X_{(n)} - \theta)^2)\)

= \( \int_{0}^{\theta} (x-\theta)^2 f_{X_{(n)}}(x) dx \)

= \( \int_{0}^{\theta} (x-\theta)^2 \frac{2n.x^{2n-1}}{\theta^{2n}} dx \)

= \( \int_{0}^{\theta} (x^2 + {\theta}^2 - 2x\theta) \frac{2n.x^{2n-1}}{\theta^{2n}} dx \)

= \( \int_{0}^{\theta} \frac{2n.x^{2n+1}}{\theta^{2n}} dx \) + \( \int_{0}^{\theta} \frac{2n.x^{2n-1}}{\theta^{2n-2}} dx \) - \( \int_{0}^{\theta} \frac{4n.x^{2n}}{\theta^{2n-1}} dx \)

= \( {\theta}^2(\frac{2n}{2n+2} + 1 - \frac{4n}{2n+1}) = \frac{1}{(2n+1)(n+1)}\)

Observe that \( \lim_{ n \to \infty} { MSE( X_{(n)})} = 0\).

Let's add a computing dimension to it and verify it by simulation.

Let's take \( \theta = 1, n = 5\). MSE is expected to be around 0.002. You can change the \(\theta\) and n and play around.

v = NULL
n = 15
theta = 1
for (i in 1:1000) {
  r = runif(n, 0, theta)
  s = theta*sqrt(r) #We use Inverse Transformation Method to generate the random variables from the distribution.
  m = max(s)
  v = c(v,m)
}
hist(v, freq = FALSE)
k = replicate(1000,1)
mse(v,k) =  0.001959095

You should also check out this link: Triangle Inequality Problems and Solutions

I hope that helps you. Stay tuned.