Get inspired by the success stories of our students in IIT JAM MS, ISI  MStat, CMI MSc DS.  Learn More

# ISI MStat PSB 2018 Problem 9 | Regression Analysis

This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

## Problem - ISI MStat PSB 2018 Problem 9

Suppose $$(y_i,x_i)$$ satisfies the regression model,

$$y_i= \alpha + \beta x_i + \epsilon_i$$ for $$i=1,2,....,n.$$

where $${ x_i : 1 \le i \le n }$$ are fixed constants and $${ \epsilon_i : 1 \le i \le n}$$ are i.i.d. $$N(0, \sigma^2)$$ errors, where $$\alpha, \beta$$ and $$\sigma^2 (>0)$$ are unknown parameters.

(a) Let $$\tilde{\alpha}$$ denote the least squares estimate of $$\alpha$$ obtained assuming $$\beta=5$$. Find the mean squared error (MSE) of $$\tilde{\alpha}$$ in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

### Prerequisites

Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

## Solution :

These problem is simple enough,

for the given model, $$y_i= \alpha + \beta x_i + \epsilon_i$$ for $$i=1,....,n$$.

The scenario is even simpler here since, it is given that $$\beta=5$$ , so our model reduces to,

$$y_i= \alpha + 5x_i + \epsilon_i$$, where $$\epsilon_i \sim N(0, \sigma^2)$$ and $$\epsilon_i$$'s are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of $$\alpha$$ is

$$\tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x}$$ (How ??) where $$\tilde{\beta}$$ is the (generally) the OLS estimate of $$\beta$$, but here $$\beta=5$$ is known, so,

$$\tilde{\alpha}= \bar{y} - 5\bar{x}$$ again,

$$E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}$$, hence $$\tilde{\alpha}$$ is a biased estimator for $$\alpha$$ with $$Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}$$.

So, the Mean Squared Error, MSE of $$\tilde{\alpha}$$ is,

$$MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha})$$ + $${Bias^2}_{\alpha}(\tilde{\alpha})$$

$$= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2$$

[ as, it follows clearly from the model, $$y_i \sim N( \alpha +\beta x_i , \sigma^2)$$ and $$x_i$$'s are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, $$y_i \sim N( \alpha + \beta x_i , \sigma^2 )$$ and we have to find the Maximum Likelihood Estimator of $$\sigma^2$$ and $$\beta$$ and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !

## Food For Thought

Suppose you don't know the value of $$\beta$$ even, What will be the MSE of $$\tilde{\alpha}$$ in that case ?

Also, find the OLS estimate of $$\beta$$ and you already have done it for $$\alpha$$, so now find the MLEs of all $$\alpha$$ and $$\beta$$. Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!

## Subscribe to Cheenta at Youtube

### Knowledge Partner

Cheenta is a knowledge partner of Aditya Birla Education Academy