 Get inspired by the success stories of our students in IIT JAM MS, ISI  MStat, CMI MSc Data Science.  Learn More

# ISI MStat PSB 2018 Problem 9 | Regression Analysis This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

## Problem - ISI MStat PSB 2018 Problem 9

Suppose $(y_i,x_i)$ satisfies the regression model,

$y_i= \alpha + \beta x_i + \epsilon_i$ for $i=1,2,....,n.$

where ${ x_i : 1 \le i \le n }$ are fixed constants and ${ \epsilon_i : 1 \le i \le n}$ are i.i.d. $N(0, \sigma^2)$ errors, where $\alpha, \beta$ and $\sigma^2 (>0)$ are unknown parameters.

(a) Let $\tilde{\alpha}$ denote the least squares estimate of $\alpha$ obtained assuming $\beta=5$. Find the mean squared error (MSE) of $\tilde{\alpha}$ in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

### Prerequisites

Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

## Solution :

These problem is simple enough,

for the given model, $y_i= \alpha + \beta x_i + \epsilon_i$ for $i=1,....,n$.

The scenario is even simpler here since, it is given that $\beta=5$ , so our model reduces to,

$y_i= \alpha + 5x_i + \epsilon_i$, where $\epsilon_i \sim N(0, \sigma^2)$ and $\epsilon_i$'s are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of $\alpha$ is

$\tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x}$ (How ??) where $\tilde{\beta}$ is the (generally) the OLS estimate of $\beta$, but here $\beta=5$ is known, so,

$\tilde{\alpha}= \bar{y} - 5\bar{x}$ again,

$E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}$, hence $\tilde{\alpha}$ is a biased estimator for $\alpha$ with $Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}$.

So, the Mean Squared Error, MSE of $\tilde{\alpha}$ is,

$MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha})$ + ${Bias^2}_{\alpha}(\tilde{\alpha})$

$= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2$

[ as, it follows clearly from the model, $y_i \sim N( \alpha +\beta x_i , \sigma^2)$ and $x_i$'s are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, $y_i \sim N( \alpha + \beta x_i , \sigma^2 )$ and we have to find the Maximum Likelihood Estimator of $\sigma^2$ and $\beta$ and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !

## Food For Thought

Suppose you don't know the value of $\beta$ even, What will be the MSE of $\tilde{\alpha}$ in that case ?

Also, find the OLS estimate of $\beta$ and you already have done it for $\alpha$, so now find the MLEs of all $\alpha$ and $\beta$. Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!

## Subscribe to Cheenta at Youtube

This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

## Problem - ISI MStat PSB 2018 Problem 9

Suppose $(y_i,x_i)$ satisfies the regression model,

$y_i= \alpha + \beta x_i + \epsilon_i$ for $i=1,2,....,n.$

where ${ x_i : 1 \le i \le n }$ are fixed constants and ${ \epsilon_i : 1 \le i \le n}$ are i.i.d. $N(0, \sigma^2)$ errors, where $\alpha, \beta$ and $\sigma^2 (>0)$ are unknown parameters.

(a) Let $\tilde{\alpha}$ denote the least squares estimate of $\alpha$ obtained assuming $\beta=5$. Find the mean squared error (MSE) of $\tilde{\alpha}$ in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

### Prerequisites

Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

## Solution :

These problem is simple enough,

for the given model, $y_i= \alpha + \beta x_i + \epsilon_i$ for $i=1,....,n$.

The scenario is even simpler here since, it is given that $\beta=5$ , so our model reduces to,

$y_i= \alpha + 5x_i + \epsilon_i$, where $\epsilon_i \sim N(0, \sigma^2)$ and $\epsilon_i$'s are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of $\alpha$ is

$\tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x}$ (How ??) where $\tilde{\beta}$ is the (generally) the OLS estimate of $\beta$, but here $\beta=5$ is known, so,

$\tilde{\alpha}= \bar{y} - 5\bar{x}$ again,

$E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}$, hence $\tilde{\alpha}$ is a biased estimator for $\alpha$ with $Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}$.

So, the Mean Squared Error, MSE of $\tilde{\alpha}$ is,

$MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha})$ + ${Bias^2}_{\alpha}(\tilde{\alpha})$

$= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2$

[ as, it follows clearly from the model, $y_i \sim N( \alpha +\beta x_i , \sigma^2)$ and $x_i$'s are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, $y_i \sim N( \alpha + \beta x_i , \sigma^2 )$ and we have to find the Maximum Likelihood Estimator of $\sigma^2$ and $\beta$ and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !

## Food For Thought

Suppose you don't know the value of $\beta$ even, What will be the MSE of $\tilde{\alpha}$ in that case ?

Also, find the OLS estimate of $\beta$ and you already have done it for $\alpha$, so now find the MLEs of all $\alpha$ and $\beta$. Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!

## Subscribe to Cheenta at Youtube

This site uses Akismet to reduce spam. Learn how your comment data is processed.

### Knowledge Partner  