Get inspired by the success stories of our students in IIT JAM MS, ISI  MStat, CMI MSc DS.  Learn More 

ISI MStat PSB 2018 Problem 9 | Regression Analysis

This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

Problem - ISI MStat PSB 2018 Problem 9


Suppose (y_i,x_i) satisfies the regression model,

y_i= \alpha + \beta x_i + \epsilon_i for i=1,2,....,n.

where { x_i : 1 \le i \le n } are fixed constants and { \epsilon_i : 1 \le i \le n} are i.i.d. N(0, \sigma^2) errors, where \alpha, \beta and \sigma^2 (>0) are unknown parameters.

(a) Let \tilde{\alpha} denote the least squares estimate of \alpha obtained assuming \beta=5. Find the mean squared error (MSE) of \tilde{\alpha} in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

Prerequisites


Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

Solution :

These problem is simple enough,

for the given model, y_i= \alpha + \beta x_i + \epsilon_i for i=1,....,n.

The scenario is even simpler here since, it is given that \beta=5 , so our model reduces to,

y_i= \alpha + 5x_i + \epsilon_i, where \epsilon_i \sim N(0, \sigma^2) and \epsilon_i's are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of \alpha is

\tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x} (How ??) where \tilde{\beta} is the (generally) the OLS estimate of \beta, but here \beta=5 is known, so,

\tilde{\alpha}= \bar{y} - 5\bar{x} again,

E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}, hence \tilde{\alpha} is a biased estimator for \alpha with Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}.

So, the Mean Squared Error, MSE of \tilde{\alpha} is,

MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha}) + {Bias^2}_{\alpha}(\tilde{\alpha})

= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2

[ as, it follows clearly from the model, y_i \sim N( \alpha +\beta x_i , \sigma^2) and x_i's are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, y_i \sim N( \alpha + \beta x_i , \sigma^2 ) and we have to find the Maximum Likelihood Estimator of \sigma^2 and \beta and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !


Food For Thought

Suppose you don't know the value of \beta even, What will be the MSE of \tilde{\alpha} in that case ?

Also, find the OLS estimate of \beta and you already have done it for \alpha, so now find the MLEs of all \alpha and \beta. Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!


Similar Problems and Solutions



ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

Problem - ISI MStat PSB 2018 Problem 9


Suppose (y_i,x_i) satisfies the regression model,

y_i= \alpha + \beta x_i + \epsilon_i for i=1,2,....,n.

where { x_i : 1 \le i \le n } are fixed constants and { \epsilon_i : 1 \le i \le n} are i.i.d. N(0, \sigma^2) errors, where \alpha, \beta and \sigma^2 (>0) are unknown parameters.

(a) Let \tilde{\alpha} denote the least squares estimate of \alpha obtained assuming \beta=5. Find the mean squared error (MSE) of \tilde{\alpha} in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

Prerequisites


Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

Solution :

These problem is simple enough,

for the given model, y_i= \alpha + \beta x_i + \epsilon_i for i=1,....,n.

The scenario is even simpler here since, it is given that \beta=5 , so our model reduces to,

y_i= \alpha + 5x_i + \epsilon_i, where \epsilon_i \sim N(0, \sigma^2) and \epsilon_i's are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of \alpha is

\tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x} (How ??) where \tilde{\beta} is the (generally) the OLS estimate of \beta, but here \beta=5 is known, so,

\tilde{\alpha}= \bar{y} - 5\bar{x} again,

E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}, hence \tilde{\alpha} is a biased estimator for \alpha with Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}.

So, the Mean Squared Error, MSE of \tilde{\alpha} is,

MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha}) + {Bias^2}_{\alpha}(\tilde{\alpha})

= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2

[ as, it follows clearly from the model, y_i \sim N( \alpha +\beta x_i , \sigma^2) and x_i's are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, y_i \sim N( \alpha + \beta x_i , \sigma^2 ) and we have to find the Maximum Likelihood Estimator of \sigma^2 and \beta and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !


Food For Thought

Suppose you don't know the value of \beta even, What will be the MSE of \tilde{\alpha} in that case ?

Also, find the OLS estimate of \beta and you already have done it for \alpha, so now find the MLEs of all \alpha and \beta. Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!


Similar Problems and Solutions



ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Knowledge Partner

Cheenta is a knowledge partner of Aditya Birla Education Academy
Cheenta

Cheenta Academy

Aditya Birla Education Academy

Aditya Birla Education Academy

Cheenta. Passion for Mathematics

Advanced Mathematical Science. Taught by olympians, researchers and true masters of the subject.
JOIN TRIAL
support@cheenta.com
Menu
Trial
Whatsapp
rockethighlight