Categories
Calculus College Mathematics I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT ISI MSTAT Miscellaneous Probability Statistics

ISI MStat PSB 2009 Problem 4 | Polarized to Normal

This is a very beautiful sample problem from ISI MStat PSB 2009 Problem 4. It is based on the idea of Polar Transformations, but need a good deal of observation o realize that. Give it a Try it !

Problem– ISI MStat PSB 2009 Problem 4


Let \(R\) and \(\theta\) be independent and non-negative random variables such that \(R^2 \sim {\chi_2}^2 \) and \(\theta \sim U(0,2\pi)\). Fix \(\theta_o \in (0,2\pi)\). Find the distribution of \(R\sin(\theta+\theta_o)\).

Prerequisites


Convolution

Polar Transformation

Normal Distribution

Solution :

This problem may get nasty, if one try to find the required distribution, by the so-called CDF method. Its better to observe a bit, before moving forward!! Recall how we derive the probability distribution of the sample variance of a sample from a normal population ??

Yes, you are thinking right, we need to use Polar Transformation !!

But, before transforming lets make some modifications, to reduce future complications,

Given, \(\theta \sim U(0,2\pi)\) and \(\theta_o \) is some fixed number in \((0,2\pi)\), so, let \(Z=\theta+\theta_o \sim U(\theta_o,2\pi +\theta_o)\).

Hence, we need to find the distribution of \(R\sin Z\). Now, from the given and modified information the joint pdf of \(R^2\) and \(Z\) are,

\(f_{R^2,Z}(r,z)=\frac{r}{2\pi}exp(-\frac{r^2}{2}) \ \ R>0, \theta_o \le z \le 2\pi +\theta_o \)

Now, let the transformation be \((R,Z) \to (X,Y)\),

\(X=R\cos Z \\ Y=R\sin Z\), Also, here \(X,Y \in \mathbb{R}\)

Hence, \(R^2=X^2+Y^2 \\ Z= \tan^{-1} (\frac{Y}{X}) \)

Hence, verify the Jacobian of the transformation \(J(\frac{r,z}{x,y})=\frac{1}{r}\).

Hence, the joint pdf of \(X\) and \(Y\) is,

\(f_{X,Y}(xy)=f_{R,Z}(x^2+y^2, \tan^{-1}(\frac{y}{x})) J(\frac{r,z}{x,y}) \\ =\frac{1}{2\pi}exp(-\frac{x^2+y^2}{2})\) , \(x,y \in \mathbb{R}\).

Yeah, Now it is looking familiar right !!

Since, we need the distribution of \(Y=R\sin Z=R\sin(\theta+\theta_o)\), we integrate \(f_{X,Y}\) w.r.t to \(X\) over the real line, and we will end up with, the conclusion that,

\(R\sin(\theta+\theta_o) \sim N(0,1)\). Hence, We are done !!


Food For Thought

From the above solution, the distribution of \(R\cos(\theta+\theta_o)\) is also determinable right !! Can you go further investigating the occurrence pattern of \(\tan(\theta+\theta_o)\) ?? \(R\) and \(\theta\) are the same variables as defined in the question.

Give it a try !!


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT Probability Statistics

ISI MStat PSB 2008 Problem 7 | Finding the Distribution of a Random Variable

This is a very beautiful sample problem from ISI MStat PSB 2008 Problem 7 based on finding the distribution of a random variable . Let’s give it a try !!

Problem– ISI MStat PSB 2008 Problem 7


Let \( X\) and \( Y\) be exponential random variables with parameters 1 and 2 respectively. Another random variable \( Z\) is defined as follows.

A coin, with probability p of Heads (and probability 1-p of Tails) is
tossed. Define \( Z\) by \( Z=\begin{cases} X & , \text { if the coin turns Heads } \\ Y & , \text { if the coin turns Tails } \end{cases} \)
Find \( P(1 \leq Z \leq 2)\)

Prerequisites


Cumulative Distribution Function

Exponential Distribution

Solution :

Let , \( F_{i} \) be the CDF for i=X,Y, Z then we have ,

\( F_{Z}(z) = P(Z \le z) = P( Z \le z | coin turns Head )P(coin turns Head) + P( Z \le z | coin turns Tail ) P( coin turns Tail) \)

=\( P( X \le z)p + P(Y \le z ) (1-p) \) = \( F_{X}(z)p+F_{Y}(y) (1-p) \)

Therefore pdf of Z is given by \( f_{Z}(z)= pf_{X}(z)+(1-p)f_{Y}(z) \) , where \( f_{X} and f_{Y} \) are pdf of X,Y respectively .

So , \( P(1 \leq Z \leq 2) = \int_{1}^{2} \{pe^{-z} + (1-p) 2e^{-2z}\} dz = p \frac{e-1}{e^2} +(1-p) \frac{e^2-1}{e^4} \)

Food For Thought

Find the the distribution function of \( K=\frac{X}{Y} \) and then find \( \lim_{K \to \infty} P(K >1 ) \)


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
Calculus College Mathematics I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT ISI MSTAT Probability Statistics Theory of Estimation

ISI MStat PSB 2009 Problem 6 | abNormal MLE of Normal

This is a very beautiful sample problem from ISI MStat PSB 2009 Problem 6. It is based on the idea of Restricted Maximum Likelihood Estimators, and Mean Squared Errors. Give it a Try it !

Problem-ISI MStat PSB 2009 Problem 6


Suppose \(X_1,…..,X_n\) are i.i.d. \(N(\theta,1)\), \(\theta_o \le \theta \le \theta_1\), where \(\theta_o < \theta_1\) are two specified numbers. Find the MLE of \(\theta\) and show that it is better than the sample mean \(\bar{X}\) in the sense of having smaller mean squared error.

Prerequisites


Maximum Likelihood Estimators

Normal Distribution

Mean Squared Error

Solution :

This is a very interesting Problem ! We all know, that if the condition “\(\theta_o \le \theta \le \theta_1\), for some specified numbers \(\theta_o < \theta_1\)” had been not given, then the MLE would have been simply \(\bar{X}=\frac{1}{n}\sum_{k=1}^n X_k\), the sample mean of the given sample. But due to the restriction over \(\theta\) things get interestingly complicated.

So, simplify a bit, lets write the Likelihood Function of \(theta\) given this sample, \(\vec{X}=(X_1,….,X_n)’\),

\(L(\theta |\vec{X})={\frac{1}{\sqrt{2\pi}}}^nexp(-\frac{1}{2}\sum_{k=1}^n(X_k-\theta)^2)\), when \(\theta_o \le \theta \le \theta_1\)ow taking natural log both sides and differentiating, we find that ,

\(\frac{d\ln L(\theta|\vec{X})}{d\theta}= \sum_{k=1}^n (X_k-\theta) \).

Now, verify that if \(\bar{X} < \theta_o\), then \(L(\theta |\vec{X})\) is always a decreasing function of \(\theta\), [ where, \(\theta_o \le \theta \le \theta_1\)], Hence the maximum likelihood attains at \(\theta_o\) itself. Similarly, when, \(\theta_o \le \bar{X} \le \theta_1\), the maximum likelihood attains at \(\bar{X}\), lastly the likelihood function will be increasing, hence the maximum likelihood will be found at \(\theta_1\).

Hence, the Restricted Maximum Likelihood Estimator of \(\theta\), say

\(\hat{\theta_{RML}} = \begin{cases} \theta_o & \bar{X} < \theta_o \\ \bar{X} & \theta_o\le \bar{X} \le \theta_1 \\ \theta_1 & \bar{X} > \theta_1 \end{cases}\)

Now, to check that, \(\hat{\theta_{RML}}\) is a better estimator than \(\bar{X}\), in terms of Mean Squared Error (MSE).

Now, \(MSE_{\theta}(\bar{X})=E_{\theta}(\bar{X}-\theta)^2=\int^{-\infty}_\infty (\bar{X}-\theta)^2f_X(x)\,dx\)

\(=\int^{-\infty}_{\theta_o} (\bar{X}-\theta)^2f_X(x)\,dx+\int^{\theta_o}_{\theta_1} (\bar{X}-\theta)^2f_X(x)\,dx+\int^{\theta_1}_\infty (\bar{X}-\theta)^2f_X(x)\,dx\).

\(\ge \int^{-\infty}_{\theta_o} (\theta_o-\theta)^2f_X(x)\,dx+\int^{\theta_o}_{\theta_1} (\bar{X}-\theta)^2f_X(x)\,dx+\int^{\theta_1}_\infty (\theta_1-\theta)^2f_X(x)\,dx\)

\(=E_{\theta}(\hat{\theta_{RML}}-\theta)^2=MSE_{\theta}(\hat{\theta_{RML}})\).

Hence proved !!


Food For Thought

Now, can you find an unbiased estimator, for \(\theta^2\) ?? Okay!! now its quite easy right !! But is the estimator you are thinking about is the best unbiased estimator !! Calculate the variance and also compare weather the Variance is attaining Cramer-Rao Lowe Bound.

Give it a try !! You may need the help of Stein’s Identity.


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
Calculus College Mathematics I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT ISI MSTAT Probability

ISI MStat PSB 2009 Problem 3 | Gamma is not abNormal

This is a very simple but beautiful sample problem from ISI MStat PSB 2009 Problem 3. It is based on recognizing density function and then using CLT. Try it !

Problem– ISI MStat PSB 2009 Problem 3


Using and appropriate probability distribution or otherwise show that,

\( \lim\limits_{x\to\infty}\int^n_0 \frac{exp(-x)x^{n-1}}{(n-1)!}\,dx =\frac{1}{2}\).

Prerequisites


Gamma Distribution

Central Limit Theorem

Normal Distribution

Solution :

Here all we need is to recognize the structure of the integrand. Look, that here, the integrand is integrated over the non-negative real numbers. Now, event though here it is not mentioned explicitly that \(x\) is a random variable, we can assume \(x\) to be some value taken by a random variable \(X\). After all we can find randomness anywhere and everywhere !!

Now observe that the integrand has a structure which is very identical to the density function of gamma random variable with parameters \(1\) ande \(n\). So, if we assume that \(X\) is a \(Gamma(1, n)\), then our limiting integral transforms to,

\(\lim\limits_{x\to\infty}P(X \le n)\).

Now, we know that if \(X \sim Gamma(1,n)\), then its mean and variance both are \(n\).

So, as \(n \uparrow \infty\), \(\frac{X-n}{\sqrt{n}} \to N(0,1)\), by Central Limit Theorem.

Hence, \(\lim\limits_{x\to\infty}P(X \le n)=\lim\limits_{x\to\infty}P(\frac{X-n}{\sqrt{n}} \le 0)=\lim\limits_{x\to\infty}\Phi (0)=\frac{1}{2}\). [ here \(\Phi(z)\) is the cdf of Normal at \(z\).]

Hence proved !!


Food For Thought

Can, you do the proof under the “Otherwise” condition !!

Give it a try !!


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT Probability Statistics

ISI MStat PSB 2013 Problem 8 | Finding the Distribution of a Random Variable

This is a very beautiful sample problem from ISI MStat PSB 2013 Problem 8 based on finding the distribution of a random variable . Let’s give it a try !!

Problem– ISI MStat PSB 2013 Problem 8


  1. Suppose \(X_{1}\) is a standard normal random variable. Define
  2. \( X_{2}= \begin{cases} – X_{1} & , \text{if } |X_{1}|<1 \\ X_{1} & \text{otherwise} \end{cases} \)
    (a) Show that \(X_{2}\) is also a standard normal random variable.
    (b) Obtain the cumulative distribution function of \(X_{1}+X_{2}\) in terms of the cumulative distribution function of a standard normal random
    variable.

Prerequisites


Cumulative Distribution Function

Normal Distribution

Solution :

(a) Let \( F_{X_{2}}(x) \) be distribution function of X_{2}\) then we can say that ,

\( F_{X_{2}}(x) = P( X_{2} \le x) = P( X_{2} \le x | |X_{1}| < 1) P( |X_{1}| <1) + P( X_{2} \le x | |X_{1}| > 1 ) P( |X_{1}| >1) \)

= \( P( – X_{1} ||X_{1}| < 1)P( |X_{1}| <1) + P( X_{1} \le x | |X_{1}| > 1 ) P( |X_{1}| >1) \)

= \( P( – X_{1}\le x , |X_{1}| < 1 ) + P( X_{1} \le x , |X_{1}| > 1 ) \)

= \( P( X_{1}\le x , |-X_{1}| < 1 ) + P( X_{1} \le x , |X_{1}| > 1 ) \)

Since \( X_{1} \sim N(0,1) \) hence it’s symmetric about 0 . So,\( X_{1}\) and\( -X_{1}\) are identically distributed .

Therefore , \( F_{X_{2}}(x) = P( X_{1}\le x , |X_{1}| < 1 ) + P( X_{1} \le x , |X_{1}| > 1 ) \)

=\( P(X_{1} \le x ) = \Phi(x) \)

Hence , \(X_{2}\) is also a standard normal random variable.

(b) Let , \(Y= X_{1} + X_{2} = \begin{cases} 0 & \text{if } |X_{1}|<1 \\ 2X_{1} & \text{ otherwise } \end{cases} \)

Distribution function \( F_{Y}(y) = P(Y \le y) \)

=\( P(Y \le y | |X_{1} < 1) P(|X_{1}| <1) + P( Y\le y | |X_{1}| >1)P(|X_{1}|>1) \)

= \( P( 0 \le y , -1 \le X_{1} \le 1 ) + P( 2X_{1} \le y , ( X_{1} >1 \cup X_{1}<-1)) \) \)

= \( P(0 \le y , -1 \le X_{1} \le 1 ) + P( X_{1} \le \frac{y}{2} , X_{1} > 1) + P( X_{1} \le \frac{y}{2} , X_{1} < -1) \)

= \( P(0 \le y , -1 \le X_{1} \le 1 ) + P( 1< X_{1} \le \frac{y}{2}) + P( X_{1} \le min{ \frac{y}{2} , -1 } ) \)

= \( \begin{cases} P( -1 \le X_{1} \le 1 ) + P( 1< X_{1} \le \frac{y}{2}) + P( X_{1} \le -1) & y \ge 2 \\ P( -1 \le X_{1} \le 1 ) + P( X_{1} \le -1) & 0 \le y < 2 \\ P( X_{1} \le -1) & -2 \le y < 0 \\ P( X_{1} \le \frac{y}{2}) & y<-2 \end{cases} \)

= \( \begin{cases} \Phi( \frac{y}{2} ) & y<-2 \\ \Phi(-1) & -2 \le y < 0 \\ \Phi(1) & 0 \le y <2 \\ \Phi(\frac{y}{2} ) & y \ge 2 \end{cases} \) .

Food For Thought

Find the the distribution function of \( 2X_{1}-X_{2} \) in terms of the cumulative distribution function of a standard normal random variable.


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT ISI MSTAT Regression Techniques Statistics Theory of Estimation

ISI MStat PSB 2018 Problem 9 | Regression Analysis

This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!

Problem – ISI MStat PSB 2018 Problem 9


Suppose \((y_i,x_i)\) satisfies the regression model,

\( y_i= \alpha + \beta x_i + \epsilon_i \) for \(i=1,2,….,n.\)

where \({ x_i : 1 \le i \le n }\) are fixed constants and \({ \epsilon_i : 1 \le i \le n}\) are i.i.d. \(N(0, \sigma^2)\) errors, where \(\alpha, \beta \) and \(\sigma^2 (>0)\) are unknown parameters.

(a) Let \(\tilde{\alpha}\) denote the least squares estimate of \(\alpha\) obtained assuming \(\beta=5\). Find the mean squared error (MSE) of \(\tilde{\alpha}\) in terms of model parameters.

(b) Obtain the maximum likelihood estimator of this MSE.

Prerequisites


Normal Distribution

Ordinary Least Square Estimates

Maximum Likelihood Estimates

Solution :

These problem is simple enough,

for the given model, \( y_i= \alpha + \beta x_i + \epsilon_i \) for \( i=1,….,n\).

The scenario is even simpler here since, it is given that \(\beta=5\) , so our model reduces to,

\(y_i= \alpha + 5x_i + \epsilon_i \), where \( \epsilon_i \sim N(0, \sigma^2)\) and \(\epsilon_i \)’s are i.i.d.

now we know that the Ordinary Least Square (OLS) estimate of \(\alpha\) is

\( \tilde{\alpha} = \bar{y} – \tilde{\beta}\bar{x} \) (How ??) where \(\tilde{\beta}\) is the (generally) the OLS estimate of \(\beta\), but here \(\beta=5\) is known, so,

\(\tilde{\alpha}= \bar{y} – 5\bar{x} \) again,

\(E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}\), hence \( \tilde{\alpha} \) is a biased estimator for \(\alpha\) with \(Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}\).

So, the Mean Squared Error, MSE of \(\tilde{\alpha}\) is,

\(MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} – \alpha)^2=Var(\tilde{\alpha}) \) + \({Bias^2}_{\alpha}(\tilde{\alpha}) \)

\(= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2 \)

[ as, it follows clearly from the model, \( y_i \sim N( \alpha +\beta x_i , \sigma^2)\) and \(x_i\)’s are non-stochastic ] .

(b) the last part follows directly from the, the note I provided at the end of part (a),

that is, \(y_i \sim N( \alpha + \beta x_i , \sigma^2 ) \) and we have to find the Maximum Likelihood Estimator of \(\sigma^2\) and \(\beta\) and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !


Food For Thought

Suppose you don’t know the value of \(\beta\) even, What will be the MSE of \(\tilde{\alpha}\) in that case ?

Also, find the OLS estimate of \(\beta\) and you already have done it for \(\alpha\), so now find the MLEs of all \(\alpha\) and \(\beta\). Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB ISI MSAT ISI MSTAT Miscellaneous Statistics Testing of Hypothesis

ISI MStat PSB 2013 Problem 7 | Bernoulli interferes Normally

This is a very simple and beautiful sample problem from ISI MStat PSB 2013 Problem 7. It is mainly based on simple hypothesis testing of normal variables where it is just modified with a bernoulli random variable. Try it!

Problem– ISI MStat PSB 2013 Problem 7


Suppose \(X_1\) and \(X_2\) are two independent and identically distributed random variables with \(N(\theta, 1)\). Further consider a Bernoulli random variable \(V\) with \(P(V=1)=\frac{1}{4}\) which is independent of \(X_1\) and \(X_2\) . Define \(X_3\) as,

\(X_3 = \begin{cases} X_1 & when & V=0 \\ X_2 & when & V=1 \end{cases}\)

For testing \(H_o: \theta= 0\) against \(H_1=\theta=1\) consider the test:

Rejects \(H_o\) if \(\frac{(X_1+X_2+X_3)}{3} >c\).

Find \(c\) such that the test has size \(0.05\).

Prerequisites


Normal Distribution

Simple Hypothesis Testing

Bernoulli Trials

Solution :

These problem is simple enough, the only trick is that to observe that the test rule is based on 3 random variables, \(X_1,X_2 \) and \(X_3\) but \(X_3\) on extension is dependent on the the other bernoulli variable \(V\).

So, here it is given that we reject \(H_o\) at size \(0.05\) if \(\frac{(X_1+X_2+X_3)}{3}> c\) such that,

\(P_{H_o}(\frac{X_1+X_2+X_3}{3}>c)=0.05\)

So, Using law of Total Probability as, \(X_3\) is conditioned on \(V\),

\(P_{H_o}(X_1+X_2+X_3>3c|V=0)P(V=0)+P_{H_o}(X_1+X_2+X_3>3c|V=1)P(V=1)=0.05\)

\(\Rightarrow P_{H_o}(2X_1+X_2>3c)\frac{3}{4}+P_{H_o}(X_1+2X_2>3c)\frac{1}{4}=0.05 \) [ remember, \(X_1\), and \(X_2\) are independent of \(V\)].

Now, under \(H_o\) , \(2X_1+X_2 \sim N(0,5) \)and \( X_1+2X_2 \sim N(0,5) \) ,

So, the rest part is quite obvious and easy to figure it out which I leave it is an exercise itself !!


Food For Thought

Lets end this discussion with some exponential,

Suppose, \(X_1,X_2,….,X_n\) are a random sample from \(exponential(\theta)\) and \(Y_1,Y_2,…..,Y_m\) is another random sample from the population of \(exponential(\mu)\). Now you are to test \(H_o: \theta=\mu\) against \(H_1: \theta \neq \mu \) .

Can you show that the test can be based on a statistic \(T\) such that, \(T= \frac{\sum X_i}{\sum X_i +\sum Y_i}\).

What distribution you think, T should follow under null hypothesis ? Think it over !!


ISI MStat PSB 2008 Problem 10
Outstanding Statistics Program with Applications

Outstanding Statistics Program with Applications

Subscribe to Cheenta at Youtube


Categories
I.S.I. and C.M.I. Entrance IIT JAM Statistics ISI M.Stat PSB Probability Theory of Estimation

Useless Data, Conditional Probability, and Independence | Cheenta Probability Series

This concept of independence, conditional probability and information contained always fascinated me. I have thus shared some thoughts upon this.

When do you think some data is useless?

Some data/ information is useless if it has no role in understanding the hypothesis we are interested in.

We are interested in understanding the following problem.

\(X\) is some event. \(Y\) is another event. How much information do \(Y\) and \(X\) give about each other?

We can model an event by a random variable. So, let’s reframe the problem as follows.

\(X\) and \(Y\) are two random variables. How much information do \(Y\) and \(X\) give about each other?

There is something called entropy. But, I will not go into that. Rather I will give a probabilistic view only. The conditional probability marches in here. We have to use the idea that we have used the information of \(Y\), i.e. conditioned on \(Y\). Hence, we will see how \(X \mid Y\) will behave?

How does \( X \mid Y\) behave? If \(Y\) has any effect on \(X\), then \(X \mid Y\) would have changed right?

But, if \(Y\) has no effect on \(X\), then \(X \mid Y\) will not change and remain same as X. Mathematically, it means

\( X \mid Y\) ~ \(X\) \(\iff\) \( X \perp \!\!\! \perp Y\)

We cannot distinguish between the initial and the final even after conditioning on \(Y\).

Theorem

\(X\) and \(Y\) are independent \( \iff \) \( f(x,y) = P(X =x \mid Y = y) \) is only a function of \(x\).

Proof

\( \Rightarrow\)

\(X\) and \(Y\) are independent \( \Rightarrow \) \( f(x,y) = P(X =x \mid Y = y) = P(X = x)\) is only a function of \(x\).

\( \Leftarrow \)

Let \( \Omega \) be the support of \(Y\).

\( P(X =x \mid Y = y) = g(x) \Rightarrow \)

\( P(X=x) = \int_{\Omega} P(X =x \mid Y = y).P(Y = y)dy \)

\(= g(x) \int_{\Omega} P(Y = y)dy = g(x) = P(X =x \mid Y = y) \)

Exercises

  1. \((X,Y)\) is a bivariate standard normal with \( \rho = 0.5\) then \( 2X – Y \perp \!\!\! \perp Y\).
  2. \(X, Y, V, W\) are independent standard normal, then \( \frac{VX + WY}{\sqrt{V^2+W^2}} \perp \!\!\! \perp (V,W) \).

Random Thoughts (?)

How to quantify the amount of information contained by a random variable in another random variable?

Information contained in \(X\) = Entropy of a random variable \(H(X)\) is defined by \( H(X) = E(-log(P(X)) \).

Now define the information of \(Y\) contained in \(X\) as \(\mid H(X) – H(X|Y) \mid\).

Thus, it turns out that \(H(X) – H(X|Y) = E_{(X,Y)} (log(\frac{P(X \mid Y)}{P(X)})) = H(Y) – H(Y|X) = D(X,Y)\).

\(D(X,Y)\) = Amount of information contained in \(X\) and \(Y\) about each other.

Exercise

  • Prove that \(H(X) \geq H(f(X))\).
  • Prove that \(X \perp \!\!\! \perp Y \Rightarrow D(X,Y) = 0\).

Note: This is just a mental construction I did, and I am not sure of the existence of the measure of this information contained in literature. But, I hope I have been able to share some statistical wisdom with you. But I believe this is a natural construction, given the properties are satisfied. It will be helpful, if you get hold of some existing literature and share it to me in the comments.

Some useful Links:

Categories
IIT JAM Statistics ISI M.Stat PSB Probability

Correlation of two ab(Normals) | ISI MStat 2016 PSB Problem 6

This problem is an interesting application of the moment generating function of normal random variable to see how the correlation behaves under monotone function. This is the problem 6 from ISI MStat 2016 PSB.

Problem

Suppose that random variables \(X\) and \(Y\) jointly have a bivariate normal distribution with \(\mathrm{E}(X)=\mathrm{E}(Y)=0, {Var}(X)={Var}(Y)=1,\) and
correlation \(\rho\). Compute the correlation between \(e^{X}\) and \(e^{Y}\).

Prerequisites

  • Correlation Coefficient
  • Moment Generating Function
  • Moment Generating Function \(M_X(t)\) of Normal \(X\) ~ \(N(\mu, \sigma^2)\) is \( e^{t\mu + \frac{\sigma^2t^2}{2}} \).

Solution

\(M_X(t) = E(e^{tX})\) is called the moment generating function.

Now, let’s try to calculate \( Cov(e^X, e^Y) = E(e^{X+Y}) – E(e^X)E(e^Y)\)

For, that we need to have the following in our arsenal.

  • \(X\) ~ \( N(0, 1)\)
  • \(Y\) ~ \( N(0, 1)\)
  • \(X+Y\) ~ \( N(0, \sigma^2 = 2(1+\rho))\) [ We will calculate this \(\sigma^2\) just below ].

\( \sigma^2 = Var(X+Y) = Var(X) + 2Cov(X,Y) + Var(Y) = 1 + 2\rho + 1 = 2(1+\rho) \).

Now observe the following:

  • \(E(e^{X+Y}) = M_{X+Y}(1) = e^{1+\rho} \)
  • \(E(e^X) = M_{X}(1) = e^{\frac{1}{2}} \)
  • \( E(e^Y) = M_{Y}(1) = e^{\frac{1}{2}} \)
  • \( \Rightarrow Cov(e^X, e^Y) = E(e^{X+Y}) – E(e^X)E(e^Y) = e^{1+\rho} – e = e(e^{\rho} – 1)\)

Important Observation

\( Cor(e^X, e^Y)\) and \(Cor(X,Y) = \rho \) always have the same sign. Can you guess why? There is, in fact, a general result, which we will mention soon.

Now, we are left to calculate \(Var(e^X) = (Var(e^Y)\).

\(Var(e^X) = E(e^{2X}) – (E(e^{X}))^2 = M_X(2) – M_X(1)^2 = e^{\frac{4}{2}} – (e^{\frac{1}{2}})^2 = e^2 – e^1 = Var(e^Y)\).

Therefore, \( Cor(e^X, e^Y) = \frac{e(e^{\rho} – 1)}{e^2 – e^1} =\frac{e^{\rho} – 1}{e – 1} \).

Observe that the mininum correlation of \(e^X\) and \(e^Y\) is \(\frac{-1}{e}\).

Back to the important observation

\( Cor(e^X, e^Y)\) and \(Cor(X,Y) = \rho \) always have the same sign. Why is this true?

Because, \( f(x) = e^x\) is an increasing function. So, if \(X\) and \(Y\) are positively correlated then, as \(X\) increases, \(Y\) also increases in general, hence, \(e^X\) also increases along with \(e^Y\) hence, the result, which is quite intuitive.

Observe that in place of \( f(x) = e^x \) if we would have taken, any increasing function \(f(x)\), this will be the case. Can you prove it?

Research Problem of the day ( Is the following true? )

Let \(f(x)\) be an increasing function of \(x\), then

  • \( Cor(f(X), f(Y)) > 0 \iff Cor(X, Y) > 0\)
  • \(Cor(f(X), f(Y)) = 0 \iff Cor(X, Y) = 0\)
  • \( Cor(f(X), f(Y)) < 0 \iff Cor(X, Y) < 0\)