Conditional Mean | Probability and Statistics Problem Solve

Probem 1
Let $X$ be the value of one roll of a fair die. If the value of the die is $x$ , we are given that $Y \lvert X=x$ has a binomial distribution with $n=x$ and $p=\frac{1}{4}$ (we use the notation $Y \lvert X=x \sim \text{binom}(x,\frac{1}{4})$ ).

Compute the mean and variance of $X$ .
Compute the mean and variance of $Y$ .
Compute the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$ .

Probem 2
Let $X$ be the value of one roll of a fair die. If the value of the die is $x$ , we are given that $Y \lvert X=x$ has a binomial distribution with $n=x$ and $p=\frac{1}{2}$ (we use the notation $Y \lvert X=x \sim \text{binom}(x,\frac{1}{2})$ ).

Compute the mean and variance of $X$ .
Compute the mean and variance of $Y$ .
Compute the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$ .

Problem 2 is left as exercise.

_________________________________________________________
Discussion of Problem 1

The joint variables $X$ and $Y$ are identical to the ones in this previous post. However, we do not plan on following the approach in the previous, which is to first find the probability functions for the joint distribution and then the marginal distribution of $Y$ . The calculation of covariance in Problem 1.3 can be very tedious by taking this approach.

Problem 1.1
We start with the easiest part, which is the random variable $X$ (the roll of the die). The variance is computed by $Var(X)=E(X^2)-E(X)^2$ .

$\displaystyle (1) \ \ \ \ \ E(X)=\frac{1}{6} \biggl[1+2+3+4+5+6 \biggr]=\frac{21}{6}=3.5$

$\displaystyle (2) \ \ \ \ \ E(X^2)=\frac{1}{6} \biggl[1^2+2^2+3^2+4^2+5^2+6^2 \biggr]=\frac{91}{6}$

$\displaystyle (3) \ \ \ \ \ Var(X)=\frac{91}{6}-\biggl[\frac{21}{6}\biggr]^2=\frac{105}{36}=\frac{35}{12}$

Problem 1.2

We now compute the mean and variance of $Y$ . The calculation of finding the joint distribution and then finding the marginal distribution of $Y$ is tedious and has been done in this previous post. We do not take this approach here. Instead, we find the unconditional mean $E(Y)$ by weighting the conditional mean $E(Y \lvert X=x)$ . The weights are the probabilities $P(X=x)$ . The following is the idea.

$\displaystyle \begin{aligned}(4) \ \ \ \ \ E(Y)&=E_X[E(Y \lvert X=x)] \\&= E(Y \lvert X=1) \times P(X=1) \\&+ E(Y \lvert X=2) \times P(X=2)\\&+ E(Y \lvert X=3) \times P(X=3) \\&+ E(Y \lvert X=4) \times P(X=4) \\&+E(Y \lvert X=5) \times P(X=5) \\&+E(Y \lvert X=6) \times P(X=6) \end{aligned}$

We have $P(X=x)=\frac{1}{6}$ for each $x$ . Before we do the weighting, we need to have some items about the conditional distribution $Y \lvert X=x$ . Since $Y \lvert X=x$ has a binomial distribution, we have:

$\displaystyle (5) \ \ \ \ \ E(Y \lvert X=x)=\frac{1}{4} \ x$

$\displaystyle (6) \ \ \ \ \ Var(Y \lvert X=x)=\frac{1}{4} \ \frac{3}{4} \ x=\frac{3}{16} \ x$

For any random variable $W$ , $Var(W)=E(W^2)-E(W)^2$ and $E(W^2)=Var(W)+E(W)^2$ . The following is the second moment of $Y \lvert X=x$ , which is needed in calculating the unconditional variance $Var(Y)$ .

$\displaystyle \begin{aligned}(7) \ \ \ \ \ E(Y^2 \lvert X=x)&=\frac{3}{16} \ x+\biggl[\frac{1}{4} \ x \biggr]^2 \\&=\frac{3x}{16}+\frac{x^2}{16} \\&=\frac{3x+x^2}{16} \end{aligned}$

We can now do the weighting to get the items of the variable $Y$ .

$\displaystyle \begin{aligned}(8) \ \ \ \ \ E(Y)&=\frac{1}{6} \biggl[\frac{1}{4} +\frac{2}{4}+\frac{3}{4}+ \frac{4}{4}+\frac{5}{4}+\frac{6}{4}\biggr] \\&=\frac{7}{8} \\&=0.875 \end{aligned}$

$\displaystyle \begin{aligned}(9) \ \ \ \ \ E(Y^2)&=\frac{1}{6} \biggl[\frac{3(1)+1^2}{16} +\frac{3(2)+2^2}{16}+\frac{3(3)+3^2}{16} \\&+ \frac{3(4)+4^2}{16}+\frac{3(5)+5^2}{16}+\frac{3(6)+6^2}{16}\biggr] \\&=\frac{154}{96} \\&=\frac{77}{48} \end{aligned}$

$\displaystyle \begin{aligned}(10) \ \ \ \ \ Var(Y)&=E(Y^2)-E(Y)^2 \\&=\frac{77}{48}-\biggl[\frac{7}{8}\biggr]^2 \\&=\frac{161}{192} \\&=0.8385 \end{aligned}$

Problem 1.3

The following is the definition of covariance of $X$ and $Y$ :

$\displaystyle (11) \ \ \ \ \ Cov(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]$

where $\mu_X=E(X)$ and $\mu_Y=E(Y)$ .

The definition $(11)$ can be simplified as:

$\displaystyle (12) \ \ \ \ \ Cov(X,Y)=E[XY]-E[X] E[Y]$

To compute $E[XY]$ , we can use the joint probability function of $X$ and $Y$ to compute this expectation. But this is tedious. Anyone who wants to try can go to this previous post to obtain the joint distribution.

Note that the conditional mean $E(Y \lvert X=x)=\frac{x}{4}$ is a linear function of $x$ . It is a well known result in probability and statistics that whenever a conditional mean $E(Y \lvert X=x)$ is a linear function of $x$ , the conditional mean can be written as:

$\displaystyle (13) \ \ \ \ \ E(Y \lvert X=x)=\mu_Y+\rho \ \frac{\sigma_Y}{\sigma_X} \ (x-\mu_X)$

where $\mu$ is the mean of the respective variable, $\sigma$ is the standard deviation of the respective variable and $\rho$ is the correlation coefficient. The following relates the correlation coefficient with the covariance.

$\displaystyle (14) \ \ \ \ \ \rho=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y}$

Comparing $(5)$ and $(13)$ , we have $\displaystyle \rho \frac{\sigma_Y}{\sigma_X}=\frac{1}{4}$ and

$\displaystyle (15) \ \ \ \ \ \rho = \frac{\sigma_X}{4 \ \sigma_Y}$

Equating $(14)$ and $(15)$ , we have $Cov(X,Y)=\frac{\sigma_X^2}{4}$ . Thus we deduce that $Cov(X,Y)$ is one-fourth of the variance of $X$ . Using $(3)$ , we have:

$\displaystyle (16) \ \ \ \ \ Cov(X,Y) = \frac{1}{4} \times \frac{35}{12}=\frac{35}{48}=0.72917$

Plug in all the items of $(3)$ , $(10)$ , and $(16)$ into $(14)$ , we obtained $\rho=0.46625$ . Both $\rho$ and $Cov(X,Y)$ are positive, an indication that both variables move together. When one increases, the other variable also increases. Thus makes sense based on the definition of the variables. For example, when the value of the die is large, the number of trials of $Y$ is greater (hence a larger mean).

Probability and Statistics Problem Solve

Tag Archives: Conditional Mean

An Example on Calculating Covariance

Follow “Probability and Statistics Problem Solve”