An Example on Calculating Covariance

Probem 1
Let X be the value of one roll of a fair die. If the value of the die is x, we are given that Y \lvert X=x has a binomial distribution with n=x and p=\frac{1}{4} (we use the notation Y \lvert X=x \sim \text{binom}(x,\frac{1}{4})).

  1. Compute the mean and variance of X.
  2. Compute the mean and variance of Y.
  3. Compute the covariance Cov(X,Y) and the correlation coefficient \rho.

Probem 2
Let X be the value of one roll of a fair die. If the value of the die is x, we are given that Y \lvert X=x has a binomial distribution with n=x and p=\frac{1}{2} (we use the notation Y \lvert X=x \sim \text{binom}(x,\frac{1}{2})).

  1. Compute the mean and variance of X.
  2. Compute the mean and variance of Y.
  3. Compute the covariance Cov(X,Y) and the correlation coefficient \rho.

Problem 2 is left as exercise.

_________________________________________________________
Discussion of Problem 1

The joint variables X and Y are identical to the ones in this previous post. However, we do not plan on following the approach in the previous, which is to first find the probability functions for the joint distribution and then the marginal distribution of Y. The calculation of covariance in Problem 1.3 can be very tedious by taking this approach.

Problem 1.1
We start with the easiest part, which is the random variable X (the roll of the die). The variance is computed by Var(X)=E(X^2)-E(X)^2.

\displaystyle (1) \ \ \ \ \ E(X)=\frac{1}{6} \biggl[1+2+3+4+5+6 \biggr]=\frac{21}{6}=3.5

\displaystyle (2) \ \ \ \ \ E(X^2)=\frac{1}{6} \biggl[1^2+2^2+3^2+4^2+5^2+6^2 \biggr]=\frac{91}{6}

\displaystyle (3) \ \ \ \ \ Var(X)=\frac{91}{6}-\biggl[\frac{21}{6}\biggr]^2=\frac{105}{36}=\frac{35}{12}

Problem 1.2

We now compute the mean and variance of Y. The calculation of finding the joint distribution and then finding the marginal distribution of Y is tedious and has been done in this previous post. We do not take this approach here. Instead, we find the unconditional mean E(Y) by weighting the conditional mean E(Y \lvert X=x). The weights are the probabilities P(X=x). The following is the idea.

\displaystyle \begin{aligned}(4) \ \ \ \ \  E(Y)&=E_X[E(Y \lvert X=x)] \\&= E(Y \lvert X=1) \times P(X=1) \\&+ E(Y \lvert X=2) \times P(X=2)\\&+ E(Y \lvert X=3)  \times P(X=3) \\&+ E(Y \lvert X=4)  \times P(X=4) \\&+E(Y \lvert X=5)  \times P(X=5) \\&+E(Y \lvert X=6)  \times P(X=6) \end{aligned}

We have P(X=x)=\frac{1}{6} for each x. Before we do the weighting, we need to have some items about the conditional distribution Y \lvert X=x. Since Y \lvert X=x has a binomial distribution, we have:

\displaystyle (5) \ \ \ \ \ E(Y \lvert X=x)=\frac{1}{4} \ x

\displaystyle (6) \ \ \ \ \ Var(Y \lvert X=x)=\frac{1}{4} \ \frac{3}{4} \ x=\frac{3}{16} \ x

For any random variable W, Var(W)=E(W^2)-E(W)^2 and E(W^2)=Var(W)+E(W)^2. The following is the second moment of Y \lvert X=x, which is needed in calculating the unconditional variance Var(Y).

\displaystyle \begin{aligned}(7) \ \ \ \ \ E(Y^2 \lvert X=x)&=\frac{3}{16} \ x+\biggl[\frac{1}{4} \ x \biggr]^2 \\&=\frac{3x}{16}+\frac{x^2}{16} \\&=\frac{3x+x^2}{16}  \end{aligned}

We can now do the weighting to get the items of the variable Y.

\displaystyle \begin{aligned}(8) \ \ \ \ \  E(Y)&=\frac{1}{6} \biggl[\frac{1}{4} +\frac{2}{4}+\frac{3}{4}+ \frac{4}{4}+\frac{5}{4}+\frac{6}{4}\biggr] \\&=\frac{7}{8} \\&=0.875  \end{aligned}

\displaystyle \begin{aligned}(9) \ \ \ \ \  E(Y^2)&=\frac{1}{6} \biggl[\frac{3(1)+1^2}{16} +\frac{3(2)+2^2}{16}+\frac{3(3)+3^2}{16} \\&+ \frac{3(4)+4^2}{16}+\frac{3(5)+5^2}{16}+\frac{3(6)+6^2}{16}\biggr] \\&=\frac{154}{96} \\&=\frac{77}{48}  \end{aligned}

\displaystyle \begin{aligned}(10) \ \ \ \ \  Var(Y)&=E(Y^2)-E(Y)^2 \\&=\frac{77}{48}-\biggl[\frac{7}{8}\biggr]^2 \\&=\frac{161}{192} \\&=0.8385 \end{aligned}

Problem 1.3

The following is the definition of covariance of X and Y:

\displaystyle (11) \ \ \ \ \ Cov(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]

where \mu_X=E(X) and \mu_Y=E(Y).

The definition (11) can be simplified as:

\displaystyle (12) \ \ \ \ \ Cov(X,Y)=E[XY]-E[X] E[Y]

To compute E[XY], we can use the joint probability function of X and Y to compute this expectation. But this is tedious. Anyone who wants to try can go to this previous post to obtain the joint distribution.

Note that the conditional mean E(Y \lvert X=x)=\frac{x}{4} is a linear function of x. It is a well known result in probability and statistics that whenever a conditional mean E(Y \lvert X=x) is a linear function of x, the conditional mean can be written as:

\displaystyle (13) \ \ \ \ \ E(Y \lvert X=x)=\mu_Y+\rho \ \frac{\sigma_Y}{\sigma_X} \ (x-\mu_X)

where \mu is the mean of the respective variable, \sigma is the standard deviation of the respective variable and \rho is the correlation coefficient. The following relates the correlation coefficient with the covariance.

\displaystyle (14) \ \ \ \ \ \rho=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y}

Comparing (5) and (13), we have \displaystyle \rho \frac{\sigma_Y}{\sigma_X}=\frac{1}{4} and

\displaystyle (15) \ \ \ \ \  \rho = \frac{\sigma_X}{4 \ \sigma_Y}

Equating (14) and (15), we have Cov(X,Y)=\frac{\sigma_X^2}{4}. Thus we deduce that Cov(X,Y) is one-fourth of the variance of X. Using (3), we have:

\displaystyle (16) \ \ \ \ \  Cov(X,Y) = \frac{1}{4} \times \frac{35}{12}=\frac{35}{48}=0.72917

Plug in all the items of (3), (10), and (16) into (14), we obtained \rho=0.46625. Both \rho and Cov(X,Y) are positive, an indication that both variables move together. When one increases, the other variable also increases. Thus makes sense based on the definition of the variables. For example, when the value of the die is large, the number of trials of Y is greater (hence a larger mean).

About these ads

Tagged: , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: