Monthly Archives: February 2013

Another Example on Calculating Covariance

In a previous post called An Example on Calculating Covariance, we calculated the covariance and correlation coefficient of a discrete joint distribution where the conditional mean E(Y \lvert X=x) is a linear function of x. In this post, we give examples in the continuous case. Problem A is worked out and Problem B is left as exercise.

The examples presented here are also found in the post called Another Example of a Joint Distribution. Some of the needed calculations are found in this previous post.

____________________________________________________________________

Problem A
Let X be a random variable with the density function f_X(x)=\alpha^2 \ x \ e^{-\alpha x} where x>0. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Calculate the density function, the mean and the variance for the conditional variable Y \lvert X=x.
  2. Calculate the density function, the mean and the variance for the conditional variable X \lvert Y=y.
  3. Use the fact that the conditional mean E(Y \lvert X=x) is a linear function of x to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

Problem B
Let X be a random variable with the density function f_X(x)=4 \ x^3 where 0<x<1. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Calculate the density function, the mean and the variance for the conditional variable Y \lvert X=x.
  2. Calculate the density function, the mean and the variance for the conditional variable X \lvert Y=y.
  3. Use the fact that the conditional mean E(Y \lvert X=x) is a linear function of x to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

____________________________________________________________________

Background Results

Here’s the idea behind the calculation of correlation coefficient in this post. Suppose X and Y are jointly distributed. When the conditional mean E(Y \lvert X=x) is a linear function of x, that is, E(Y \lvert X=x)=a+bx for some constants a and b, it can be written as the following:

    \displaystyle E(Y \lvert X=x)=\mu_Y + \rho \ \frac{\sigma_Y}{\sigma_X} \ (x - \mu_X)

Here, \mu_X=E(X) and \mu_Y=E(Y). The notations \sigma_X and \sigma_Y refer to the standard deviation of X and Y, respectively. Of course, \rho refers to the correlation coefficient in the joint distribution of X and Y and is defined by:

    \displaystyle \rho=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y}

where Cov(X,Y) is the covariance of X and Y and is defined by

    Cov(X,Y)=E[(X-\mu_X) \ (Y-\mu_Y)]

or equivalently by Cov(X,Y)=E(X,Y)-\mu_X \mu_Y.

Just to make it clear, in the joint distribution of X and Y, if the conditional mean E(X \lvert Y=y) is a linear function of y, then we have:

    \displaystyle E(X \lvert Y=y)=\mu_X + \rho \ \frac{\sigma_X}{\sigma_Y} \ (y - \mu_Y)

____________________________________________________________________

Discussion of Problem A

Problem A-1

Since for each x, Y \lvert X=x has the uniform distribution U(0,x), we have the following:

    \displaystyle f_{Y \lvert X=x}=\frac{1}{x} for x>0

    \displaystyle E(Y \lvert X=x)=\frac{x}{2}

    \displaystyle Var(Y \lvert X=x)=\frac{x^2}{12}

Problem A-2

In a previous post called Another Example of a Joint Distribution, the joint density function of X and Y is calculated to be: f_{X,Y}(x,y)=\alpha^2 \ e^{-\alpha x}. In the same post, the marginal density of Y is calculated to be: f_Y(y)=\alpha e^{-\alpha y} (exponentially distributed). Thus we have:

    \displaystyle \begin{aligned} f_{X \lvert Y=y}(x \lvert y)&=\frac{f_{X,Y}(x,y)}{f_Y(y)} \\&=\frac{\alpha^2 \ e^{-\alpha x}}{\alpha \ e^{-\alpha \ y}} \\&=\alpha \ e^{-\alpha \ (x-y)} \text{ where } y<x<\infty \end{aligned}

Thus the conditional variable X \lvert Y=y has an exponential distribution that is shifted to the right by the amount y. Thus we have:

    \displaystyle E(X \lvert Y=y)=\frac{1}{\alpha}+y

    \displaystyle Var(Y \lvert X=x)=\frac{1}{\alpha^2}

Problem A-3

To compute the covariance Cov(X,Y), one approach is to use the definition indicated above (to see this calculation, see Another Example of a Joint Distribution). Here we use the idea that the conditional mean \displaystyle E(Y \lvert X=x) is linear in x. From the previous post Another Example of a Joint Distribution, we have:

    \displaystyle \sigma_X=\frac{\sqrt{2}}{\alpha}

    \displaystyle \sigma_Y=\frac{1}{\alpha}

Plugging in \sigma_X and \sigma_Y, we have the following calculation:

    \displaystyle \rho \ \frac{\sigma_Y}{\sigma_X}=\frac{1}{2}

    \displaystyle \rho = \frac{\sigma_X}{\sigma_Y} \times \frac{1}{2}=\frac{\sqrt{2}}{2}=\frac{1}{\sqrt{2}}=0.7071

    \displaystyle Cov(X,Y)=\rho \ \sigma_X \ \sigma_Y=\frac{1}{\alpha^2}

____________________________________________________________________

Answers for Problem B

Problem B-1

    \displaystyle E(Y \lvert X=x)=\frac{x}{2}

    \displaystyle Var(Y \lvert X=x)=\frac{x^2}{12}

Problem B-2

    \displaystyle f_{X \lvert Y=y}(x \lvert y)=\frac{4 \ x^2}{1-y^3} where 0<y<1 and y<x<1

Problem B-3

    \displaystyle \rho=\frac{\sqrt{3}}{2 \ \sqrt{7}}=0.3273268

    \displaystyle Cov(X,Y)=\frac{1}{75}

____________________________________________________________________
\copyright \ 2013

Advertisements

Another Example of a Joint Distribution

In an earlier post called An Example of a Joint Distribution, we worked a problem involving a joint distribution that is constructed from taking product of a conditional distribution and a marginial distribution (both discrete distributions). In this post, we work on similar problems for the continuous case. We work problem A. Problem B is left as exercises.

_________________________________________________________________

Problem A
Let X be a random variable with the density function f_X(x)=\alpha^2 \ x \ e^{-\alpha x} where x>0. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Discuss the joint density function for X and Y.
  2. Calculate the marginal distribution of X, in particular the mean and variance.
  3. Calculate the marginal distribution of Y, in particular, the density function, mean and variance.
  4. Use the joint density in part A-1 to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

_________________________________________________________________

Problem B
Let X be a random variable with the density function f_X(x)=4 \ x^3 where 0<x<1. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Discuss the joint density function for X and Y.
  2. Calculate the marginal distribution of X, in particular the mean and variance.
  3. Calculate the marginal distribution of Y, in particular, the density function, mean and variance.
  4. Use the joint density in part B-1 to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

_________________________________________________________________

Discussion of Problem A

Problem A-1

The support of the joint density function f_{X,Y}(x,y) is the unbounded lower triangle in the xy-plane (see the shaded region in green in the figure below).

Figure 1

The unbounded green region consists of vertical lines: for each x>0, y ranges from 0 to x (the red vertical line in the figure below is one such line).

Figure 2

For each point (x,y) in each vertical line, we assign a density value f_{X,Y}(x,y) which is a positive number. Taken together these density values sum to 1.0 and describe the behavior of the variables X and Y across the green region. If a realized value of X is x, then the conditional density function of Y \lvert X=x is:

    \displaystyle f_{Y \lvert X=x}(y \lvert x)=\frac{f_{X,Y}(x,y)}{f_X(x)}

Thus we have f_{X,Y}(x,y) = f_{Y \lvert X=x}(y \lvert x) \times f_X(x). In our problem at hand, the joint density function is:

    \displaystyle \begin{aligned} f_{X,Y}(x,y)&=f_{Y \lvert X=x}(y \lvert x) \times f_X(x) \\&=\frac{1}{x} \times \alpha^2 \ x \ e^{-\alpha x} \\&=\alpha^2 \ e^{-\alpha x}  \end{aligned}

As indicated above, the support of f_{X,Y}(x,y) is the region x>0 and 0<y<x (the region shaded green in the above figures).

Problem A-2

The unconditional density function of X is f_X(x)=\alpha^2 \ x \ e^{-\alpha x} (given above in the problem) is the density function of the sum of two independent exponential variables with the common density f(x)=\alpha e^{-\alpha x} (see this blog post for the derivation using convolution method). Since X is the independent sum of two identical exponential distributions, the mean and variance of X is twice that of the same item of the exponential distribution. We have:

    \displaystyle E(X)=\frac{2}{\alpha}

    \displaystyle Var(X)=\frac{2}{\alpha^2}

Problem A-3

To find the marginal density of Y, for each applicable y, we need to sum out the x. According to the following figure, for each y, we sum out all x values in a horizontal line such that y<x<\infty (see the blue horizontal line).

Figure 3

Thus we have:

    \displaystyle \begin{aligned} f_Y(y)&=\int_y^\infty f_{X,Y}(x,y) \ dy \ dx \\&=\int_y^\infty \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\alpha \int_y^\infty \alpha \ e^{-\alpha x} \ dy \ dx \\&= \alpha e^{-\alpha y}  \end{aligned}

Thus the marginal distribution of Y is an exponential distribution. The mean and variance of Y are:

    \displaystyle E(Y)=\frac{1}{\alpha}

    \displaystyle Var(Y)=\frac{1}{\alpha^2}

Problem A-4

The covariance of X and Y is defined as Cov(X,Y)=E[(X-\mu_X) (Y-\mu_Y)], which is equivalent to:

    \displaystyle Cov(X,Y)=E(X Y)-\mu_X \mu_Y

where \mu_X=E(X) and \mu_Y=E(Y). Knowing the joint density f_{X,Y}(x,y), we can calculate Cov(X,Y) directly. We have:

    \displaystyle \begin{aligned} E(X Y)&=\int_0^\infty \int_0^x  xy \ f_{X,Y}(x,y) \ dy \ dx \\&=\int_0^\infty \int_0^x xy \ \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\int_0^\infty \frac{\alpha^2}{2} \ x^3 \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \int_0^\infty \frac{\alpha^4}{3!} \ x^{4-1} \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \end{aligned}

Note that the last integrand in the last integral in the above derivation is that of a Gamma distribution (hence the integral is 1.0). Now the covariance of X and Y is:

    \displaystyle Cov(X,Y)=\frac{3}{\alpha^2}-\frac{2}{\alpha} \frac{1}{\alpha}=\frac{1}{\alpha^2}

The following is the calculation of the correlation coefficient:

    \displaystyle \begin{aligned} \rho&=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y} = \frac{\displaystyle \frac{1}{\alpha^2}}{\displaystyle \frac{\sqrt{2}}{\alpha} \ \frac{1}{\alpha}} \\&=\frac{1}{\sqrt{2}} = 0.7071 \end{aligned}

Even without the calculation of \rho, we know that X and Y are positively and quite strongly correlated. The conditional distribution of Y \lvert X=x is U(0,x) which increases with x. The calculation of Cov(X,Y) and \rho confirms our observation.

_________________________________________________________________

Answers for Problem B

Problem B-1

    \displaystyle f_{X,Y}(x,y)=4 \ x^2 where x>0, and 0<y<x.

Problem B-2

    \displaystyle E(X)=\frac{4}{5}
    \displaystyle Var(X)=\frac{2}{75}

Problem B-3

    \displaystyle f_Y(y)=\frac{4}{3} \ (1- y^3)

    \displaystyle E(Y)=\frac{2}{5}

    \displaystyle Var(Y)=\frac{14}{225}

Problem B-4

    \displaystyle Cov(X,Y)=\frac{1}{75}

    \displaystyle \rho = \frac{\sqrt{3}}{2 \sqrt{7}}=0.327327

_________________________________________________________________

Mixing Bowls of Balls

We present problems involving mixture distributions in the context of choosing bowls of balls, as well as related problems involving Bayes’ formula. Problem 1a and Problem 1b are discussed. Problem 2a and Problem 2b are left as exercises.

____________________________________________________________

Problem 1a
There are two identical looking bowls. Let’s call them Bowl 1 and Bowl 2. In Bowl 1, there are 1 red ball and 4 white balls. In Bowl 2, there are 4 red balls and 1 white ball. One bowl is selected at random and its identify is kept from you. From the chosen bowl, you randomly select 5 balls (one at a time, putting it back before picking another one). What is the expected number of red balls in the 5 selected balls? What the variance of the number of red balls?

Problem 1b
Use the same information in Problem 1a. Suppose there are 3 red balls in the 5 selected balls. What is the probability that the unknown chosen bowl is Bowl 1? What is the probability that the unknown chosen bowl is Bowl 2?

____________________________________________________________

Problem 2a
There are three identical looking bowls. Let’s call them Bowl 1, Bowl 2 and Bowl 3. Bowl 1 has 1 red ball and 9 white balls. Bowl 2 has 4 red balls and 6 white balls. Bowl 3 has 6 red balls and 4 white balls. A bowl is chosen according to the following probabilities:

\displaystyle \begin{aligned}\text{Probabilities:} \ \ \ \ \ &P(\text{Bowl 1})=0.6 \\&P(\text{Bowl 2})=0.3 \\&P(\text{Bowl 3})=0.1 \end{aligned}

The bowl is chosen so that its identity is kept from you. From the chosen bowl, 5 balls are selected sequentially with replacement. What is the expected number of red balls in the 5 selected balls? What is the variance of the number of red balls?

Problem 2b
Use the same information in Problem 2a. Given that there are 4 red balls in the 5 selected balls, what is the probability that the chosen bowl is Bowl i, where i = 1,2,3?

____________________________________________________________
Solution – Problem 1a

Problem 1a is a mixture of two binomial distributions and is similar to Problem 1 in the previous post Mixing Binomial Distributions. Let X be the number of red balls in the 5 balls chosen from the unknown bowl. The following is the probability function:

    \displaystyle P(X=x)=0.5 \binom{5}{x} \biggl[\frac{1}{5}\biggr]^x \biggl[\frac{4}{5}\biggr]^{4-x}+0.5 \binom{5}{x} \biggl[\frac{4}{5}\biggr]^x \biggl[\frac{1}{5}\biggr]^{4-x}

where X=0,1,2,3,4,5.

The above probability function is the weighted average of two conditional binomial distributions (with equal weights). Thus the mean (first moment) and the second moment of X would be the weighted averages of the two same items of the conditional distributions. We have:

    \displaystyle E(X)=0.5 \biggl[ 5 \times \frac{1}{5} \biggr] + 0.5 \biggl[ 5 \times \frac{4}{5} \biggr] =\frac{5}{2}
    \displaystyle E(X^2)=0.5 \biggl[ 5 \times \frac{1}{5} \times \frac{4}{5} +\biggl( 5 \times \frac{1}{5} \biggr)^2 \biggr]

      \displaystyle + 0.5 \biggl[ 5 \times \frac{4}{5} \times \frac{1}{5} +\biggl( 5 \times \frac{4}{5} \biggr)^2 \biggr]=\frac{93}{10}
    \displaystyle Var(X)=\frac{93}{10} - \biggl( \frac{5}{2} \biggr)^2=\frac{61}{20}=3.05

See Mixing Binomial Distributions for a more detailed explanation of the calculation.

____________________________________________________________

Solution – Problem 1b
As above, let X be the number of red balls in the 5 selected balls. The probability P(X=3) must account for the two bowls. Thus it is obtained by mixing two binomial probabilities:

    \displaystyle P(X=3)=\frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2+\frac{1}{2} \binom{5}{3} \biggl(\frac{4}{5}\biggr)^3 \biggl(\frac{1}{5}\biggr)^2

The following is the conditional probability P(\text{Bowl 1} \lvert X=3):

    \displaystyle \begin{aligned} P(\text{Bowl 1} \lvert X=3)&=\frac{\displaystyle \frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2}{P(X=3)} \\&=\frac{16}{16+64} \\&=\frac{1}{5} \end{aligned}

Thus \displaystyle P(\text{Bowl 1} \lvert X=3)=\frac{4}{5}

____________________________________________________________

Answers for Problem 2

Problem 2a
Let X be the number of red balls in the 5 balls chosen random from the unknown bowl.

    E(X)=1.2
    Var(X)=1.56

Problem 2b

    \displaystyle P(\text{Bowl 1} \lvert X=4)=\frac{27}{4923}=0.0055

    \displaystyle P(\text{Bowl 2} \lvert X=4)=\frac{2304}{4923}=0.4680

    \displaystyle P(\text{Bowl 3} \lvert X=4)=\frac{2592}{4923}=0.5265