Tag Archives: Bayes’ Formula

Another Example of a Joint Distribution

In an earlier post called An Example of a Joint Distribution, we worked a problem involving a joint distribution that is constructed from taking product of a conditional distribution and a marginial distribution (both discrete distributions). In this post, we work on similar problems for the continuous case. We work problem A. Problem B is left as exercises.

_________________________________________________________________

Problem A
Let X be a random variable with the density function f_X(x)=\alpha^2 \ x \ e^{-\alpha x} where x>0. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Discuss the joint density function for X and Y.
  2. Calculate the marginal distribution of X, in particular the mean and variance.
  3. Calculate the marginal distribution of Y, in particular, the density function, mean and variance.
  4. Use the joint density in part A-1 to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

_________________________________________________________________

Problem B
Let X be a random variable with the density function f_X(x)=4 \ x^3 where 0<x<1. For each realized value X=x, the conditional variable Y \lvert X=x is uniformly distributed over the interval (0,x), denoted symbolically by Y \lvert X=x \sim U(0,x). Obtain solutions for the following:

  1. Discuss the joint density function for X and Y.
  2. Calculate the marginal distribution of X, in particular the mean and variance.
  3. Calculate the marginal distribution of Y, in particular, the density function, mean and variance.
  4. Use the joint density in part B-1 to calculate the covariance Cov(X,Y) and the correlation coefficient \rho.

_________________________________________________________________

Discussion of Problem A

Problem A-1

The support of the joint density function f_{X,Y}(x,y) is the unbounded lower triangle in the xy-plane (see the shaded region in green in the figure below).

Figure 1

The unbounded green region consists of vertical lines: for each x>0, y ranges from 0 to x (the red vertical line in the figure below is one such line).

Figure 2

For each point (x,y) in each vertical line, we assign a density value f_{X,Y}(x,y) which is a positive number. Taken together these density values sum to 1.0 and describe the behavior of the variables X and Y across the green region. If a realized value of X is x, then the conditional density function of Y \lvert X=x is:

    \displaystyle f_{Y \lvert X=x}(y \lvert x)=\frac{f_{X,Y}(x,y)}{f_X(x)}

Thus we have f_{X,Y}(x,y) = f_{Y \lvert X=x}(y \lvert x) \times f_X(x). In our problem at hand, the joint density function is:

    \displaystyle \begin{aligned} f_{X,Y}(x,y)&=f_{Y \lvert X=x}(y \lvert x) \times f_X(x) \\&=\frac{1}{x} \times \alpha^2 \ x \ e^{-\alpha x} \\&=\alpha^2 \ e^{-\alpha x}  \end{aligned}

As indicated above, the support of f_{X,Y}(x,y) is the region x>0 and 0<y<x (the region shaded green in the above figures).

Problem A-2

The unconditional density function of X is f_X(x)=\alpha^2 \ x \ e^{-\alpha x} (given above in the problem) is the density function of the sum of two independent exponential variables with the common density f(x)=\alpha e^{-\alpha x} (see this blog post for the derivation using convolution method). Since X is the independent sum of two identical exponential distributions, the mean and variance of X is twice that of the same item of the exponential distribution. We have:

    \displaystyle E(X)=\frac{2}{\alpha}

    \displaystyle Var(X)=\frac{2}{\alpha^2}

Problem A-3

To find the marginal density of Y, for each applicable y, we need to sum out the x. According to the following figure, for each y, we sum out all x values in a horizontal line such that y<x<\infty (see the blue horizontal line).

Figure 3

Thus we have:

    \displaystyle \begin{aligned} f_Y(y)&=\int_y^\infty f_{X,Y}(x,y) \ dy \ dx \\&=\int_y^\infty \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\alpha \int_y^\infty \alpha \ e^{-\alpha x} \ dy \ dx \\&= \alpha e^{-\alpha y}  \end{aligned}

Thus the marginal distribution of Y is an exponential distribution. The mean and variance of Y are:

    \displaystyle E(Y)=\frac{1}{\alpha}

    \displaystyle Var(Y)=\frac{1}{\alpha^2}

Problem A-4

The covariance of X and Y is defined as Cov(X,Y)=E[(X-\mu_X) (Y-\mu_Y)], which is equivalent to:

    \displaystyle Cov(X,Y)=E(X Y)-\mu_X \mu_Y

where \mu_X=E(X) and \mu_Y=E(Y). Knowing the joint density f_{X,Y}(x,y), we can calculate Cov(X,Y) directly. We have:

    \displaystyle \begin{aligned} E(X Y)&=\int_0^\infty \int_0^x  xy \ f_{X,Y}(x,y) \ dy \ dx \\&=\int_0^\infty \int_0^x xy \ \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\int_0^\infty \frac{\alpha^2}{2} \ x^3 \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \int_0^\infty \frac{\alpha^4}{3!} \ x^{4-1} \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \end{aligned}

Note that the last integrand in the last integral in the above derivation is that of a Gamma distribution (hence the integral is 1.0). Now the covariance of X and Y is:

    \displaystyle Cov(X,Y)=\frac{3}{\alpha^2}-\frac{2}{\alpha} \frac{1}{\alpha}=\frac{1}{\alpha^2}

The following is the calculation of the correlation coefficient:

    \displaystyle \begin{aligned} \rho&=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y} = \frac{\displaystyle \frac{1}{\alpha^2}}{\displaystyle \frac{\sqrt{2}}{\alpha} \ \frac{1}{\alpha}} \\&=\frac{1}{\sqrt{2}} = 0.7071 \end{aligned}

Even without the calculation of \rho, we know that X and Y are positively and quite strongly correlated. The conditional distribution of Y \lvert X=x is U(0,x) which increases with x. The calculation of Cov(X,Y) and \rho confirms our observation.

_________________________________________________________________

Answers for Problem B

Problem B-1

    \displaystyle f_{X,Y}(x,y)=4 \ x^2 where x>0, and 0<y<x.

Problem B-2

    \displaystyle E(X)=\frac{4}{5}
    \displaystyle Var(X)=\frac{2}{75}

Problem B-3

    \displaystyle f_Y(y)=\frac{4}{3} \ (1- y^3)

    \displaystyle E(Y)=\frac{2}{5}

    \displaystyle Var(Y)=\frac{14}{225}

Problem B-4

    \displaystyle Cov(X,Y)=\frac{1}{75}

    \displaystyle \rho = \frac{\sqrt{3}}{2 \sqrt{7}}=0.327327

_________________________________________________________________

Advertisements

Mixing Bowls of Balls

We present problems involving mixture distributions in the context of choosing bowls of balls, as well as related problems involving Bayes’ formula. Problem 1a and Problem 1b are discussed. Problem 2a and Problem 2b are left as exercises.

____________________________________________________________

Problem 1a
There are two identical looking bowls. Let’s call them Bowl 1 and Bowl 2. In Bowl 1, there are 1 red ball and 4 white balls. In Bowl 2, there are 4 red balls and 1 white ball. One bowl is selected at random and its identify is kept from you. From the chosen bowl, you randomly select 5 balls (one at a time, putting it back before picking another one). What is the expected number of red balls in the 5 selected balls? What the variance of the number of red balls?

Problem 1b
Use the same information in Problem 1a. Suppose there are 3 red balls in the 5 selected balls. What is the probability that the unknown chosen bowl is Bowl 1? What is the probability that the unknown chosen bowl is Bowl 2?

____________________________________________________________

Problem 2a
There are three identical looking bowls. Let’s call them Bowl 1, Bowl 2 and Bowl 3. Bowl 1 has 1 red ball and 9 white balls. Bowl 2 has 4 red balls and 6 white balls. Bowl 3 has 6 red balls and 4 white balls. A bowl is chosen according to the following probabilities:

\displaystyle \begin{aligned}\text{Probabilities:} \ \ \ \ \ &P(\text{Bowl 1})=0.6 \\&P(\text{Bowl 2})=0.3 \\&P(\text{Bowl 3})=0.1 \end{aligned}

The bowl is chosen so that its identity is kept from you. From the chosen bowl, 5 balls are selected sequentially with replacement. What is the expected number of red balls in the 5 selected balls? What is the variance of the number of red balls?

Problem 2b
Use the same information in Problem 2a. Given that there are 4 red balls in the 5 selected balls, what is the probability that the chosen bowl is Bowl i, where i = 1,2,3?

____________________________________________________________
Solution – Problem 1a

Problem 1a is a mixture of two binomial distributions and is similar to Problem 1 in the previous post Mixing Binomial Distributions. Let X be the number of red balls in the 5 balls chosen from the unknown bowl. The following is the probability function:

    \displaystyle P(X=x)=0.5 \binom{5}{x} \biggl[\frac{1}{5}\biggr]^x \biggl[\frac{4}{5}\biggr]^{4-x}+0.5 \binom{5}{x} \biggl[\frac{4}{5}\biggr]^x \biggl[\frac{1}{5}\biggr]^{4-x}

where X=0,1,2,3,4,5.

The above probability function is the weighted average of two conditional binomial distributions (with equal weights). Thus the mean (first moment) and the second moment of X would be the weighted averages of the two same items of the conditional distributions. We have:

    \displaystyle E(X)=0.5 \biggl[ 5 \times \frac{1}{5} \biggr] + 0.5 \biggl[ 5 \times \frac{4}{5} \biggr] =\frac{5}{2}
    \displaystyle E(X^2)=0.5 \biggl[ 5 \times \frac{1}{5} \times \frac{4}{5} +\biggl( 5 \times \frac{1}{5} \biggr)^2 \biggr]

      \displaystyle + 0.5 \biggl[ 5 \times \frac{4}{5} \times \frac{1}{5} +\biggl( 5 \times \frac{4}{5} \biggr)^2 \biggr]=\frac{93}{10}
    \displaystyle Var(X)=\frac{93}{10} - \biggl( \frac{5}{2} \biggr)^2=\frac{61}{20}=3.05

See Mixing Binomial Distributions for a more detailed explanation of the calculation.

____________________________________________________________

Solution – Problem 1b
As above, let X be the number of red balls in the 5 selected balls. The probability P(X=3) must account for the two bowls. Thus it is obtained by mixing two binomial probabilities:

    \displaystyle P(X=3)=\frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2+\frac{1}{2} \binom{5}{3} \biggl(\frac{4}{5}\biggr)^3 \biggl(\frac{1}{5}\biggr)^2

The following is the conditional probability P(\text{Bowl 1} \lvert X=3):

    \displaystyle \begin{aligned} P(\text{Bowl 1} \lvert X=3)&=\frac{\displaystyle \frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2}{P(X=3)} \\&=\frac{16}{16+64} \\&=\frac{1}{5} \end{aligned}

Thus \displaystyle P(\text{Bowl 1} \lvert X=3)=\frac{4}{5}

____________________________________________________________

Answers for Problem 2

Problem 2a
Let X be the number of red balls in the 5 balls chosen random from the unknown bowl.

    E(X)=1.2
    Var(X)=1.56

Problem 2b

    \displaystyle P(\text{Bowl 1} \lvert X=4)=\frac{27}{4923}=0.0055

    \displaystyle P(\text{Bowl 2} \lvert X=4)=\frac{2304}{4923}=0.4680

    \displaystyle P(\text{Bowl 3} \lvert X=4)=\frac{2592}{4923}=0.5265

An Example of a Joint Distribution

Probem 1
Let X be the value of one roll of a fair die. If the value of the die is x, we are given that Y \lvert X=x has a binomial distribution with n=x and p=\frac{1}{4} (we use the notation Y \lvert X=x \sim \text{binom}(x,\frac{1}{4})).

  1. Discuss how the joint probability function P[X=x,Y=y] is computed for x=1,2,3,4,5,6 and y=0,1, \cdots, x.
  2. Compute the conditional binomial distributions Y \lvert X=x where x=1,2,3,4,5,6.
  3. Compute the marginal probability function of Y and the mean and variance of Y.
  4. Compute P(X=x \lvert Y=y) for all applicable x and y.

_____________________________________________________________
Discussion of Problem 1

Problem 2 is found at the end of the post.

Problem 1.1
This is an example of a joint distribution that is constructed from taking product of conditional distributions and a marginial distribution. The marginal distribution of X is a uniform distribution on the set \left\{1,2,3,4,5,6 \right\} (rolling a fiar die). Conditional of X=x, Y has a binomial distribution \text{binom}(x,\frac{1}{4}). Think of the conditional variable of Y \lvert X=x as tossing a coin x times where the probability of a head is p=\frac{1}{4}. The following is the sample space of the joint distribution of X and Y.

Figure 1

The joint probability function of X and Y may be written as:

\displaystyle (1) \ \ \ \ \ P(X=x,Y=y)=P(Y=y \lvert X=x) \times P(X=x)

Thus the probability at each point in Figure 1 is the product of P(X=x), which is \frac{1}{6}, with the conditional probability P(Y=y \lvert X=x), which is binomial. For example, the following diagram and equation demonstrate the calculation of P(X=4,Y=3)

Figure 2

\displaystyle \begin{aligned}(1a) \ \ \ \ \ P(X=4,Y=3)&=P(Y=3 \lvert X=4) \times P(X=4) \\&=\binom{4}{3} \biggl[\frac{1}{4}\biggr]^3 \biggl[\frac{3}{4}\biggr]^1 \times \frac{1}{6} \\&=\frac{12}{256}  \end{aligned}

Problem 1.2
The following shows the calculation of the binomial distributions.

\displaystyle \begin{aligned} (2) \ \ \ Y \lvert X=1 \ \ \ \ \ &P(Y=0 \lvert X=1)=\frac{3}{4} \\&P(Y=1 \lvert X=1)=\frac{1}{4} \end{aligned}

\displaystyle \begin{aligned} (3) \ \ \ Y \lvert X=2 \ \ \ \ \ &P(Y=0 \lvert X=2)=\binom{2}{0} \biggl(\frac{1}{4}\biggr)^0 \biggl(\frac{3}{4}\biggr)^2=\frac{9}{16} \\&P(Y=1 \lvert X=2)=\binom{2}{1} \biggl(\frac{1}{4}\biggr)^1 \biggl(\frac{3}{4}\biggr)^1=\frac{6}{16} \\&P(Y=2 \lvert X=2)=\binom{2}{2} \biggl(\frac{1}{4}\biggr)^2 \biggl(\frac{3}{4}\biggr)^0=\frac{1}{16} \end{aligned}

\displaystyle \begin{aligned} (4) \ \ \ Y \lvert X=3 \ \ \ \ \ &P(Y=0 \lvert X=3)=\binom{3}{0} \biggl(\frac{1}{4}\biggr)^0 \biggl(\frac{3}{4}\biggr)^3=\frac{27}{64} \\&P(Y=1 \lvert X=3)=\binom{3}{1} \biggl(\frac{1}{4}\biggr)^1 \biggl(\frac{3}{4}\biggr)^2=\frac{27}{64} \\&P(Y=2 \lvert X=3)=\binom{3}{2} \biggl(\frac{1}{4}\biggr)^2 \biggl(\frac{3}{4}\biggr)^1=\frac{9}{64} \\&P(Y=3 \lvert X=3)=\binom{3}{3} \biggl(\frac{1}{4}\biggr)^3 \biggl(\frac{3}{4}\biggr)^0=\frac{1}{64} \end{aligned}

\displaystyle \begin{aligned} (5) \ \ \ Y \lvert X=4 \ \ \ \ \ &P(Y=0 \lvert X=4)=\binom{4}{0} \biggl(\frac{1}{4}\biggr)^0 \biggl(\frac{3}{4}\biggr)^4=\frac{81}{256} \\&P(Y=1 \lvert X=4)=\binom{4}{1} \biggl(\frac{1}{4}\biggr)^1 \biggl(\frac{3}{4}\biggr)^3=\frac{108}{256} \\&P(Y=2 \lvert X=4)=\binom{4}{2} \biggl(\frac{1}{4}\biggr)^2 \biggl(\frac{3}{4}\biggr)^2=\frac{54}{256} \\&P(Y=3 \lvert X=4)=\binom{4}{3} \biggl(\frac{1}{4}\biggr)^3 \biggl(\frac{3}{4}\biggr)^1=\frac{12}{256} \\&P(Y=4 \lvert X=4)=\binom{4}{4} \biggl(\frac{1}{4}\biggr)^4 \biggl(\frac{3}{4}\biggr)^0=\frac{1}{256} \end{aligned}

\displaystyle \begin{aligned} (6) \ \ \ Y \lvert X=5 \ \ \ \ \ &P(Y=0 \lvert X=5)=\binom{5}{0} \biggl(\frac{1}{4}\biggr)^0 \biggl(\frac{3}{4}\biggr)^5=\frac{243}{1024} \\&P(Y=1 \lvert X=5)=\binom{5}{1} \biggl(\frac{1}{4}\biggr)^1 \biggl(\frac{3}{4}\biggr)^4=\frac{405}{1024} \\&P(Y=2 \lvert X=5)=\binom{5}{2} \biggl(\frac{1}{4}\biggr)^2 \biggl(\frac{3}{4}\biggr)^3=\frac{270}{1024} \\&P(Y=3 \lvert X=5)=\binom{5}{3} \biggl(\frac{1}{4}\biggr)^3 \biggl(\frac{3}{4}\biggr)^2=\frac{90}{1024} \\&P(Y=4 \lvert X=5)=\binom{5}{4} \biggl(\frac{1}{4}\biggr)^4 \biggl(\frac{3}{4}\biggr)^1=\frac{15}{1024} \\&P(Y=5 \lvert X=5)=\binom{5}{5} \biggl(\frac{1}{4}\biggr)^5 \biggl(\frac{3}{4}\biggr)^0=\frac{1}{1024} \end{aligned}

\displaystyle \begin{aligned} (7) \ \ \ Y \lvert X=6 \ \ \ \ \ &P(Y=0 \lvert X=6)=\binom{6}{0} \biggl(\frac{1}{4}\biggr)^0 \biggl(\frac{3}{4}\biggr)^6=\frac{729}{4096} \\&P(Y=1 \lvert X=6)=\binom{6}{1} \biggl(\frac{1}{4}\biggr)^1 \biggl(\frac{3}{4}\biggr)^5=\frac{1458}{4096} \\&P(Y=2 \lvert X=6)=\binom{6}{2} \biggl(\frac{1}{4}\biggr)^2 \biggl(\frac{3}{4}\biggr)^4=\frac{1215}{4096} \\&P(Y=3 \lvert X=6)=\binom{6}{3} \biggl(\frac{1}{4}\biggr)^3 \biggl(\frac{3}{4}\biggr)^3=\frac{540}{4096} \\&P(Y=4 \lvert X=6)=\binom{6}{4} \biggl(\frac{1}{4}\biggr)^4 \biggl(\frac{3}{4}\biggr)^2=\frac{135}{4096} \\&P(Y=5 \lvert X=6)=\binom{6}{5} \biggl(\frac{1}{4}\biggr)^5 \biggl(\frac{3}{4}\biggr)^1=\frac{18}{4096} \\&P(Y=6 \lvert X=6)=\binom{6}{6} \biggl(\frac{1}{4}\biggr)^6 \biggl(\frac{3}{4}\biggr)^0=\frac{1}{4096} \end{aligned}

Problem 1.3
To find the marginal probability P(Y=y), we need to sum P(X=x,Y=y) over all x. For example, P(Y=2) is the sum of P(X=x,Y=2) for all x=2,3,4,5,6. See the following diagram

Figure 3

As indicated in (1), each P(X=x,Y=2) is the product of a conditional probability P(Y=y \lvert X=x) and P(X=x)=\frac{1}{6}. Thus the probability indicated in Figure 3 can be translated as:

\displaystyle \begin{aligned}(8) \ \ \ \ \ P(Y=2)&=\sum \limits_{x=2}^6 P(Y=2 \lvert X=x) P(X=x)  \end{aligned}

We now begin the calculation.

\displaystyle \begin{aligned}(9) \ \ \ \ \ P(Y=0)&=\sum \limits_{x=1}^6 P(Y=0 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{3}{4}+\frac{9}{16}+\frac{27}{64} \\&+ \ \ \ \frac{81}{256}+\frac{243}{1024}+\frac{729}{4096} \biggr] \\&=\frac{10101}{24576} \end{aligned}

\displaystyle \begin{aligned}(10) \ \ \ \ \ P(Y=1)&=\sum \limits_{x=1}^6 P(Y=1 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{4}+\frac{6}{16}+\frac{27}{64} \\&+ \ \ \ \frac{108}{256}+\frac{405}{1024}+\frac{1458}{4096} \biggr] \\&=\frac{9094}{24576} \end{aligned}

\displaystyle \begin{aligned}(11) \ \ \ \ \ P(Y=2)&=\sum \limits_{x=2}^6 P(Y=2 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{16}+\frac{9}{64} \\&+ \ \ \ \frac{54}{256}+\frac{270}{1024}+\frac{1215}{4096} \biggr] \\&=\frac{3991}{24576} \end{aligned}

\displaystyle \begin{aligned}(12) \ \ \ \ \ P(Y=3)&=\sum \limits_{x=3}^6 P(Y=3 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{64} \\&+ \ \ \ \frac{12}{256}+\frac{90}{1024}+\frac{540}{4096} \biggr] \\&=\frac{1156}{24576} \end{aligned}

\displaystyle \begin{aligned}(13) \ \ \ \ \ P(Y=4)&=\sum \limits_{x=4}^6 P(Y=4 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{256}+\frac{15}{1024}+\frac{135}{4096} \biggr] \\&=\frac{211}{24576} \end{aligned}

\displaystyle \begin{aligned}(14) \ \ \ \ \ P(Y=5)&=\sum \limits_{x=5}^6 P(Y=5 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{1024}+\frac{18}{4096} \biggr] \\&=\frac{22}{24576} \end{aligned}

\displaystyle \begin{aligned}(15) \ \ \ \ \ P(Y=6)&=\sum \limits_{x=6}^6 P(Y=6 \lvert X=x) P(X=x) \\&=\frac{1}{6} \biggl[ \frac{1}{4096} \biggr] \\&=\frac{1}{24576} \end{aligned}

The following is the calculation of the mean and variance of Y.

\displaystyle \begin{aligned}(16) \ \ \ \ \ E(Y)&=\frac{10101}{24576} \times 0+\frac{9094}{24576} \times 1+\frac{3991}{24576} \times 2  \\&+ \ \ \ \ \frac{1156}{24576} \times 3+\frac{211}{24576} \times 4+\frac{22}{24576} \times 5 \\&+ \ \ \ \ \frac{1}{24576} \times 6  \\&=\frac{21504}{24576}\\&=0.875 \end{aligned}

\displaystyle \begin{aligned}(17) \ \ \ \ \ E(Y^2)&=\frac{10101}{24576} \times 0+\frac{9094}{24576} \times 1+\frac{3991}{24576} \times 2^2  \\&+ \ \ \ \ \frac{1156}{24576} \times 3^2+\frac{211}{24576} \times 4^2+\frac{22}{24576} \times 5^2 \\&+ \ \ \ \ \frac{1}{24576} \times 6^2  \\&=\frac{39424}{24576}\\&=\frac{77}{48} \end{aligned}

\displaystyle (18) \ \ \ \ \ Var(Y)=\frac{77}{48}-0.875^2=\frac{161}{192}=0.8385

Problem 1.4
The conditional probability P(Y=y \lvert X=x) is easy to compute since it is a given that Y is a binomial variable conditional on a value of X. Now we want to find the backward probability P(X= x \lvert Y=y). Given the binomial observation is Y=y, what is the probability that the roll of the die is X=x? This is an application of the Bayes’ theorem. We can start by looking at Figure 3 once more.

Consider P(X=x \lvert Y=2). In calculating this conditional probability, we only consider the 5 sample points encircled in Figure 3 and disregard all the other points. These 5 points become a new sample space if you will (this is the essence of conditional probability and conditional distribution). The sum of the joint probability P(X=x,Y=y) for these 5 points is P(Y=2), calculated in the previous step. The conditional probability P(X=x \lvert Y=2) is simply the probability of one of these 5 points as a fraction of the total probability P(Y=2). Thus we have:

\displaystyle \begin{aligned}(19) \ \ \ \ \ P(X=x \lvert Y=2)&=\frac{P(X=x,Y=2)}{P(Y=2)} \end{aligned}

We do not have to evaluate the components that go into (19). As a practical matter, to find P(X=x \lvert Y=2) is to take each of 5 probabilities shown in (11) and evaluate it as a fraction of the total probability P(Y=2). Thus we have:

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 2 \bold )
\displaystyle \begin{aligned}(20a) \ \ \ \ \ P(X=2 \lvert Y=2)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{16}}{\displaystyle \frac{3991}{24576}} =\frac{256}{3991} \end{aligned}

\displaystyle \begin{aligned}(20b) \ \ \ \ \ P(X=3 \lvert Y=2)&=\frac{\displaystyle \frac{1}{6} \times \frac{9}{64}}{\displaystyle \frac{3991}{24576}} =\frac{576}{3991} \end{aligned}

\displaystyle \begin{aligned}(20c) \ \ \ \ \ P(X=4 \lvert Y=2)&=\frac{\displaystyle \frac{1}{6} \times \frac{54}{256}}{\displaystyle \frac{3991}{24576}} =\frac{864}{3991} \end{aligned}

\displaystyle \begin{aligned}(20d) \ \ \ \ \ P(X=5 \lvert Y=2)&=\frac{\displaystyle \frac{1}{6} \times \frac{270}{1024}}{\displaystyle \frac{3991}{24576}} =\frac{1080}{3991} \end{aligned}

\displaystyle \begin{aligned}(20e) \ \ \ \ \ P(X=6 \lvert Y=2)&=\frac{\displaystyle \frac{1}{6} \times \frac{1215}{4096}}{\displaystyle \frac{3991}{24576}} =\frac{1215}{3991} \end{aligned}

Here’s the rest of the Bayes’ calculation:

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 0 \bold )
\displaystyle \begin{aligned}(21a) \ \ \ \ \ P(X=1 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{3}{4}}{\displaystyle \frac{10101}{24576}} =\frac{3072}{10101} \end{aligned}

\displaystyle \begin{aligned}(21b) \ \ \ \ \ P(X=2 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{9}{16}}{\displaystyle \frac{10101}{24576}} =\frac{2304}{10101} \end{aligned}

\displaystyle \begin{aligned}(21c) \ \ \ \ \ P(X=3 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{27}{64}}{\displaystyle \frac{10101}{24576}} =\frac{1728}{10101} \end{aligned}

\displaystyle \begin{aligned}(21d) \ \ \ \ \ P(X=4 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{81}{256}}{\displaystyle \frac{10101}{24576}} =\frac{1296}{10101} \end{aligned}

\displaystyle \begin{aligned}(21e) \ \ \ \ \ P(X=5 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{243}{1024}}{\displaystyle \frac{10101}{24576}} =\frac{972}{10101} \end{aligned}

\displaystyle \begin{aligned}(21f) \ \ \ \ \ P(X=6 \lvert Y=0)&=\frac{\displaystyle \frac{1}{6} \times \frac{729}{4096}}{\displaystyle \frac{10101}{24576}} =\frac{3729}{10101} \end{aligned}

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 1 \bold )
\displaystyle \begin{aligned}(22a) \ \ \ \ \ P(X=1 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{4}}{\displaystyle \frac{9094}{24576}} =\frac{1024}{9094} \end{aligned}

\displaystyle \begin{aligned}(22b) \ \ \ \ \ P(X=2 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{6}{16}}{\displaystyle \frac{9094}{24576}} =\frac{1536}{9094} \end{aligned}

\displaystyle \begin{aligned}(22c) \ \ \ \ \ P(X=3 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{27}{64}}{\displaystyle \frac{9094}{24576}} =\frac{1728}{9094} \end{aligned}

\displaystyle \begin{aligned}(22d) \ \ \ \ \ P(X=4 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{108}{256}}{\displaystyle \frac{9094}{24576}} =\frac{1728}{9094} \end{aligned}

\displaystyle \begin{aligned}(22e) \ \ \ \ \ P(X=5 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{405}{1024}}{\displaystyle \frac{9094}{24576}} =\frac{1620}{9094} \end{aligned}

\displaystyle \begin{aligned}(22f) \ \ \ \ \ P(X=6 \lvert Y=1)&=\frac{\displaystyle \frac{1}{6} \times \frac{1458}{4096}}{\displaystyle \frac{9094}{24576}} =\frac{1458}{9094} \end{aligned}

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 2 \bold ) done earlier

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 3 \bold )
\displaystyle \begin{aligned}(23a) \ \ \ \ \ P(X=3 \lvert Y=3)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{64}}{\displaystyle \frac{1156}{24576}} =\frac{64}{1156} \end{aligned}

\displaystyle \begin{aligned}(23b) \ \ \ \ \ P(X=4 \lvert Y=3)&=\frac{\displaystyle \frac{1}{6} \times \frac{12}{256}}{\displaystyle \frac{1156}{24576}} =\frac{192}{1156} \end{aligned}

\displaystyle \begin{aligned}(23c) \ \ \ \ \ P(X=5 \lvert Y=3)&=\frac{\displaystyle \frac{1}{6} \times \frac{90}{1024}}{\displaystyle \frac{1156}{24576}} =\frac{360}{1156} \end{aligned}

\displaystyle \begin{aligned}(23d) \ \ \ \ \ P(X=6 \lvert Y=3)&=\frac{\displaystyle \frac{1}{6} \times \frac{540}{4096}}{\displaystyle \frac{1156}{24576}} =\frac{540}{1156} \end{aligned}

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 4 \bold )
\displaystyle \begin{aligned}(24a) \ \ \ \ \ P(X=4 \lvert Y=4)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{256}}{\displaystyle \frac{211}{24576}} =\frac{16}{211} \end{aligned}

\displaystyle \begin{aligned}(24b) \ \ \ \ \ P(X=5 \lvert Y=4)&=\frac{\displaystyle \frac{1}{6} \times \frac{15}{1024}}{\displaystyle \frac{211}{24576}} =\frac{60}{211} \end{aligned}

\displaystyle \begin{aligned}(24c) \ \ \ \ \ P(X=6 \lvert Y=4)&=\frac{\displaystyle \frac{1}{6} \times \frac{135}{4096}}{\displaystyle \frac{211}{24576}} =\frac{135}{211} \end{aligned}

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 5 \bold )
\displaystyle \begin{aligned}(25a) \ \ \ \ \ P(X=5 \lvert Y=5)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{1024}}{\displaystyle \frac{22}{24576}} =\frac{4}{22} \end{aligned}

\displaystyle \begin{aligned}(25b) \ \ \ \ \ P(X=6 \lvert Y=5)&=\frac{\displaystyle \frac{1}{6} \times \frac{18}{1024}}{\displaystyle \frac{22}{24576}} =\frac{18}{22} \end{aligned}

Calculation of \bold P \bold ( \bold X \bold = \bold x \bold \lvert \bold Y \bold = \bold 6 \bold )
\displaystyle \begin{aligned}(26) \ \ \ \ \ P(X=6 \lvert Y=6)&=\frac{\displaystyle \frac{1}{6} \times \frac{1}{4096}}{\displaystyle \frac{1}{24576}} =1 \end{aligned}

_____________________________________________________________
Probem 2
Let X be the value of one roll of a fair die. If the value of the die is x, we are given that Y \lvert X=x has a binomial distribution with n=x and p=\frac{1}{2} (we use the notation Y \lvert X=x \sim \text{binom}(x,\frac{1}{2})).

  1. Discuss how the joint probability function P[X=x,Y=y] is computed for x=1,2,3,4,5,6 and y=0,1, \cdots, x.
  2. Compute the conditional binomial distributions Y \lvert X=x where x=1,2,3,4,5,6.
  3. Compute the marginal probability function of Y and the mean and variance of Y.
  4. Compute P(X=x \lvert Y=y) for all applicable x and y.

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

\text{ }

_____________________________________________________________
Answers to Probem 2

Problem 2.3

\displaystyle \begin{aligned} P(Y=y): \ \ \ \ &P(Y=0)=\frac{63}{384} \\&\text{ }  \\&P(Y=1)=\frac{120}{384} \\&\text{ } \\&P(Y=2)=\frac{99}{384} \\&\text{ } \\&P(Y=3)=\frac{64}{384} \\&\text{ } \\&P(Y=4)=\frac{29}{384} \\&\text{ } \\&P(Y=5)=\frac{8}{384} \\&\text{ } \\&P(Y=6)=\frac{1}{384} \end{aligned}

\displaystyle E(Y)=\frac{7}{4}=1.75

\displaystyle Var(Y)=\frac{77}{48}

Problem 2.4

\displaystyle \begin{aligned} P(X=x \lvert Y=0): \ \ \ \ &P(X=1 \lvert Y=0)=\frac{32}{63} \\&\text{ }  \\&P(X=2 \lvert Y=0)=\frac{16}{63} \\&\text{ } \\&P(X=3 \lvert Y=0)=\frac{8}{63} \\&\text{ } \\&P(X=4 \lvert Y=0)=\frac{4}{63} \\&\text{ } \\&P(X=5 \lvert Y=0)=\frac{2}{63} \\&\text{ } \\&P(X=6 \lvert Y=0)=\frac{1}{63}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=1): \ \ \ \ &P(X=1 \lvert Y=1)=\frac{32}{120} \\&\text{ }  \\&P(X=2 \lvert Y=1)=\frac{32}{120} \\&\text{ } \\&P(X=3 \lvert Y=1)=\frac{24}{120} \\&\text{ } \\&P(X=4 \lvert Y=1)=\frac{16}{120} \\&\text{ } \\&P(X=5 \lvert Y=1)=\frac{10}{120} \\&\text{ } \\&P(X=6 \lvert Y=1)=\frac{6}{120}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=2): \ \ \ \ &P(X=2 \lvert Y=2)=\frac{16}{99} \\&\text{ } \\&P(X=3 \lvert Y=2)=\frac{24}{99} \\&\text{ } \\&P(X=4 \lvert Y=2)=\frac{24}{99} \\&\text{ } \\&P(X=5 \lvert Y=2)=\frac{20}{99} \\&\text{ } \\&P(X=6 \lvert Y=2)=\frac{15}{99}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=3): \ \ \ \ &P(X=3 \lvert Y=3)=\frac{8}{64} \\&\text{ } \\&P(X=4 \lvert Y=3)=\frac{16}{64} \\&\text{ } \\&P(X=5 \lvert Y=3)=\frac{20}{64} \\&\text{ } \\&P(X=6 \lvert Y=3)=\frac{20}{64}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=4): \ \ \ \ &P(X=4 \lvert Y=4)=\frac{4}{29} \\&\text{ } \\&P(X=5 \lvert Y=4)=\frac{10}{29} \\&\text{ } \\&P(X=6 \lvert Y=4)=\frac{15}{29}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=5): \ \ \ \ &P(X=5 \lvert Y=5)=\frac{2}{8} \\&\text{ } \\&P(X=6 \lvert Y=5)=\frac{6}{8}  \end{aligned}

\displaystyle \begin{aligned} P(X=x \lvert Y=6): \ \ \ \ &P(X=6 \lvert Y=6)=1  \end{aligned}