## Calculating the occupancy problem

The occupancy problem refers to the experiment of randomly throwing $k$ balls into $n$ cells. Out of this experiment, there are many problems that can be asked. In this post we focus on question: what is the probability that exactly $j$ of the cells are empty after randomly throwing $k$ balls into $n$ cells, where $0 \le j \le n-1$?

For a better perspective, there are many other ways to describe the experiment of throwing $k$ balls into $n$ cells. For example, throwing $k$ balls into 6 cells can be interpreted as rolling $k$ dice (or rolling a die $k$ times). Throwing $k$ balls into 365 cells can be interpreted as randomly selecting $k$ people and classifying them according to their dates of birth (assuming 365 days in a year). Another context is coupon collecting – the different types of coupons represent the $n$ cells and the coupons being collected represent the $k$ balls.

_____________________________________________________________________________________

Practice Problems

Let $X_{k,n}$ be the number of cells that are empty (i.e. not occupied) when throwing $k$ balls into $n$ cells. As noted above, the problem discussed in this post is to find the probability function $P(X_{k,n}=j)$ where $j=0,1,2,\cdots,n-1$. There are two elementary ways to do this problem. One is the approach of using double multinomial coefficient (see this post) and the other is to use a formula developed in this post. In addition to $X_{k,n}$, let $Y_{k,n}$ be the number of cells that are occupied, i.e. $Y_{k,n}=n-X_{k,n}$.

Practice Problems
Compute $P(X_{k,n}=j)$ where $j=0,1,2,\cdots,n-1$ for the following pairs of $k$ and $n$.

1. $k=5$ and $n=5$
2. $k=6$ and $n=5$
3. $k=7$ and $n=5$
4. $k=6$ and $n=6$
5. $k=7$ and $n=6$
6. $k=8$ and $n=6$

We work the problem for $k=7$ and $n=5$. We show both the double multinomial coefficient approach and the formula approach. Recall the $k$ here is the number of balls and $n$ is the number of cells.

_____________________________________________________________________________________

Example – Double Multinomial Coefficient

Note that the double multinomial coefficient approach calculate the probabilities $P(Y_{7,5}=j)$ where $Y_{7,5}$ is the number of occupied cells when throwing 7 balls into 5 cells. In throwing 7 balls into 5 cells, there is a total of $5^7=$ 78125 many ordered samples. To calculate $P(Y_{7,5}=j)$, first write down the representative occupancy sets for the event $Y_{7,5}=j$. For each occupancy set, calculate the number of ordered samples (out of 78125) that belong to that occupancy set. Then we add up all the counts for all the occupancy sets for $Y_{7,5}=j$. This is best illustrated with an example. To see the development of this idea, see this post.

Fist, consider the event of $Y_{7,5}=1$ (only one cell is occupied, i.e. all the balls going into one cell). A representative occupancy set is (0, 0, 0, 0, 7), all the balls going into the 5th cell. The first multinomial coefficient is on the 5 cells and the second multinomial coefficient is on the 7 balls.

(0, 0, 0, 0, 7)

$\displaystyle \frac{5!}{4! 1!} \times \frac{7!}{7!}=5 \times 1 =5$

Total = 5

So there are 5 ordered samples out of 16807 that belong to the event $Y_{7,5}=1$. Note that the first multinomial coefficient is the number of to order the 5 cells where 4 of the cells are empty and one of the cells has 7 balls. The second multinomial coefficient is the number of ways to order the 7 balls where all 7 balls go into the 5th cell.

Now consider the event $Y_{7,5}=2$ (all 7 balls go into 2 cells). There are three representative occupancy sets. The following shows the calculation for each set.

(0, 0, 0, 1, 6)

$\displaystyle \frac{5!}{3! 1! 1!} \times \frac{7!}{1! 6!}=20 \times 7 =140$
(0, 0, 0, 2, 5)

$\displaystyle \frac{5!}{3! 1! 1!} \times \frac{7!}{2! 5!}=20 \times 21 =420$
(0, 0, 0, 3, 4)

$\displaystyle \frac{5!}{3! 1! 1!} \times \frac{7!}{3! 4!}=20 \times 35 =700$

Total = 140 + 420 + 700 = 1260

Here’s an explanation for the occupancy set (0, 0, 0, 3, 4). The occupancy set refers to the scenario that 3 of the 7 balls go into the 4th cell and 4 of the 7 balls go into the 5th cell. But we want to count all the possibilities such that 3 of the 7 balls go into one cell and 4 of the 7 balls go into another cell. Thus the first multinomial coefficient count the number of ways to order 5 cells where three of the cells are empty and one cell has 3 balls and the remaining cell has 4 balls (20 ways) The second multinomial coefficient is the number of ways to order the 7 balls where 3 of the 7 balls go into the 4th cell and 4 of the 7 balls go into the 5th cell (35 ways). So the total number of possibilities for the occupancy set (0, 0, 0, 3, 4) is 20 times 35 = 700. The sum total for three occupancy sets is 1260.

We now show the remaining calculation without further elaboration.

Now consider the event $Y_{7,5}=3$ (all 7 balls go into 3 cells). There are four representative occupancy sets. The following shows the calculation for each set.

(0, 0, 1, 1, 5)

$\displaystyle \frac{5!}{2! 2! 1!} \times \frac{7!}{1! 1! 5!}=30 \times 42 =1260$
(0, 0, 1, 2, 4)

$\displaystyle \frac{5!}{2! 1! 1! 1!} \times \frac{7!}{1! 2! 4!}=60 \times 105 =6300$
(0, 0, 1, 3, 3)

$\displaystyle \frac{5!}{2! 1! 2!} \times \frac{7!}{1! 3! 3!}=30 \times 140 =4200$
(0, 0, 2, 2, 3)

$\displaystyle \frac{5!}{2! 2! 1!} \times \frac{7!}{2! 2! 3!}=30 \times 210 =6300$

Total = 1260 + 6300 + 4200 + 6300 = 18060

Now consider the event $Y_{7,5}=4$ (all 7 balls go into 4 cells, i.e. one empty cell). There are three representative occupancy sets. The following shows the calculation for each set.

(0, 1, 1, 1, 4)

$\displaystyle \frac{5!}{1! 3! 1!} \times \frac{7!}{1! 1! 1! 4!}=20 \times 210 =4200$
(0, 1, 1, 2, 3)

$\displaystyle \frac{5!}{1! 2! 1! 1!} \times \frac{7!}{1! 1! 2! 3!}=60 \times 420 =25200$
(0, 1, 2, 2, 2)

$\displaystyle \frac{5!}{1! 1! 3!} \times \frac{7!}{1! 2! 2! 2!}=20 \times 630 =12600$

Total = 4200 + 25200 + 12600 = 42000

Now consider the event $Y_{7,5}=4$ (all 7 balls go into 5 cells, i.e. no empty cell). There are two representative occupancy sets. The following shows the calculation for each set.

(1, 1, 1, 1, 3)

$\displaystyle \frac{5!}{4! 1!} \times \frac{7!}{1! 1! 1! 1! 3!}=5 \times 840 =4200$
(1, 1, 1, 2, 2)

$\displaystyle \frac{5!}{3! 2!} \times \frac{7!}{1! 1! 1! 2! 2!}=10 \times 1260 =12600$

Total = 4200 + 12600 = 16800

The following is the distribution for the random variable $Y_{7,5}$.

$\displaystyle P(Y_{7,5}=1)=\frac{5}{78125}=0.000064$

$\displaystyle P(Y_{7,5}=2)=\frac{1260}{78125}=0.016128$

$\displaystyle P(Y_{7,5}=3)=\frac{18060}{78125}=0.231168$

$\displaystyle P(Y_{7,5}=4)=\frac{42000}{78125}=0.5376$

$\displaystyle P(Y_{7,5}=5)=\frac{16800}{78125}=0.21504$

3.951424

Remarks
In throwing 7 balls at random into 5 cells, it is not like that the balls are in only one or two cells (about 1.6% chance). The mean number of occupied cells is about 3.95. More than 50% of the times, 4 cells are occupied.

The above example has small numbers of balls and cells and is an excellent example for practicing the calculation. Working such problems can help build the intuition for the occupancy problem. However, when the numbers are larger, the calculation using double multinomial coefficient can be lengthy and tedious. Next we show how to use a formula for occupancy problem using the same example.

_____________________________________________________________________________________

Example – Formula Approach

The formula we use is developed in this post. Recall that $X_{k,n}$ is the number of empty cells when throwing $k$ balls into $n$ cells. The formula calculates the probabilities $P(X_{7,5}=j)$ where
$0 \le j \le 5$. The first step is to compute the probabilities $P(X_{7,m}=0)$ for $m=5,4,3,2,1$. Each of these is the probability that all m cells are occupied (when throwing 7 balls into $m$ cells). These 5 probabilities will be used to calculate $P(X_{7,5}=j)$. This is a less direct implementation of the formula but gives a more intuitive explanation.

\displaystyle \begin{aligned} P(X_{7,5}=0)&=1-P(X_{7,5} \ge 1) \\&=1-\sum \limits_{j=1}^5 (-1)^{j+1} \binom{5}{j} \biggl[ 1-\frac{j}{5} \biggr]^7 \\&=1-\biggl[5 \biggl(\frac{4}{5}\biggr)^7-10 \biggl(\frac{3}{5}\biggr)^7+10 \biggl(\frac{2}{5}\biggr)^7 -5 \biggl(\frac{1}{5}\biggr)^7 + 0 \biggr] \\&=1-\frac{81920-21870+1280-5}{78125} \\&=\frac{16800}{78125} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,4}=0)&=1-P(X_{7,4} \ge 1) \\&=1-\sum \limits_{j=1}^4 (-1)^{j+1} \binom{4}{j} \biggl[ 1-\frac{j}{4} \biggr]^7 \\&=1-\biggl[4 \biggl(\frac{3}{4}\biggr)^7-6 \biggl(\frac{2}{4}\biggr)^7+4 \biggl(\frac{1}{4}\biggr)^7 - 0 \biggr] \\&=1-\frac{8748-768+4}{16384} \\&=\frac{8400}{16384} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,3}=0)&=1-P(X_{7,3} \ge 1) \\&=1-\sum \limits_{j=1}^3 (-1)^{j+1} \binom{3}{j} \biggl[ 1-\frac{j}{3} \biggr]^7 \\&=1-\biggl[3 \biggl(\frac{2}{3}\biggr)^7-3 \biggl(\frac{1}{3}\biggr)^7+0 \biggr] \\&=1-\frac{384-3}{2187} \\&=\frac{1806}{2187} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,2}=0)&=1-P(X_{7,2} \ge 1) \\&=1-\sum \limits_{j=1}^2 (-1)^{j+1} \binom{2}{j} \biggl[ 1-\frac{j}{2} \biggr]^7 \\&=1-\biggl[2 \biggl(\frac{1}{2}\biggr)^7-0 \biggr] \\&=1-\frac{2}{128} \\&=\frac{126}{128} \end{aligned}

$P(X_{7,1}=0)=1$

Each of the above 5 probabilities (except the last one) is based on the probability $P(X_{7,m} \ge 1)$, which is the probability that there is at least one cell that is empty when throwing 7 balls into $m$ cells. The inclusion-exclusion principle is used to derive $P(X_{7,m} \ge 1)$. The last of the five does not need calculation. When throwing 7 balls into one cell, there will be no empty cells.

Now the rest of the calculation:

$\displaystyle P(X_{7,5}=0)=\frac{16800}{78125}$

\displaystyle \begin{aligned} P(X_{7,5}=1)&=P(\text{1 empty cell}) \times P(\text{none of the other 4 cells is empty}) \\&=\binom{5}{1} \biggl(1-\frac{1}{5} \biggr)^7 \times P(X_{7,4}=0) \\&=5 \ \frac{16384}{78125} \times \frac{8400}{16384} \\&=\frac{42000}{78125} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,5}=2)&=P(\text{2 empty cells}) \times P(\text{none of the other 3 cells is empty}) \\&=\binom{5}{2} \biggl(1-\frac{2}{5} \biggr)^7 \times P(X_{7,3}=0) \\&=10 \ \frac{2187}{78125} \times \frac{1806}{2187} \\&=\frac{18060}{78125} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,5}=3)&=P(\text{3 empty cells}) \times P(\text{none of the other 2 cells is empty}) \\&=\binom{5}{3} \biggl(1-\frac{3}{5} \biggr)^7 \times P(X_{7,2}=0) \\&=10 \ \frac{128}{78125} \times \frac{126}{128} \\&=\frac{1260}{78125} \end{aligned}

\displaystyle \begin{aligned} P(X_{7,5}=4)&=P(\text{4 empty cells}) \times P(\text{the other 1 cell is not empty}) \\&=\binom{5}{4} \biggl(1-\frac{4}{5} \biggr)^7 \times P(X_{7,1}=0) \\&=5 \ \frac{1}{78125} \times 1 \\&=\frac{5}{78125} \end{aligned}

Note that these answers agree with the ones from the double multinomial coefficient approach after making the adjustment $Y_{7,5}=5-X_{7,5}$.

_____________________________________________________________________________________

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$
_____________________________________________________________________________________

Problem 1
5 balls into 5 cells

$\displaystyle P(X_{5,5}=0)=P(Y_{5,5}=5)=\frac{120}{3125}$

$\displaystyle P(X_{5,5}=1)=P(Y_{5,5}=4)=\frac{1200}{3125}$

$\displaystyle P(X_{5,5}=2)=P(Y_{5,5}=3)=\frac{1500}{3125}$

$\displaystyle P(X_{5,5}=3)=P(Y_{5,5}=2)=\frac{300}{3125}$

$\displaystyle P(X_{5,5}=4)=P(Y_{5,5}=1)=\frac{5}{3125}$

Problem 2
6 balls into 5 cells

$\displaystyle P(X_{6,5}=0)=P(Y_{6,5}=5)=\frac{1800}{15625}$

$\displaystyle P(X_{6,5}=1)=P(Y_{6,5}=4)=\frac{7800}{15625}$

$\displaystyle P(X_{6,5}=2)=P(Y_{6,5}=3)=\frac{5400}{15625}$

$\displaystyle P(X_{6,5}=3)=P(Y_{6,5}=2)=\frac{620}{15625}$

$\displaystyle P(X_{6,5}=4)=P(Y_{6,5}=1)=\frac{5}{15625}$

Problem 3
7 balls into 5 cells. See above.

Problem 4
6 balls into 6 cells

$\displaystyle P(X_{6,6}=0)=P(Y_{6,6}=6)=\frac{720}{46656}$

$\displaystyle P(X_{6,6}=1)=P(Y_{6,6}=5)=\frac{10800}{46656}$

$\displaystyle P(X_{6,6}=2)=P(Y_{6,6}=4)=\frac{23400}{46656}$

$\displaystyle P(X_{6,6}=3)=P(Y_{6,6}=3)=\frac{10800}{46656}$

$\displaystyle P(X_{6,6}=4)=P(Y_{6,6}=2)=\frac{930}{46656}$

$\displaystyle P(X_{6,6}=5)=P(Y_{6,6}=1)=\frac{6}{46656}$

Problem 5
7 balls into 6 cells

$\displaystyle P(X_{7,6}=0)=P(Y_{7,6}=6)=\frac{15120}{279936}$

$\displaystyle P(X_{7,6}=1)=P(Y_{7,6}=5)=\frac{100800}{279936}$

$\displaystyle P(X_{7,6}=2)=P(Y_{7,6}=4)=\frac{126000}{279936}$

$\displaystyle P(X_{7,6}=3)=P(Y_{7,6}=3)=\frac{36120}{279936}$

$\displaystyle P(X_{7,6}=4)=P(Y_{7,6}=2)=\frac{1890}{279936}$

$\displaystyle P(X_{7,6}=5)=P(Y_{7,6}=1)=\frac{6}{279936}$

Problem 6
8 balls into 6 cells

$\displaystyle P(X_{8,6}=0)=P(Y_{8,6}=6)=\frac{191520}{1679616}$

$\displaystyle P(X_{8,6}=1)=P(Y_{8,6}=5)=\frac{756000}{1679616}$

$\displaystyle P(X_{8,6}=2)=P(Y_{8,6}=4)=\frac{612360}{1679616}$

$\displaystyle P(X_{8,6}=3)=P(Y_{8,6}=3)=\frac{115920}{1679616}$

$\displaystyle P(X_{8,6}=4)=P(Y_{8,6}=2)=\frac{3810}{1679616}$

$\displaystyle P(X_{8,6}=5)=P(Y_{8,6}=1)=\frac{6}{1679616}$

_____________________________________________________________________________________

$\copyright \ 2015 \text{ by Dan Ma}$

## Practice problems for the Poisson distribution

This post has practice problems on the Poisson distribution. For a good discussion of the Poisson distribution and the Poisson process, see this blog post in the companion blog.

_____________________________________________________________________________________

Practice Problems

Practice Problem 1
Two taxi arrive on average at a certain street corner for every 15 minutes. Suppose that the number of taxi arriving at this street corner follows a Poisson distribution. Three people are waiting at the street corner for taxi (assuming they do not know each other and each one will have his own taxi). Each person will be late for work if he does not catch a taxi within the next 15 minutes. What is the probability that all three people will make it to work on time?

Practice Problem 2
A 5-county area in Kansas is hit on average by 3 tornadoes a year (assuming annual Poisson tornado count). What is the probability that the number of tornadoes will be more than the historical average next year in this area?

Practice Problem 3
A certain airline estimated that 0.8% of its customers with purchased tickets fail to show up for their flights. For one particular flight, the plane has 500 seats and the flight has been fully booked. How many additional tickets can the airline sell so that there is at least a 90% chance that everyone who shows up will have a seat?

Practice Problem 4
A life insurance insured 9000 men aged 45. The probability that a 45-year old man will die within one year is 0.0035. Within the next year, what is the probability that the insurance company will pay between 30 and 33 claims (both inclusive) among these 7000 men?

Practice Problem 5
In a certain manuscript of 1000 pages, 300 typographical errors occur.

• What is the probability that a randomly selected page will be error free?
• What is the probability that 10 randomly selected pages will have at most 3 errors?

Practice Problem 6
Trisomy 13, also called Patau syndrome, is a chromosomal condition associated with severe intellectual disability and physical abnormalities in many parts of the body. Trisomy 13 occurs , on the average, once in every 16,000 births. Suppose that in one country, 100,000 babies are born in a year. What is the probability that at most 3 births will develop this chromosomal condition?

Practice Problem 7
Traffic accidents occur along a 50-mile stretch of highway at the rate of 0.85 during the hour from 5 PM to 6 PM. Suppose that the number of traffic accidents in this stretch of highway follows a Poisson distribution. The department of transportation plans to observe the traffic flow in this stretch of highway during this hour in a two-day period. What is the probability that more than three accidents occur in this observation period?

Practice Problem 8
The odds of winning the Mega Million lottery is one in 176 million. Out of 176 million lottery tickets sold, what is the probability of having no winning ticket? What is the exact probability model in this problem?

_____________________________________________________________________________________

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$
_____________________________________________________________________________________

Practice Problem 1

• $1-5e^{-2}=0.323323584$

Practice Problem 2

• $1-13e^{-3}=0.352768111$

Practice Problem 3

• Can oversell by 2 tickets.

Practice Problem 4

• 0.278162459

Practice Problem 5

• 0.740818221
• 0.647231889

Practice Problem 6

• 0.130250355 (using Poisson)
• 0.130242377 (using Binomial)

Practice Problem 7

• 0.093189434

Practice Problem 8

• 0.367879441

_____________________________________________________________________________________

$\copyright \ 2015 \text{ by Dan Ma}$

## Practice Problems for Conditional Distributions, Part 2

The following are practice problems on conditional distributions. The thought process of how to work with these practice problems can be found in the blog post Conditionals Distribution, Part 2.

_____________________________________________________________________________________

Practice Problems

Practice Problem 1

Suppose that $X$ is the lifetime (in years) of a brand new machine of a certain type. The following is the density function.

$\displaystyle f(x)=\frac{1}{8 \sqrt{x}}, \ \ \ \ \ \ \ \ \ 1

You just purchase a 9-year old machine of this type that is in good working condition. Compute the following:

• What is the expected lifetime of this 9-year old machine?
• What is the expected remaining life of this 9-year old machine?

$\text{ }$

Practice Problem 2

Suppose that $X$ is the total amount of damages (in millions of dollars) resulting from the occurrence of a severe wind storm in a certain city. The following is the density function of $X$.

$\displaystyle f(x)=\frac{81}{(x+3)^4}, \ \ \ \ \ \ \ \ \ 0

Suppose that the next storm is expected to cause damages exceeding one million dollars. Compute the following:

• What is the expected total amount of damages for the next storm given that it will exceeds one million dollars?
• The city has a reserve fund of one million dollars to cover the damages from the next storm. Given the amount of damages for the next storm will exceeds one million dollars, what is the expected total amount of damages in excess of the amount in the reserve fund?

_____________________________________________________________________________________

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$
_____________________________________________________________________________________

The thought process of how to work with these practice problems can be found in the blog post Conditionals Distribution, Part 2.

Practice Problem 1

$\displaystyle E(X \lvert X>9)=\frac{49}{3}=16.33 \text{ years}$

$\displaystyle E(X-9 \lvert X>9)=\frac{22}{3}=7.33 \text{ years}$

Practice Problem 2

$\displaystyle E(X \lvert X>1)=3 \text{ millions}$

$\displaystyle E(X-1 \lvert X>1)=2 \text{ millions}$

_____________________________________________________________________________________

$\copyright \ 2013 \text{ by Dan Ma}$

## Practice Problems for Conditional Distributions, Part 1

The following are practice problems on conditional distributions. The thought process of how to work with these practice problems can be found in the blog post Conditionals Distribution, Part 1.

_____________________________________________________________________________________

Description of Problems

Suppose $X$ and $Y$ are independent binomial distributions with the following parameters.

For $X$, number of trials $n=5$, success probability $\displaystyle p=\frac{1}{2}$

For $Y$, number of trials $n=5$, success probability $\displaystyle p=\frac{3}{4}$

We can think of these random variables as the results of two students taking a multiple choice test with 5 questions. For example, let $X$ be the number of correct answers for one student and $Y$ be the number of correct answers for the other student. For the practice problems below, passing the test means having 3 or more correct answers.

Suppose we have some new information about the results of the test. The problems below are to derive the conditional distributions of $X$ or $Y$ based on the new information and to compare the conditional distributions with the unconditional distributions.

Practice Problem 1

• New information: $X.
• Derive the conditional distribution for $X \lvert X.
• Derive the conditional distribution for $Y \lvert X.
• Compare these conditional distributions with the unconditional ones with respect to mean and probability of passing.
• What is the effect of the new information on the test performance of each of the students?
• Explain why the new information has the effect on the test performance?

Practice Problem 2

• New information: $X>Y$.
• Derive the conditional distribution for $X \lvert X>Y$.
• Derive the conditional distribution for $Y \lvert X>Y$.
• Compare these conditional distributions with the unconditional ones with respect to mean and probability of passing.
• What is the effect of the new information on the test performance of each of the students?
• Explain why the new information has the effect on the test performance?

Practice Problem 3

• New information: $Y=X+1$.
• Derive the conditional distribution for $X \lvert Y=X+1$.
• Derive the conditional distribution for $Y \lvert Y=X+1$.
• Compare these conditional distributions with the unconditional ones with respect to mean and probability of passing.
• What is the effect of the new information on the test performance of each of the students?
• Explain why the new information has the effect on the test performance?

_____________________________________________________________________________________

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$

$\text{ }$
_____________________________________________________________________________________

To let you know that you are on the right track, the conditional distributions are given below.

The thought process of how to work with these practice problems can be found in the blog post Conditional Distributions, Part 1.

Practice Problem 1

$\displaystyle P(X=0 \lvert X

$\displaystyle P(X=1 \lvert X

$\displaystyle P(X=2 \lvert X

$\displaystyle P(X=3 \lvert X

$\displaystyle P(X=4 \lvert X

____________________

$\displaystyle P(Y=1 \lvert X

$\displaystyle P(Y=2 \lvert X

$\displaystyle P(Y=3 \lvert X

$\displaystyle P(Y=4 \lvert X

$\displaystyle P(Y=5 \lvert X

Practice Problem 2

$\displaystyle P(X=1 \lvert X>Y)=\frac{5}{3386}=0.0013$

$\displaystyle P(X=2 \lvert X>Y)=\frac{160}{3386}=0.04$

$\displaystyle P(X=3 \lvert X>Y)=\frac{1060}{3386}=0.2728$

$\displaystyle P(X=4 \lvert X>Y)=\frac{1880}{3386}=0.4838$

$\displaystyle P(X=5 \lvert X>Y)=\frac{781}{3386}=0.2$

____________________

$\displaystyle P(Y=0 \lvert X>Y)=\frac{31}{3386}=0.008$

$\displaystyle P(Y=1 \lvert X>Y)=\frac{390}{3386}=0.1$

$\displaystyle P(Y=2 \lvert X>Y)=\frac{1440}{3386}=0.37$

$\displaystyle P(Y=3 \lvert X>Y)=\frac{1620}{3386}=0.417$

$\displaystyle P(Y=4 \lvert X>Y)=\frac{405}{3386}=0.104$

Practice Problem 3

$\displaystyle P(X=0 \lvert Y=X+1)=\frac{15}{8430}=0.002$

$\displaystyle P(X=1 \lvert Y=X+1)=\frac{450}{8430}=0.053$

$\displaystyle P(X=2 \lvert Y=X+1)=\frac{2700}{8430}=0.32$

$\displaystyle P(X=3 \lvert Y=X+1)=\frac{4050}{8430}=0.48$

$\displaystyle P(X=4 \lvert Y=X+1)=\frac{1215}{8430}=0.144$

____________________

$\displaystyle P(Y=1 \lvert Y=X+1)=\frac{15}{8430}=0.002$

$\displaystyle P(Y=2 \lvert Y=X+1)=\frac{450}{8430}=0.053$

$\displaystyle P(Y=3 \lvert Y=X+1)=\frac{2700}{8430}=0.32$

$\displaystyle P(Y=4 \lvert Y=X+1)=\frac{4050}{8430}=0.48$

$\displaystyle P(Y=5 \lvert Y=X+1)=\frac{1215}{8430}=0.144$

_____________________________________________________________________________________

$\copyright \ 2013 \text{ by Dan Ma}$

## Another Example on Calculating Covariance

In a previous post called An Example on Calculating Covariance, we calculated the covariance and correlation coefficient of a discrete joint distribution where the conditional mean $E(Y \lvert X=x)$ is a linear function of $x$. In this post, we give examples in the continuous case. Problem A is worked out and Problem B is left as exercise.

The examples presented here are also found in the post called Another Example of a Joint Distribution. Some of the needed calculations are found in this previous post.

____________________________________________________________________

Problem A
Let $X$ be a random variable with the density function $f_X(x)=\alpha^2 \ x \ e^{-\alpha x}$ where $x>0$. For each realized value $X=x$, the conditional variable $Y \lvert X=x$ is uniformly distributed over the interval $(0,x)$, denoted symbolically by $Y \lvert X=x \sim U(0,x)$. Obtain solutions for the following:

1. Calculate the density function, the mean and the variance for the conditional variable $Y \lvert X=x$.
2. Calculate the density function, the mean and the variance for the conditional variable $X \lvert Y=y$.
3. Use the fact that the conditional mean $E(Y \lvert X=x)$ is a linear function of $x$ to calculate the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$.

Problem B
Let $X$ be a random variable with the density function $f_X(x)=4 \ x^3$ where $0. For each realized value $X=x$, the conditional variable $Y \lvert X=x$ is uniformly distributed over the interval $(0,x)$, denoted symbolically by $Y \lvert X=x \sim U(0,x)$. Obtain solutions for the following:

1. Calculate the density function, the mean and the variance for the conditional variable $Y \lvert X=x$.
2. Calculate the density function, the mean and the variance for the conditional variable $X \lvert Y=y$.
3. Use the fact that the conditional mean $E(Y \lvert X=x)$ is a linear function of $x$ to calculate the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$.

____________________________________________________________________

Background Results

Here’s the idea behind the calculation of correlation coefficient in this post. Suppose $X$ and $Y$ are jointly distributed. When the conditional mean $E(Y \lvert X=x)$ is a linear function of $x$, that is, $E(Y \lvert X=x)=a+bx$ for some constants $a$ and $b$, it can be written as the following:

$\displaystyle E(Y \lvert X=x)=\mu_Y + \rho \ \frac{\sigma_Y}{\sigma_X} \ (x - \mu_X)$

Here, $\mu_X=E(X)$ and $\mu_Y=E(Y)$. The notations $\sigma_X$ and $\sigma_Y$ refer to the standard deviation of $X$ and $Y$, respectively. Of course, $\rho$ refers to the correlation coefficient in the joint distribution of $X$ and $Y$ and is defined by:

$\displaystyle \rho=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y}$

where $Cov(X,Y)$ is the covariance of $X$ and $Y$ and is defined by

$Cov(X,Y)=E[(X-\mu_X) \ (Y-\mu_Y)]$

or equivalently by $Cov(X,Y)=E(X,Y)-\mu_X \mu_Y$.

Just to make it clear, in the joint distribution of $X$ and $Y$, if the conditional mean $E(X \lvert Y=y)$ is a linear function of $y$, then we have:

$\displaystyle E(X \lvert Y=y)=\mu_X + \rho \ \frac{\sigma_X}{\sigma_Y} \ (y - \mu_Y)$

____________________________________________________________________

Discussion of Problem A

Problem A-1

Since for each $x$, $Y \lvert X=x$ has the uniform distribution $U(0,x)$, we have the following:

$\displaystyle f_{Y \lvert X=x}=\frac{1}{x}$ for $x>0$

$\displaystyle E(Y \lvert X=x)=\frac{x}{2}$

$\displaystyle Var(Y \lvert X=x)=\frac{x^2}{12}$

Problem A-2

In a previous post called Another Example of a Joint Distribution, the joint density function of $X$ and $Y$ is calculated to be: $f_{X,Y}(x,y)=\alpha^2 \ e^{-\alpha x}$. In the same post, the marginal density of $Y$ is calculated to be: $f_Y(y)=\alpha e^{-\alpha y}$ (exponentially distributed). Thus we have:

\displaystyle \begin{aligned} f_{X \lvert Y=y}(x \lvert y)&=\frac{f_{X,Y}(x,y)}{f_Y(y)} \\&=\frac{\alpha^2 \ e^{-\alpha x}}{\alpha \ e^{-\alpha \ y}} \\&=\alpha \ e^{-\alpha \ (x-y)} \text{ where } y

Thus the conditional variable $X \lvert Y=y$ has an exponential distribution that is shifted to the right by the amount $y$. Thus we have:

$\displaystyle E(X \lvert Y=y)=\frac{1}{\alpha}+y$

$\displaystyle Var(Y \lvert X=x)=\frac{1}{\alpha^2}$

Problem A-3

To compute the covariance $Cov(X,Y)$, one approach is to use the definition indicated above (to see this calculation, see Another Example of a Joint Distribution). Here we use the idea that the conditional mean $\displaystyle E(Y \lvert X=x)$ is linear in $x$. From the previous post Another Example of a Joint Distribution, we have:

$\displaystyle \sigma_X=\frac{\sqrt{2}}{\alpha}$

$\displaystyle \sigma_Y=\frac{1}{\alpha}$

Plugging in $\sigma_X$ and $\sigma_Y$, we have the following calculation:

$\displaystyle \rho \ \frac{\sigma_Y}{\sigma_X}=\frac{1}{2}$

$\displaystyle \rho = \frac{\sigma_X}{\sigma_Y} \times \frac{1}{2}=\frac{\sqrt{2}}{2}=\frac{1}{\sqrt{2}}=0.7071$

$\displaystyle Cov(X,Y)=\rho \ \sigma_X \ \sigma_Y=\frac{1}{\alpha^2}$

____________________________________________________________________

Problem B-1

$\displaystyle E(Y \lvert X=x)=\frac{x}{2}$

$\displaystyle Var(Y \lvert X=x)=\frac{x^2}{12}$

Problem B-2

$\displaystyle f_{X \lvert Y=y}(x \lvert y)=\frac{4 \ x^2}{1-y^3}$ where $0 and $y

Problem B-3

$\displaystyle \rho=\frac{\sqrt{3}}{2 \ \sqrt{7}}=0.3273268$

$\displaystyle Cov(X,Y)=\frac{1}{75}$

____________________________________________________________________
$\copyright \ 2013$

## Another Example of a Joint Distribution

In an earlier post called An Example of a Joint Distribution, we worked a problem involving a joint distribution that is constructed from taking product of a conditional distribution and a marginial distribution (both discrete distributions). In this post, we work on similar problems for the continuous case. We work problem A. Problem B is left as exercises.

_________________________________________________________________

Problem A
Let $X$ be a random variable with the density function $f_X(x)=\alpha^2 \ x \ e^{-\alpha x}$ where $x>0$. For each realized value $X=x$, the conditional variable $Y \lvert X=x$ is uniformly distributed over the interval $(0,x)$, denoted symbolically by $Y \lvert X=x \sim U(0,x)$. Obtain solutions for the following:

1. Discuss the joint density function for $X$ and $Y$.
2. Calculate the marginal distribution of $X$, in particular the mean and variance.
3. Calculate the marginal distribution of $Y$, in particular, the density function, mean and variance.
4. Use the joint density in part A-1 to calculate the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$.

_________________________________________________________________

Problem B
Let $X$ be a random variable with the density function $f_X(x)=4 \ x^3$ where $0. For each realized value $X=x$, the conditional variable $Y \lvert X=x$ is uniformly distributed over the interval $(0,x)$, denoted symbolically by $Y \lvert X=x \sim U(0,x)$. Obtain solutions for the following:

1. Discuss the joint density function for $X$ and $Y$.
2. Calculate the marginal distribution of $X$, in particular the mean and variance.
3. Calculate the marginal distribution of $Y$, in particular, the density function, mean and variance.
4. Use the joint density in part B-1 to calculate the covariance $Cov(X,Y)$ and the correlation coefficient $\rho$.

_________________________________________________________________

Discussion of Problem A

Problem A-1

The support of the joint density function $f_{X,Y}(x,y)$ is the unbounded lower triangle in the xy-plane (see the shaded region in green in the figure below).

Figure 1

The unbounded green region consists of vertical lines: for each $x>0$, $y$ ranges from $0$ to $x$ (the red vertical line in the figure below is one such line).

Figure 2

For each point $(x,y)$ in each vertical line, we assign a density value $f_{X,Y}(x,y)$ which is a positive number. Taken together these density values sum to 1.0 and describe the behavior of the variables $X$ and $Y$ across the green region. If a realized value of $X$ is $x$, then the conditional density function of $Y \lvert X=x$ is:

$\displaystyle f_{Y \lvert X=x}(y \lvert x)=\frac{f_{X,Y}(x,y)}{f_X(x)}$

Thus we have $f_{X,Y}(x,y) = f_{Y \lvert X=x}(y \lvert x) \times f_X(x)$. In our problem at hand, the joint density function is:

\displaystyle \begin{aligned} f_{X,Y}(x,y)&=f_{Y \lvert X=x}(y \lvert x) \times f_X(x) \\&=\frac{1}{x} \times \alpha^2 \ x \ e^{-\alpha x} \\&=\alpha^2 \ e^{-\alpha x} \end{aligned}

As indicated above, the support of $f_{X,Y}(x,y)$ is the region $x>0$ and $0 (the region shaded green in the above figures).

Problem A-2

The unconditional density function of $X$ is $f_X(x)=\alpha^2 \ x \ e^{-\alpha x}$ (given above in the problem) is the density function of the sum of two independent exponential variables with the common density $f(x)=\alpha e^{-\alpha x}$ (see this blog post for the derivation using convolution method). Since $X$ is the independent sum of two identical exponential distributions, the mean and variance of $X$ is twice that of the same item of the exponential distribution. We have:

$\displaystyle E(X)=\frac{2}{\alpha}$

$\displaystyle Var(X)=\frac{2}{\alpha^2}$

Problem A-3

To find the marginal density of $Y$, for each applicable $y$, we need to sum out the $x$. According to the following figure, for each $y$, we sum out all $x$ values in a horizontal line such that $y (see the blue horizontal line).

Figure 3

Thus we have:

\displaystyle \begin{aligned} f_Y(y)&=\int_y^\infty f_{X,Y}(x,y) \ dy \ dx \\&=\int_y^\infty \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\alpha \int_y^\infty \alpha \ e^{-\alpha x} \ dy \ dx \\&= \alpha e^{-\alpha y} \end{aligned}

Thus the marginal distribution of $Y$ is an exponential distribution. The mean and variance of $Y$ are:

$\displaystyle E(Y)=\frac{1}{\alpha}$

$\displaystyle Var(Y)=\frac{1}{\alpha^2}$

Problem A-4

The covariance of $X$ and $Y$ is defined as $Cov(X,Y)=E[(X-\mu_X) (Y-\mu_Y)]$, which is equivalent to:

$\displaystyle Cov(X,Y)=E(X Y)-\mu_X \mu_Y$

where $\mu_X=E(X)$ and $\mu_Y=E(Y)$. Knowing the joint density $f_{X,Y}(x,y)$, we can calculate $Cov(X,Y)$ directly. We have:

\displaystyle \begin{aligned} E(X Y)&=\int_0^\infty \int_0^x xy \ f_{X,Y}(x,y) \ dy \ dx \\&=\int_0^\infty \int_0^x xy \ \alpha^2 \ e^{-\alpha x} \ dy \ dx \\&=\int_0^\infty \frac{\alpha^2}{2} \ x^3 \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \int_0^\infty \frac{\alpha^4}{3!} \ x^{4-1} \ e^{-\alpha x} \ dy \ dx \\&= \frac{3}{\alpha^2} \end{aligned}

Note that the last integrand in the last integral in the above derivation is that of a Gamma distribution (hence the integral is 1.0). Now the covariance of $X$ and $Y$ is:

$\displaystyle Cov(X,Y)=\frac{3}{\alpha^2}-\frac{2}{\alpha} \frac{1}{\alpha}=\frac{1}{\alpha^2}$

The following is the calculation of the correlation coefficient:

\displaystyle \begin{aligned} \rho&=\frac{Cov(X,Y)}{\sigma_X \ \sigma_Y} = \frac{\displaystyle \frac{1}{\alpha^2}}{\displaystyle \frac{\sqrt{2}}{\alpha} \ \frac{1}{\alpha}} \\&=\frac{1}{\sqrt{2}} = 0.7071 \end{aligned}

Even without the calculation of $\rho$, we know that $X$ and $Y$ are positively and quite strongly correlated. The conditional distribution of $Y \lvert X=x$ is $U(0,x)$ which increases with $x$. The calculation of $Cov(X,Y)$ and $\rho$ confirms our observation.

_________________________________________________________________

Problem B-1

$\displaystyle f_{X,Y}(x,y)=4 \ x^2$ where $x>0$, and $0.

Problem B-2

$\displaystyle E(X)=\frac{4}{5}$
$\displaystyle Var(X)=\frac{2}{75}$

Problem B-3

$\displaystyle f_Y(y)=\frac{4}{3} \ (1- y^3)$

$\displaystyle E(Y)=\frac{2}{5}$

$\displaystyle Var(Y)=\frac{14}{225}$

Problem B-4

$\displaystyle Cov(X,Y)=\frac{1}{75}$

$\displaystyle \rho = \frac{\sqrt{3}}{2 \sqrt{7}}=0.327327$

_________________________________________________________________

## Mixing Bowls of Balls

We present problems involving mixture distributions in the context of choosing bowls of balls, as well as related problems involving Bayes’ formula. Problem 1a and Problem 1b are discussed. Problem 2a and Problem 2b are left as exercises.

____________________________________________________________

Problem 1a
There are two identical looking bowls. Let’s call them Bowl 1 and Bowl 2. In Bowl 1, there are 1 red ball and 4 white balls. In Bowl 2, there are 4 red balls and 1 white ball. One bowl is selected at random and its identify is kept from you. From the chosen bowl, you randomly select 5 balls (one at a time, putting it back before picking another one). What is the expected number of red balls in the 5 selected balls? What the variance of the number of red balls?

Problem 1b
Use the same information in Problem 1a. Suppose there are 3 red balls in the 5 selected balls. What is the probability that the unknown chosen bowl is Bowl 1? What is the probability that the unknown chosen bowl is Bowl 2?

____________________________________________________________

Problem 2a
There are three identical looking bowls. Let’s call them Bowl 1, Bowl 2 and Bowl 3. Bowl 1 has 1 red ball and 9 white balls. Bowl 2 has 4 red balls and 6 white balls. Bowl 3 has 6 red balls and 4 white balls. A bowl is chosen according to the following probabilities:

\displaystyle \begin{aligned}\text{Probabilities:} \ \ \ \ \ &P(\text{Bowl 1})=0.6 \\&P(\text{Bowl 2})=0.3 \\&P(\text{Bowl 3})=0.1 \end{aligned}

The bowl is chosen so that its identity is kept from you. From the chosen bowl, 5 balls are selected sequentially with replacement. What is the expected number of red balls in the 5 selected balls? What is the variance of the number of red balls?

Problem 2b
Use the same information in Problem 2a. Given that there are 4 red balls in the 5 selected balls, what is the probability that the chosen bowl is Bowl i, where $i = 1,2,3$?

____________________________________________________________
Solution – Problem 1a

Problem 1a is a mixture of two binomial distributions and is similar to Problem 1 in the previous post Mixing Binomial Distributions. Let $X$ be the number of red balls in the 5 balls chosen from the unknown bowl. The following is the probability function:

$\displaystyle P(X=x)=0.5 \binom{5}{x} \biggl[\frac{1}{5}\biggr]^x \biggl[\frac{4}{5}\biggr]^{4-x}+0.5 \binom{5}{x} \biggl[\frac{4}{5}\biggr]^x \biggl[\frac{1}{5}\biggr]^{4-x}$

where $X=0,1,2,3,4,5$.

The above probability function is the weighted average of two conditional binomial distributions (with equal weights). Thus the mean (first moment) and the second moment of $X$ would be the weighted averages of the two same items of the conditional distributions. We have:

$\displaystyle E(X)=0.5 \biggl[ 5 \times \frac{1}{5} \biggr] + 0.5 \biggl[ 5 \times \frac{4}{5} \biggr] =\frac{5}{2}$
$\displaystyle E(X^2)=0.5 \biggl[ 5 \times \frac{1}{5} \times \frac{4}{5} +\biggl( 5 \times \frac{1}{5} \biggr)^2 \biggr]$

$\displaystyle + 0.5 \biggl[ 5 \times \frac{4}{5} \times \frac{1}{5} +\biggl( 5 \times \frac{4}{5} \biggr)^2 \biggr]=\frac{93}{10}$
$\displaystyle Var(X)=\frac{93}{10} - \biggl( \frac{5}{2} \biggr)^2=\frac{61}{20}=3.05$

See Mixing Binomial Distributions for a more detailed explanation of the calculation.

____________________________________________________________

Solution – Problem 1b
As above, let $X$ be the number of red balls in the 5 selected balls. The probability $P(X=3)$ must account for the two bowls. Thus it is obtained by mixing two binomial probabilities:

$\displaystyle P(X=3)=\frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2+\frac{1}{2} \binom{5}{3} \biggl(\frac{4}{5}\biggr)^3 \biggl(\frac{1}{5}\biggr)^2$

The following is the conditional probability $P(\text{Bowl 1} \lvert X=3)$:

\displaystyle \begin{aligned} P(\text{Bowl 1} \lvert X=3)&=\frac{\displaystyle \frac{1}{2} \binom{5}{3} \biggl(\frac{1}{5}\biggr)^3 \biggl(\frac{4}{5}\biggr)^2}{P(X=3)} \\&=\frac{16}{16+64} \\&=\frac{1}{5} \end{aligned}

Thus $\displaystyle P(\text{Bowl 1} \lvert X=3)=\frac{4}{5}$

____________________________________________________________

Problem 2a
Let $X$ be the number of red balls in the 5 balls chosen random from the unknown bowl.

$E(X)=1.2$
$Var(X)=1.56$

Problem 2b

$\displaystyle P(\text{Bowl 1} \lvert X=4)=\frac{27}{4923}=0.0055$

$\displaystyle P(\text{Bowl 2} \lvert X=4)=\frac{2304}{4923}=0.4680$

$\displaystyle P(\text{Bowl 3} \lvert X=4)=\frac{2592}{4923}=0.5265$