9 Conditional Probability

9.1 Objectives

Define conditional probability and distinguish it from joint probability.
Find a conditional probability using its definition.
Using conditional probability, determine whether two events are independent.
Apply Bayes’ Rule mathematically and via simulation.

9.2 Homework

9.2.1 Problem 1

Consider Exercise 1 from Lesson 2. Recall: \(A\), \(B\) and \(C\) are events such that \(\mbox{P}(A)=0.5\), \(\mbox{P}(B)=0.3\), \(\mbox{P}(C)=0.4\), \(\mbox{P}(A \cap B)=0.2\), \(\mbox{P}(B \cap C)=0.12\), \(\mbox{P}(A \cap C)=0.1\), and \(\mbox{P}(A \cap B \cap C)=0.05\).

Are \(A\) and \(B\) independent?

No. \(\mbox{P}(A)\mbox{P}(B)=0.15\neq \mbox{P}(A\cap B)\).

Are \(B\) and \(C\) independent?

Yes. \(\mbox{P}(B)\mbox{P}(C)=0.12 = \mbox{P}(B\cap C)\). Also, \[ \mbox{P}(B|C)=\frac{\mbox{P}(B\cap C)}{\mbox{P}(C)}= 0.12/0.4 = 0.3 =\mbox{P}(B) \]

9.2.2 Problem 2

Suppose I have a biased coin (the probability I flip a heads is 0.6). I flip that coin twice. Assume that the coin is memoryless (flips are independent of one another).

What is the probability that the second flip results in heads?

0.6

What is the probability that the second flip results in heads, given the first also resulted in heads?

The coin is memoryless. So, \[ \mbox{P}(\mbox{2nd flip heads}|\mbox{1st flip heads}) = 0.6 \]

What is the probability both flips result in heads?

Since the flips are independent, \[ \mbox{P}(\mbox{both heads})=\mbox{P}(\mbox{1st flip heads})\mbox{P}(\mbox{2nd flip heads}) = 0.6*0.6=0.36 \]

What is the probability exactly one coin flip results in heads?

This could happen in two ways. The first could be heads OR the second could be heads. \[ \mbox{P}(\mbox{exactly one heads})=\mbox{P}(\mbox{1st flip heads})\mbox{P}(\mbox{2nd flip tails}) + \mbox{P}(\mbox{1st flip tails})\mbox{P}(\mbox{2nd flip heads}) \] \[ 0.6*0.4+0.4*0.6 = 0.48 \]

Now assume I flip the coin five times. What is the probability the result is 5 heads? \[ \mbox{P}(\mbox{5 heads})= 0.6^5 = 0.0778 \]

0.6^5

## [1] 0.07776

What is the probability the result is exactly 2 heads (out of 5 flips)?

There are \(\binom{5}{2} = 10\) ways for this to happen ({HHTTT},{HTHTT},…). So, \[ \mbox{P}(\mbox{2 heads out of 5 flips})=\binom{5}{2} 0.6^2(1-0.6)^3 = 0.2304 \]

choose(5,2)*0.6^2*0.4^3

## [1] 0.2304

9.2.3 Problem 3

(Adapted from IPSUR, (Kerns 2010)). Suppose there are three assistants working at a company: Moe, Larry and Curly. All three assist with a filing process. Only one filing assistant is needed at a time. Moe assists 60% of the time, Larry assists 30% of the time and Curly assists the remaining 10% of the time. Occasionally, they make errors (misfiles); Moe has a misfile rate of 0.01, Larry has a misfile rate of 0.025, and Curly has a rate of 0.05. Suppose a misfile was discovered, but it is unknown who was on schedule when it occurred. Who is most likely to have committed the misfile? Calculate the probabilities for each of the three assistants.

Let \(E\) be the event a misfile was committed. Also, let \(M\), \(L\), and \(C\) denote the events that Moe, Larry and Curly was the assistant at the time, respectively.

\[ \mbox{P}(E)=\mbox{P}(E \cap M)+\mbox{P}(E \cap L)+\mbox{P}(E\cap C) \] \[ = \mbox{P}(E|M)\mbox{P}(M)+\mbox{P}(E|L)\mbox{P}(L)+\mbox{P}(E|C)\mbox{P}(C) = 0.01*0.6+0.025*0.3+0.05*0.1 = 0.0185 \]

Thus, \[ \mbox{P}(M|E)=\frac{\mbox{P}(E \cap M)}{\mbox{P}(E)}= \frac{0.01*0.6}{0.0185}=0.3243 \]

Similarly, \(\mbox{P}(L|E)=0.4054\) and \(\mbox{P}(C|E)=0.2702\).

Larry is the assistant most likely to have committed the error.

9.2.4 Problem 4

You are playing a game where there are two coins. One coin is fair and the other comes up heads 80% of the time. One coin is flipped 3 times and the result is three heads, what is the probability that the coin flipped is the fair coin? You will need to make an assumption about the probability of either coin being selected.

Use Bayes formula to solve this problem.

I will assume either coin is selected with a 50% probability.

\[ \mbox{P}(Fair) = \mbox{P}(Biased) = .5 \] \[ \mbox{P}(3 Heads|Fair)=\frac{1}{2}^3=\frac{1}{8} \] \[ \mbox{P}(3 Heads|Biased)=.8^3=0.512 \]

Now

\[ \mbox{P}(Fair | 3 Heads) = \frac{\mbox{P}(3 Heads | Fair)\mbox{P}(Fair)}{\mbox{P}(3 Heads | Fair)\mbox{P}(Fair)+\mbox{P}(3 Heads| Biased)\mbox{P}(Biased)} \]

Which is

\[ \mbox{P}(Fair | 3 Heads) = \frac{\frac{1}{8}\frac{1}{2}}{\frac{1}{8}\frac{1}{2}+.8^{3}\frac{1}{2}} = 0.196 \]

.125*.5/(.125*.5+.8^3*.5)

## [1] 0.1962323

Use simulation to solve this problem.

Let’s use the same assumptions. We could do this problem in two ways. We could flip each coin a fixed number of times and combine the information or use a random process to pick a flipped coin and then flip it three times. Let’s do the first.

Let’s flip a fair coin 50,000 times and count how many heads we get.

set.seed(1154)
data.frame(do(50000)*rflip(3)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 6157

Now flip the biased coin.

data.frame(do(50000)*rflip(3,prob=0.8)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 25743

So we have a total 6157 + 25743 heads of which 6157 came from the fair coin.

Thus the probability of the coin being fair given 3 heads on the flips is:

6157/(6157 + 25743)

## [1] 0.1930094

Or 19.3%.

Next pick a one of the coins with equal probability 100,000 times.

set.seed(501)
results <- rflip(100000,summarize = TRUE)
results

##       n heads tails prob
## 1 1e+05 50226 49774  0.5

Now the fair coin was flipped 50226 times.

Let’s see how many times we get 3 heads when we flip that coin 3 times.

data.frame(do(50226)*rflip(3)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 6270

We have 6270 cases with 3 heads. Now for the biased coin.

data.frame(do(49774)*rflip(3,prob=0.8)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 25512

Now we can determine the probability of a fair coin given 3 heads.

6270/(6270+25512)

## [1] 0.1972815

This code we could easily adapt if we don’t think each coin is being selected with the same frequency. Suppose we think the fair coin has a 75% chance of being selected. The analysis would look like this:

set.seed(9021)
results <- rflip(100000,prob=.75,summarize = TRUE)
results

##       n heads tails prob
## 1 1e+05 75023 24977 0.75

Now the fair coin was flipped 75023 times.

Let’s see how many times we get 3 heads when we flip that coin 3 times.

data.frame(do(75023)*rflip(3)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 9579

We have 9579 cases with 3 heads. Now for the biased coin.

data.frame(do(24977)*rflip(3,prob=0.8)) %>%
  filter(heads==3) %>%
  summarise(count=n()) %>%
  pull()

## [1] 12789

Now we can determine the probability of a fair coin given 3 heads.

9579/(9579+12789)

## [1] 0.4282457

A much different answer. That is because prior to getting the data we believed the fair coin would be selected with a 75% probability. The data indicates that we need to update and lower this probability. We only flipped 3 times but the evidence is so in favor of the biased coin, that our probability dropped substantially. This is why Bayes is such a powerful tool.

Think about what we just did with this problem. We started with a subjective believe that either coin would be selected with equal probability. This is called the prior probability. We then collected data on three flips of the coin. We used this empirical data to update our belief into a posterior probability. This is the basis for Bayesian statistical analysis. Bayesian statistics is an entire discipline unto itself.

8 Probability Rules

10 Random Variables