Bayes Theorem -- Problem Solutions

OK we got a great set of responses to the two problems I posed yesterday to help discuss Bayes Theorem, and eventually its application to the game we all know and love. First, let's review the two problems in case you did not read yesterday and/or are too lazy to just scroll down a bit and read yesterday's post (and the corresponding comments) first:

Question #1: You are on the popular 70s game show Let's Make a Deal. Monty Hall is up there and he shows you 3 boxes on the stage, and asks you to pick one at random, explaining that under one box is a million dollars cash, but under the other two is a dead rodent of some kind. You choose Box #1. He then opens up Box #3 and shows you that there is in fact a dead rodent under that box. Now he offers you to either keep Box #1 or switch to Box #2, your choice. What do you do? Do you switch, do you keep your first box, or does it make any difference?

OK the answer to Question #1, as maybe half of the commenters intuited, is that you have to switch boxes here. Normally I would give my standard explanation that I have repeated many times to friends and family of mine, but I really like the way that Goat introduced it in his first comment to yesterday's post. So for starters I am just going to repeat what he said here:

"Always switch. I'm betting the comments I haven't read have already explained why, but the basic deal is when you pick box #1, there is an approx. 33% shot at a million bucks.

So you're holding 33%. The 'field' is holding 66%. When Monty lifts box #3 to show the dead rat, the 'field' is narrowed to box #2. Which still holds 66% equity. The 'field's' equity has not shrunk with your new information. You just happen to know which of the two boxes now holds all of the 66%."

In a nutshell, this is the reason why you should always switch boxes in this scenario, and why as a few of the commenters correctly pointed out, you literally double your chances of winning the million if you switch. Because at the time you chose box #1, you had a 33% chance of being right. We all can agree on that. Random pick of 1 out of 3 items. And that means that the odds that the money is in (either box #2 or box #3) = 66%. Now when box #3 is taken out of the equation, hopefully you can see that, since we know that the probability of the money being in (either box #2 or box #3) = 66%, now we know that the probability of the money being in (box #3) = 66%. Which makes sense, since we all know that we picked a random 1 out of 3 chance to get the money correct when the game began.

Let me just address quickly here those of you who make the very, very common argument that this is incorrect because obviously we have a 50-50 chance of having the money box since we have chosen one box out of two boxes total, or 1/2 = 50%. This would be the right answer if, instead of opening what he knew to be a dead rat box, Monty Hall had instead opened up a dead rat box, and then randomly spun box #1 and box #2 such that it was now an independent 50% chance that the money was behind either box #1 or box #2. But that is not the case here. In the example posed, we already had the existing probability of 33% chance of our box #1 being the box with the money, and that fact never changes. Again, if Monty eliminated box #3, and then followed up by hiding the other remaining two boxes and randomly reassigning the money to one of those two boxes, then of course you would be right, we are 50-50 to have the correct box with box #1. But here, we hold a 33% chance of being right, leaving 66% to the other two boxes, and when Monty eliminates one of those other two boxes (the "field", as Goat helpfully defines it in his comment above) without re-randomizing the remaining two boxes, then our box #1 retains its 33% chance, and box #2 doubles to the remaining 66% chance of being the one with the million inside.

This solution has proven over time to be a very, very difficult thing to explain, so I probably did not do a good enough job here for most of you, and I'm sorry about that. Despite having had this conversation with maybe 100 people in my day, I've never yet been able to devise the explanation that convinces everyone or even mostly everyone in any case. But rest assured, this explanation is the correct one. The reason I used this problem is that it is one of the best possible examples to show how the human brain tends to misapply what is essentially the concept underlying Bayes Theorem in applying new changes to probability when there is already an existing underlying probability assumption in place. And that fact in my opinion tends to spill right over onto the poker table as well with many people I run into on a nightly basis on the virtual felt.

Now on to Question #2 from yesterday:

Question #2 (this one comes directly from The Mathematics of Poker): Suppose doctors have a screening process for, say, Lupus, and that if a person is screened who actually has Lupus, the screening process will return a positive result for Lupus 80% of the time. Assume also that if a person who does not have Lupus is screened, the test will return a (false) positive result for Lupus 10% of the time. Lastly, assume that we know that 5% of the total population on average actually has Lupus.

A person is selected at random from the population at large and screened, and the test returns a positive result for Lupus. What is the likelihood that this person, who just tested positive for Lupus, actually has Lupus?

I was impressed with how many of the commenters seemed to get this one right on, which is great. I have read about this in some other studies, and I was really hoping that some of our doctor bloggers would have weighed in with their own thoughts, because in reality something like more than 75% of doctors gave answers or estimations to this very problem that were at a good 40-50 percentage points away from the right answer. The right answer, as a number of the commenters figured, is that even having received a positive test result, the chances that this randomly-selected patient does in fact have Lupus are 29.5%. This can be intuited very easily actually if you just pick an actual sample size for the total population, and work it out (something many of the commenters did):

So, assume the world has 1000 people in it. 50 of them will actually have Lupus (since 5% of the population on average has it). Of the 50 Lupus people, 40 of them will test positive given the 80% true positive rate that the test reports. Of the other 950 people who do not in fact have Lupus, 95 of them will test positive given the 10% false positive rate that the test reports. So, this means that a total of 135 people out of our 1000 population will test positive for Lupus under this screening process -- 40 of whom actually have Lupus, and 95 of whom actually do not. So, given the fact that someone is among the 135 people who test positive for Lupus, the odds are actually 40/135 that they actually have Lupus, and 95/135 that they do not. This equates to a 29.4% chance that the guy who tests positive actually is positive for Lupus.

So think about that again -- we have a test with an 80% effectiveness and only a low 10% false positive rate. That's actually pretty good. But in stark contrast to the analysis in some of the comments (Goat goat goat), the most relevant figure in this problem from a mathematical perspective is the fact that only 5% of the underlying population actually has Lupus, the relative rareness of which drags down the 80% effectiveness rate of the screening process to mean that even one positive test result still makes it only 29.6% likely that the subject actually has Lupus. Funny enough as I mentioned earlier, hundreds of doctors were asked this question, and I think more than 75% estimated the chances of the subject actually having Lupus to be somewhere between 70 and 80%. I myself originally estimated a higher number before reading about the simplicity of the solution that produces the 29.6% outcome, as did most of the people I have asked this question to myself over the past couple of days.

As I alluded to above, the results of both of the above two problems go to show how much the human mind is hard-wired to underweight the relative importance of existing, known probabilities and instead to put more importance than it should on later changes to that probability. Thus, even though there was already a 1/3 chance that your box was the money box in problem #1, the human mind by its intuitive nature tends to be more willing than it should be to believe that that probability is much higher than it actually is once there are now only two boxes left in the equation. Similarly, in problem #2, the inclination of most people is to significantly overweight the 80% effectiveness of the test, and to significantly underweight the already-given probability of 5% of the total population having the condition to begin with. Bayes Theorem and the concepts underlying it help us to understand and identify these situations where people commonly make faulty assumptions and faulty estimates -- sometimes arriving at the completely wrong result, such as the doctor who assumes that his positive-testing patient is 75% likely to have Lupus when in reality he is actually less than 30% to have the condition. These errors are commonly made at the poker table as well, and tomorrow I plan to discuss some of those situations where Bayes' rule can be used to help properly make such calculations.