The purpose of this thread is to illustrate how counter-intuitive statistics can be.

Here's an example:

In the old TV show "Let's Make A Deal", Monte Hall would pick a contestant from the audience, tell her that a "shiny new car" was behind one of three doors, and ask her to pick one of the doors. The other two doors hid booby prizes. Monte would ask the pretty assistant to open one of the other two doors. Monte's door always hid a goat, or something equally useless. (Monte knew which door hid the car). Monte then turned back to the contestant and asked if she would like to stay with the door she originally picked or switch to the other door.

Which is the best strategy (stay or switch) statistically?

Hint: Most people think that the odds are the same (1/2) with either strategy. This is not correct. One strategy is much better than the other.

The purpose of this thread is to illustrate how counter-intuitive statistics can be.

Here's an example:

In the old TV show "Let's Make A Deal", Monte Hall would pick a contestant from the audience, tell her that a "shiny new car" was behind one of three doors, and ask her to pick one of the doors. The other two doors hid booby prizes. Monte would ask the pretty assistant to open one of the other two doors. Monte's door always hid a goat, or something equally useless. (Monte knew which door hid the car). Monte then turned back to the contestant and asked if she would like to stay with the door she originally picked or switch to the other door.

Which is the best strategy (stay or switch) statistically?

Hint: Most people think that the odds are the same (1/2) with either strategy. This is not correct. One strategy is much better than the other.
Switch!

Humm.. lets see. The first time he picked a door, the odds were 1/3 of that one being the right door. But during the other chance to pick, the second door (the one he did not choose, but can switch to) has a 1/2 odds of being the correct one.
Getting close?

Humm.. lets see. The first time he picked a door, the odds were 1/3 of that one being the right door. But during the other chance to pick, the second door (the one he did not choose, but can switch to) has a 1/2 odds of being the correct one.
Getting close?
Imagine that there were a hundred doors. You start by choosing one at random. Monty then opens 98 of the remaining doors, revealing goats galore. In this scenario, what do you think the probability is of having chosen the correct door at the start: 50% or 1% ?

(Hint: It's not 50%.)

Humm.. lets see. The first time he picked a door, the odds were 1/3 of that one being the right door. But during the other chance to pick, the second door (the one he did not choose, but can switch to) has a 1/2 odds of being the correct one.
Getting close?

Nope.

Switch!

Correct!

Here's one way of looking at this problem.
If your strategy is to stay, you have decided <i>a-priori</i> to ignore the evidence Monte shows you. Your original odds of 1/3 are not changed by what Monte shows (and besides, he can always show a goat.)

On the other hand, if your strategy is to switch, then
On the off-chance your original choice was correct, switching doors is a stupid strategy, but this happens only 1 out of 3 times.
When your original choice was incorrect, Monte is forced to show you the other losing door. The only door that is left has to hide the car. Switching doors is a smart strategy here, because you always win, and this happens 2 out of 3 times.

Therefore, your odds of winning with an <i>a-priori </i> strategy of switching doors is 2/3 (no chance of winning in the 1/3 times the original guess was correct + 100% chance of winning in the 2/3 times the original guess was incorrect).

Bayesian reasoning provides a much more mathematically grounded way of coming up with the same answer.

There's no reason to sweat it, though. Everybody gets it wrong the first time. Hell, even Paul Erdós got it wrong...

Nope.
Well I got the second probability wrong, but I was correct about the switch, wasn't I. I got a little close :p

In the old TV show "Let's Make A Deal", Monte Hall would pick a contestant from the audience, tell her that a "shiny new car" was behind one of three doors, and ask her to pick one of the doors. The other two doors hid booby prizes. Monte would ask the pretty assistant to open one of the other two doors. Monte's door always hid a goat, or something equally useless. (Monte knew which door hid the car). Monte then turned back to the contestant and asked if she would like to stay with the door she originally picked or switch to the other door.


The rules for Let's Make a Deal allowed Monty the option to not offer the switch. If the contestant picked the door with the car, Monty would always offer the switch. If the contestant didn't pick the door with the car, Monty did what Monty felt like doing.

Assuming Monty decision to offer the switch when the contestant didn't pick the door with the car was completely random (offer-1/2; not offer-1/2), both strategies offer the same probability of winning.

Picked door with car; switch offered: 1/3
*switch: never wins
*stick: always wins
Did not pick door with car; switch offered: 1/3
*switch: always wins
*stick: never wins
Did not pick door with car; switch not offered: 1/3
*Never wins

Imagine that there were a hundred doors. You start by choosing one at random. Monty then opens 98 of the remaining doors, revealing goats galore. In this scenario, what do you think the probability is of having chosen the correct door at the start: 50% or 1% ?

(Hint: It's not 50%.)

This was Marilyn vos Savant's way of explaining it. Exagerration works pretty well.

The rules for Let's Make a Deal allowed Monty the option to not offer the switch. If the contestant picked the door with the car, Monty would always offer the switch. If the contestant didn't pick the door with the car, Monty did what Monty felt like doing. Thanks for reminding people of that...the very fact that Monty was offering you a switch increased the odds that you had picked the correct door initially, since switch offers were only made 1/2 of the time for incorrect initial choices. This threw off the normal assumption that the last remaining door had a 2/3 probability of being correct.

Contestants were always most likely to lose, since switching doors didn’t improve your chances of winning and there was a 1/3 chance that you would simply lose immediately by picking the wrong door initially and not being offered a choice to switch. People always seem to leave that part out…

People always seem to leave that part out…

I seem to recall Monty did other things too like offer you cash to give up your door or sometimes he'd just open your door without giving you any choice at all, but I only saw the show when I was very young. In short, Monty was too unpredictible to analyze so the problem that's now become known as "The Monty Hall Problem" is the simplified version D H presented. If you can't answer a given problem try to answer an easier one, and that's exactly what's happened. Plus, this modified version is so counter-intuitive and yet very simple at the same time, it's no suprise it became so polular (the vos Savant article greatly helped too)

There are many generalizations going around, but they'll all include some concrete assumptions on Monty's behavior so the generalized problem is still solvable. Raphael's is one of those, and it assumes Monty switches with a given fixed probability of 1/2.You can also ask how he should choose this probability to increase his odds. Or modify in other ways like more doors, more prizes, more selections of doors, etc.

See H. Bailey, "Monty Hall Uses a Mixed Strategy", Mathematics Magazine, vol. 73, No. 2, April 2000, pages 135-141

has some generalizations and a good list of references.

Here's something for people to think about: On the show "Who Wants to be a Millionaire," there's an option where two incorrect answers of the four possible answers will be eliminated. Suppose you are on the show and have no idea what the correct answer is. You pick one at random and answer with it. The host then asks, "Is that your final answer?" and you reconsider. You decide to take to option to eliminate two incorrect answers. Your initial choice is still there, along with one other. At this point you might reason, "This is like the Monty Hall problem. There was initially only a 1/4 chance that I picked the correct answer, so there is a 3/4 chance that the other remaining answer is correct." Would that be right?

Depends on the elimination strategy. If your first (truly random) choice is never eliminated, then, yes, switching would be a good strategy.

If your first (wrong) choice can be eliminated, then it obviously isn't the Monty Hall problem anymore.

At this point you might reason, "This is like the Monty Hall problem. There was initially only a 1/4 chance that I picked the correct answer, so there is a 3/4 chance that the other remaining answer is correct." Would that be right? No that would not be right. After two of the incorrect answers are taken away, there are only two remaining. It doesn't matter what your odds were at the beginning. Probability doesn't know what your intentions were, or what you did. All that matters at this point is that there are two answers, and one solution: a 50% chance. It is the same with the door problem. You can't include irrelevant information (i.e. what happened before the door was revealed) in your calculation of the probability. It is still a 50% chance, whether or not you switch.

No that would not be right. After two of the incorrect answers are taken away, there are only two remaining. It doesn't matter what your odds were at the beginning. Probability doesn't know what your intentions were, or what you did. All that matters at this point is that there are two answers, and one solution: a 50% chance. It is the same with the door problem. You can't include irrelevant information (i.e. what happened before the door was revealed) in your calculation of the probability. It is still a 50% chance, whether or not you switch.
Again, this depends on the elimination strategy. If the quizmaster knows your first choice (i.e. you tell him), and never eliminates it when removing two incorrect choices, then it's not a fifty/fifty shot. Then it's 25%/75%.

Think about it

I introduced this thread with the Monte Hall problem. That's been covered fairly well. Here's an example of another paradox:

In 1981, a study of 326 defendants found guily of murder was undertaken to determine whether there were signs of discrimination in the penalties given out. The study was limited to cases where the defendant and the victim was either white or black. The racial breakdown of the defendants was 160 white and 166 black. The percentages of those given the death penalty were 11.88% of the white defendants and 10.24% of the black defendants. Not much of a difference here.

However, a different picture arises when one looks at the race of the victim. In the case of a white victim (214 out of the 326 total), 12.58% of the white defendants with a white victim received the death penalty as did 17.46% of the black defendants with a white victim. Similarly, none of the white defendants with a black victim received the death penalty but 5.8% of the black defendants with a black victim did receive the death penalty.

To summarize, the gross statistics have a slightly greater percentage of the white defendants receiving the death penalty than black defendants. But when broken down by the race of the victims, a significantly greater percentage of the black defendants received the death penalty in the case of both black and white defendants. Similar studies have been undertaken in several other states with similar results.

The same paradox appears in many other studies. Two more examples:

Bias in admissions to the University of California, Berkeley Graduate School. A smaller percentage of the male versus female applicants was accepted to each of six Berkeley graduate schoolsm but in aggregate, 44% of the male applicants were accepted versus only 35% of the female applicants.
On-time performance of one airline versus that of another. Airline A is better than airline B in aggregate, but airline B is better than airline A when the data are examined on a city-by-city basis.


What is going on here?

Money talks. I do not gamble now for ethical and religeous reasons. Thirty years ago, I discovered a sure-fire gambling system based on this same reasoning. A few days in Las Vegas really surprised me. I left there with less bucks than going in.

How many gillian dollars has this thread starter won in casinos and race tracks with this system? :bugeye:

Money talks. I do not gamble now for ethical and religeous reasons. Thirty years ago, I discovered a sure-fire gambling system based on this same reasoning. A few days in Las Vegas really surprised me. I left there with less bucks than going in.

How many gillian dollars has this thread starter won in casinos and race tracks with this system? :bugeye:Well, the thread starter is correct...under the normal statement of the Montey Hall problem (where the chance to switch is always offered) you will be 2/3 more likely to win if you switch.

HELLO! IS THIS MIKE TURNED ON?

HOW MUCH MONEY HAS THIS THREAD STARTER WON USING A GAMBLING SYSTEM BASED ON THIS LOGIC? :rolleyes:

How many gillian dollars has this thread starter won in casinos and race tracks with this system? :bugeye:
It's not a system, you dope.

HELLO! IS THIS MIKE TURNED ON?

HOW MUCH MONEY HAS THIS THREAD STARTER WON USING A GAMBLING SYSTEM BASED ON THIS LOGIC? :rolleyes:No casino is stupid enough to offer this as one of their games. That doesn't mean that the odds are wrong. On the contrary, it means that the odds are correct and a person could beat the house quite easily, and the house knows it.

Your point was irrelevant, that is why you didn't get any responses. "Shouting" and the :rolleyes: the second time just makes you look silly.

-Dale

I have looked silly so many times I'm accustomed to it. But, my most gracious thanks to you you for pointing out my silliness. I am eagerly anticipating the NEXT time that I can point out YOUR silliness, but have not yet done so.

You seem to have some measureable amount of knowledge about math, but, where physics is concerned, you shoot yourself in the foot so often that it is amazing if you have any toes left.

Do you not understand what a simple question is?

Did I present an arguement against any math or logic presented here? Or, did I simply ask a plain question?

Do you understand the difference?

Seriously, with no silliness, I tell you that I knew the difference between a question and a statement before I entered the first grade of school.

You? :rolleyes: :rolleyes:

How many gillian dollars has this thread starter won in casinos and race tracks with this system? :bugeye:The original poster never said anything about a gambling system. What makes you think that DH has ever used this in a gambling system?

There is no way to make a gambling system out of this, since no one (as far as I know) offers a game like the Montey Hall problem. Actually, even Montey didn't really offer a game like the Montey Hall problem...

The original poster never said anything about a gambling system. What makes you think that DH has ever used this in a gambling system?

There is no way to make a gambling system out of this, since no one (as far as I know) offers a game like the Montey Hall problem. Actually, even Montey didn't really offer a game like the Montey Hall problem...

It was a close approximation. The show obviously did not try to take money from the participants. The money they lost to the participants was chump change compared to the money they won by selling advertising time.

Do you not understand what a simple question is?"How many gillian dollars has this thread starter won in casinos and race tracks with this system?" sure seems like a rhetorical question.

I have looked silly so many times I'm accustomed to it. But, my most gracious thanks to you you for pointing out my silliness. I am eagerly anticipating the NEXT time that I can point out YOUR silliness, but have not yet done so.Well, I am sure you will not have to wait too long to be able to point one out :). My irritation at your previous post wasn't about its content, but rather the insistent all-caps "shouting". It just seemed rude to me. I guess I kind of expect that kind of rudeness from the "kook" element on this forum (since they usually have nothing substantive to say), but was somewhat upset to see it from a non-kook like you (since you usually make good and valid points).

You seem to have some measureable amount of knowledge about math, but, where physics is concerned, you shoot yourself in the foot so often that it is amazing if you have any toes left.I understand my physics limitations. I only ever need to use classical physics in my work, so whenever I opine about QM or GR related stuff I know that I am going out on a limb where my knowledge is weak. I have already been corrected many times on this forum and expect to be corrected many times in the future.

I won't address your "first-grade" remark. I don't want to get into an unnecessary argument with you since I think that we are generally in the same camp (despite this current thread). I just think that the rational element on this forum needs to hold itself to a higher standard of debate and civility. If we don't then, by being rude, I think we weaken our own claim on reason and logic. I apologize if you feel that I unfairly singled you out. You are probably right, I have indeed let much worse go without comment from others because I am not very concerned about how the fringe contingent on this forum presents itself.

-Sincerely
Dale

I introduced this thread with the Monte Hall problem. That's been covered fairly well. Here's an example of another paradox:

In 1981, a study of 326 defendants found guily of murder was undertaken to determine whether there were signs of discrimination in the penalties given out. The study was limited to cases where the defendant and the victim was either white or black. The racial breakdown of the defendants was 160 white and 166 black. The percentages of those given the death penalty were 11.88% of the white defendants and 10.24% of the black defendants. Not much of a difference here.

However, a different picture arises when one looks at the race of the victim. In the case of a white victim (214 out of the 326 total), 12.58% of the white defendants with a white victim received the death penalty as did 17.46% of the black defendants with a white victim. Similarly, none of the white defendants with a black victim received the death penalty but 5.8% of the black defendants with a black victim did receive the death penalty.

To summarize, the gross statistics have a slightly greater percentage of the white defendants receiving the death penalty than black defendants. But when broken down by the race of the victims, a significantly greater percentage of the black defendants received the death penalty in the case of both black and white defendants. Similar studies have been undertaken in several other states with similar results.

What is going on here?
Interesting stuff... I was sure that these numbers could not be correct and set out to prove it. To my surprise, I found that I was wrong!

Looking at the actual numbers helps:

Of the 326 cases,
There were 103 cases of black defendants with black victims, of whom 6 received the death penalty. (5.80%)
There were 63 cases of black defendants with white victims, of whom 11 received the death penalty. (17.46%)
There were 9 cases of white defendants with black victims, of whom 0 received the death penalty. (0.00%)
There were 151 cases of white defendants with white victims, of whom 19 received the death penalty. (12.58%)

Can you see what's going on now? The percentage numbers are not equally weighted. When adding sections together, the percentages are irrelevant - for example, the 17.46% item is only worth half the 12.58% item.

Interesting stuff... I was sure that these numbers could not be correct and set out to prove it. To my surprise, I found that I was wrong!

Can you see what's going on now? The percentage numbers are not equally weighted. When adding sections together, the percentages are irrelevant - for example, the 17.46% item is only worth half the 12.58% item.

Exactly. This is Simpson's Paradox, aka "their are lies, damn lies, and statistics". Percentages don't add (and one cannot average percentages) unless all the percentages are equally weighted.

Three other things are going on in this particular case:
More whites were murdered than blacks,
Victims and killers tend to be of the same race, and
Murderers of white victims are more likely to get the death penalty than murderers of black victims. (Independent statistics do bear this bias out.)

In the parlance of statisticians, these other things are called "lurking variables". I think of them as "ways to lie with statistics". Watch the news closely and you will see Simpson's Paradox lurking in many reported statistics.