An Impossible Coin Flip?

[BillyJoe]

This is a quote from Leonard Susskind (theoretical physicist):

Conversation With a Slow Student

Student: Hi Prof. I've got a problem. I decided to do a little probability experiment—you know, coin flipping—and check some of the stuff you taught us. But it didn't work.

Professor: Well I'm glad to hear that you're interested. What did you do?

Student: I flipped this coin 1,000 times. You remember, you taught us that the probability to flip heads is one half. I figured that meant that if I flip 1,000 times I ought to get 500 heads. But it didn't work. I got 513. What's wrong?

Professor: Yeah, but you forgot about the margin of error. If you flip a certain number of times then the margin of error is about the square root of the number of flips. For 1,000 flips the margin of error is about 30. So you were within the margin of error.

Student: Ah, now I get if. Every time I flip 1,000 times I will always get something between 970 and 1,030 heads. Every single time! Wow, now that's a fact I can count on.

Professor: No, no! What it means is that you will probably get between 970 and 1,030.

Student: You mean I could get 200 heads? Or 850 heads? Or even all heads?

Professor: Probably not.

Student: Maybe the problem is that I didn't make enough flips. Should I go home and try it 1,000,000 times? Will it work better?

Professor: Probably.

Student: Aw come on Prof. Tell me something I can trust. You keep telling me what probably means by giving me more probablies. Tell me what probability means without using the word probably.

Professor: Hmmm. Well how about this: It means I would be surprised if the answer were outside the margin of error.

Student: My god! You mean all that stuff you taught us about statistical mechanics and quantum mechanics and mathematical probability: all it means is that you'd personally be surprised if it didn't work?

Professor: Well, uh...

If I were to flip a coin a million times I'd be damn sure I wasn't going to get all heads. I'm not a betting man but I'd be so sure that I'd bet my life or my soul. I'd even go the whole way and bet a year's salary. I'm absolutely certain the laws of large numbers—probability theory—will work and protect me. All of science is based on it. But, I can't prove it and I don't really know why it works. That may be the reason why Einstein said, "God doesn't play dice." It probably is.

This sounds interesting but what actually is Susskind trying to say? Is he saying that it could not possibly happen (even though he can't prove it)? Surely there is some chance that it could happen?

Does anyone have any comments?


BillyJoe

 

[SGT]

You can't prove anything using probabilities. What you can have is a degree of confidence in the outcome of an experiment, based on the laws of probability.
The event of obtaining one million heads in the tossing of one million coins is so unlikely that anione would be very confident in betting his life against it.
If someone (God ?) started tossing one million coins each second since the Big Bang, it is very unlikely that in 2005 CE he would have already got one million heads in any individual toss.

 

[BillyJoe]

Sargeant,

Is that all he's saying?
What's all this about not being able to prove it then?

That's like saying "I have no chance of winning the lotto, well, not 'no chance' but the chance is as close to zero as to not matter......but.....I can't prove it!"

Perhaps he (Susskind) only sounds clever???

BillyJoe

 

[bjornart]

What it means is you can't explain probability without using the word probably.

 

[bjornart]

Sargeant,

Is that all he's saying?
What's all this about not being able to prove it then?

That's like saying "I have no chance of winning the lotto, well, not 'no chance' but the chance is as close to zero as to not matter......but.....I can't prove it!"

Perhaps he (Susskind) only sounds clever???

BillyJoe

No, that's not the bit that he can't prove. Look at the example and what's being asked.

"Student: Ah, now I get if. Every time I flip 1,000 times I will always get something between 970 and 1,030 heads. Every single time! Wow, now that's a fact I can count on."

No, it's not a fact you can count on, but it's probable. Increase the number and it's even more probable. But how do you prove that?

Why does it land on heads one time and tails the other? Differences in initial velocity? Different timing in catching it (if you don't let it drop to the ground)?

Why do the result of these variables sum up to the 50/50 probability we're used to for coin flips?

We don't know, and we can't prove anything.

 

[Donks]

Student: Ah, now I get if. Every time I flip 1,000 times I will always get something between 970 and 1,030 heads. Every single time! Wow, now that's a fact I can count on.

Professor: No, no! What it means is that you will probably get between 970 and 1,030. [/B]

Is that a typo or am I missing something? Seems to me it's quite impossible to get 1,030 anythings with only 1000 flips.

 

[pgwenthold]

But these are simple statistical statements that were being made. The short answer is that we know (assuming it is a fair coin), the probabilty of any result. If the standard error is 30, then the answer is that if you run the exercise a very large number of times, then about 2/3 of the time you will get between 470 and 530.

Sure, you can get 400, or 300 or even 100. However, the average number of trials you need to run to get at least one instance of 100 heads can be calculated, and it is very large. Most of the time, over a large number of trials, it will be between 470 and 530.

The statement about statistical mechanics works the same way. Sure, it's possible that everything could arrange in a peculiar way. However, given the number of objects (say, 1e23), the probability is so small that basically the likelyhood of finding that arrangement anywhere in the universe is very small.

 

[rppa]

This sounds interesting but what actually is Susskind trying to say? Is he saying that it could not possibly happen (even though he can't prove it)? Surely there is some chance that it could happen?

Does anyone have any comments?

I'm not sure what Susskind is trying to say, but perhaps it's related to this quote in the middle of the dialogue:

You keep telling me what probably means by giving me more probablies. Tell me what probability means without using the word probably.

Probability has been put on a firm theoretical basis starting from a small set of axioms, but there is still a point at which there is a fundamental concept of "probable" which can't be rigorously defined. The mathematics is sound, how we connect it to the real world is fuzzy and ultimately relies on a gut feeling for what "probably" means, as well as such things as "probability 0" or "probability 1". And there are at least two violently disagreeing schools of thought on the concept, the frequentists and the Bayesians. Someday there's going to be a civil war between the two and you'll all have to choose up sides.

 

[RussDill]

This is a quote from Leonard Susskind (theoretical physicist):



This sounds interesting but what actually is Susskind trying to say? Is he saying that it could not possibly happen (even though he can't prove it)? Surely there is some chance that it could happen?

Does anyone have any comments?


BillyJoe

If you flip a million coins enough times, I would be surprised if it did not happen

 

[Number Six]

I don't think the conversation should ever take place because the student should have taken basic statistics before he got to physics. Sure, physics can generate some complicated statistics problems but they're talking about a simple one. The problem is that the student doesn't understand what a probability distribution is, which he should learn about in his first statistics course.

Even the million flip case is straightforward...you can compute the probabilities (if you have some really powerful computers) and after you compute them then you have a probability distribution and that's that. The probability of X heads is Y. The probability of a million consecutive heads is _extremely_ small but not zero.

I guess maybe the question is that are there events whose probabilities are so small and yet nonzero that it becomes impossible for them to occur?...that is, if a probability becomes so small, does the probability become zero rather 0.athousandzeroes1? If that is true for some particular phenomenon then the problem is that the probability distribution becomes hard to specify. But it's a philosophical argument at that point since such a small probability can be ignored anyway.

 

[RussDill]

I think the point is that the testing of probabilty does not follow common sense. It is difficult to come up with a test that would disprove probability. No matter what result someone obtains from a test, it does not disprove probability, it is just exteremly unlikely.

That is why probability has its foundations in mathamatics, not observation.

 

[pgwenthold]

I guess maybe the question is that are there events whose probabilities are so small and yet nonzero that it becomes impossible for them to occur?...that is, if a probability becomes so small, does the probability become zero rather 0.athousandzeroes1? If that is true for some particular phenomenon then the problem is that the probability distribution becomes hard to specify. But it's a philosophical argument at that point since such a small probability can be ignored anyway.

the usual consideration in this case is something like, "assume all the particles in the universe were to do this every picosecond." If the probability is so low that you would likely not observe it even if you used all the particles in the universe as your sample size, then it is far too say that it is more or less not going to happen.

But that is still a philosophical argument.

 

[Jorghnassen]

What it means is you can't explain probability without using the word probably.

Oh yes, you can. But you're not going to understand it (unless you're one of those really really smart math people) and it'll give you a headache.

/not very fond of measure theory...

 

[69dodge]

Re: Re: An Impossible Coin Flip?

Originally posted by rppa
And there are at least two violently disagreeing schools of thought on the concept, the frequentists and the Bayesians. Someday there's going to be a civil war between the two and you'll all have to choose up sides.

Choose Bayesians, obviously.

Frequentists have exactly the problem Susskind describes. If you try to explain the meaning of "a flipped coin will show heads with probability 1/2" by saying "in a long series of flips, the coin will show heads half the time" or even "in a long series of flips, the coin will show heads about half the time", you run into the problem that it isn't really true. It's only probably true. We can calculate exactly what the probability is that it's true, if we specify how long "long" is and how close "about" is, but that doesn't help us at all: if we don't know what "probability 1/2" means, why should we have any better idea of what any other probability means?

Practically speaking, you round off to 0 (i.e., impossibility) any probability that is sufficiently close to 0, and you round off to 1 (i.e., certainty) any probability that is sufficiently close to 1. But only as the very last step of a calculation. If you plan on using the calculated probability in a further calculation, you don't round it off; otherwise, for some further calculations, you will end up with a very wrong answer.

For example, if you flip a coin 100 times, it's practically certain that you will not get 100 heads. But if you repeat the whole 100-coin-flip experiment 2²⁰⁰ times, it's practically certain that at least one of the experiments will result in 100 heads. Had I rounded the first "practically certain" to "absolutely certain", I would not have gotten the correct answer in the second case.

 

[SGT]

What it means is you can't explain probability without using the word probably.

From Wikipedia:

Like other theories, the theory of probability is a representation of probabilistic concepts in formal terms -- that is, in terms that can be considered separately from their meaning. These formal terms are manipulated by the rules of mathematics and logic, and any results are then interpreted or translated back into the problem domain.
There have been at least two successful attempts to formalize probability, namely the Kolmogorov formulation and the Cox formulation. In Kolmogorov's formulation, sets are interpreted as events and probability itself as a measure on a class of sets. In Cox's formulation, probability is taken as a primitive (that is, not further analyzed) and the emphasis is on constructing a consistent assignment of probability values to propositions. In both cases, the laws of probability are the same, except for technical details:

1. a probability is a number between 0 and 1;
2. the probability of an event or proposition and its complement must add up to 1; and
3. the joint probability of two events or propositions is the product of the probability of one of them and the probability of the second, conditional on the first.

Probability has a very sound mathematic definition. What can be confusing is application of the theory of probability to real events.

 

[69dodge]

Originally posted by Jorghnassen
Oh yes, you can. But you're not going to understand it (unless you're one of those really really smart math people) and it'll give you a headache.

/not very fond of measure theory...

You can understand it as math. But how do you connect the mathematical theory with the real world? If the weather report says there's a 90% chance of rain today, should I take my umbrella or not? Measure theory doesn't help me answer that question.

 

[pgwenthold]

You can understand it as math. But how do you connect the mathematical theory with the real world?

Ask the stat mech people. It's an entire branch of chemistry based on statistics and probability.

Actually, most of life is far, far, far easier to describe using statistical theory than when trying to be discrete. My pet project is the description of sporting competitions as an exercise in phase space theory.

If the weather report says there's a 90% chance of rain today, should I take my umbrella or not? Measure theory doesn't help me answer that question.

More or less, it does. Do you want to get wet? If not, you want to bring your umbrella.

Now, when it gets to something like 50/50, then there is a good question, and it depends on how concerned you are about getting wet.

 

[69dodge]

Originally posted by pgwenthold
More or less, it does. Do you want to get wet? If not, you want to bring your umbrella.

That's the right answer, of course. But did you arrive at it using measure theory, or did you arrive at it using your intuitive understanding that "the chance of rain is 90%" means it will probably rain?

 

[Jorghnassen]

You can understand it as math. But how do you connect the mathematical theory with the real world? If the weather report says there's a 90% chance of rain today, should I take my umbrella or not? Measure theory doesn't help me answer that question.

Hey, my only claim was that you could explain probability without using the word "probably". I made no claim as to connecting it with the real world (hence the 'you wouldn't understand it...'). Anyway, weather forecasting is actually done through a deterministic approach (they basically pick the mode of a set of models) and that 'probability of precipitation' actually means "it will rain on 90% of the area covered by the forecast". Research has been and is being done to use a probabilistic approach to weather forecasting.

/trying to remember a seminar given by Prof. Adrian Raftery...

 

[SkepticalScience]

To 69 Dodge.

This is an interesting thread. . .

I do think I know the answer to 69 Dodge's question though.

If the weather man says there is a 70% chance of rain, I think all they mean is that from the time they started collecting wether data, for days with conditions similar to today, 70% of those days rained.