Double Headed Coins and skepticism

[Simon Bridge]

Yes it does. Some particular sequences are much more probable given cheating than random chance.

Yes - good - we agree on that - you did not put it that way to start with. Futher: the calculation you showed me does not take this into account. That is what I am saying - you need to adjust the math to take account of the liklyhood of cheating and you didn't. Bayes theorum does, as you point out below:

The calculation I have in mind is Pr(H/E) = Pr(E/H) x Pr(H) / Pr(E) (Bayes Theorem). if (H)pothesis is "the coin is fair" and (E)vidence is "100 heads in a row", then Pr(H/E) will drop to nearly zero.

Well - that's not the calculation you gave me before - Bayes theorum is what I have used in the paper ... you read it right? I used Bayes theorum explicitly.... by name.

You've handled it differently from me since I had to quantify my terms while you are the general idea qualitatively. Crunch the numbers and let me know if you get anything different to me.

The term 1/Pr(E) (your notation) is determined by normalization for example and has quite a big impact on the calculation. Pr(H) is the prior probability ... in the article I referred to it as the a-priori suspicion of honesty ... I used the same one. This actually has a strong effect on the outcome. Pr(E|H) (hint: use the "pipe" character - shift+backslash - for the conditional separator) is the forward probability or likelyhood function- which is the same for any specific sequence of heads and tails. C'mon, you know this! This is the term that is (0.5)^n for n tosses. That is the figure you kept quoting to me ... as it turns out - out of context. If you use this figure by itself for your hypothesis testing, you will reject the hypothesis too soon for a particular prior.

Now - you don't have to do it this way. Instead you can just rig the number of trials in a test to be so large it makes no difference since Pr(H|E)--->Pr(E|H) for n --> \inf ... as you point out

The trouble is (read first post) I don't want very big numbers of trials. I want to avoid the argument that I need a much larger number of trials than I have performed in order to be certain. To do this I need to model what happens for small numbers of trials ... in particular, where is the threshold? How do I adjust the liklyhood while evidence is still rolling (or tossing) in? Bayesian stats is great at this.

One of the principle results in the article is that the rejection thresholds change in the manner of a learning curve (go look at the graph) ... for high priors one may be quite justified in maintaining a strong belief for a counter-intuitively long time... but the prior has to be pretty strong (0.9999 was the strongest considered).

I'd have to reread the article.

Would it be more helpful if I posted the stats in this thread? I didn't want to post the whole thing since it is quite long, not final, and involves a lot of math which looks ugly here even in TeX

I was bogged down in the statistical derail. However, if you [the reader?] don't understand how statistically significant results, like a run of 100 heads, affect hypotheses, the article won't do you much good.

You don't really mean that all education is useless.? The usefulness of the article depends on how it is written and how it is expected to be used. Improving on it is why I asked for comment remember?

 

[Piggy]

I think that's what piggy's trying to describe: perhaps a string of identicals is attracting his attention more than other strings because there aren't a lot of special combinations out there he would recognize easily on account of being, well, normal, and not having a math degree or something.

No, that's not it. It's a long string of physical events occurring in a turbulent environment which reach the same end state. This puts it in a separate category from the types of strings of events -- of many various configurations -- which we would expect, for good reason, to result from actions within such an environment.

An extremely long string of identical end-states is not the kind of behavior that results from a situation such as a fair coin toss in the real world, where randomness is the primary forcer.

It's like if someone drops a metal cup and it goes clanking down a long stairway.

If it hits flat on the bottom of the cup two steps in a row, that's a coincidence. But if it does exactly three flips and hits on the bottom of the cup at each step all the way down the flight, then something's up, because the natural world doesn't work that way.

Now, you can do all your math and determine some small probability for that occurrence, but that doesn't matter unless you can demonstrate that your math completely describes the real world and isn't in any way idealized -- which, of course, you can't because it is idealized, as is the math being presented for the coin toss case.

 

[bokonon]

It's a long string of physical events occurring in a turbulent environment which reach the same end state.

It doesn't much matter how long the string of events is, or how turbulent the environment is, the "end state" of a flipped coin is constrained by gravity, the shape of a coin, and the nature of flat surfaces to be either "heads" or "tails."

All your handwaving about "reality's rough edges" doesn't mean squat in this system. If you flipped a coin 1,000,000,000,000,000,000,000,000,000,000,000 times, there would be a run of 100 in there somewhere.

 

[Piggy]

Piggy, if the rough edges of the real world are somehow intervening to stop the coin from turning up heads, perhaps you can answer the following question:

It's a fascinating question, and in a way it's one I don't have an answer for, but in another way I do have an answer.

Let's go back to the monkey example for a moment.

There is nothing that prevents a monkey from pressing the "u" key after he presses the "q" key.

And there's nothing that prevents a monkey from pressing the space bar after pressing the "n" key. Or from pressing the dot key after pressing the "g" key.

Yet despite that fact, observation of actual monkey-and-keyboard setups demonstrates that they will never produce the string "The quick brown fox jumped over the lazy dog." no matter how long we give them to do it.

Each pair is possible, and yet the extended string will never occur, because in reality they have a much narrower range of output than that at that length, with a lot of repetition of identical or similar patterns.

Similarly, in the case of the metal cup dropped down the long flight of stairs, it may hit on the same spot on two consecutive steps, but if it's a random drop (and not a carefully orchestrated effort) it will not make identical strikes all the way down, no matter how many times you drop it, because our rough and bumpy world doesn't work like that.

Now, precisely why is that the case? That, I couldn't tell you.

But a truly fair toss of a fair coin, in the real world, when repeated many times in sequence, is exactly the kind of thing that we should expect to reflect the messiness of physical reality. If we find that this messiness is absent -- which would certainly be the case if we got 100 heads or tails in a row -- we would be wise to conclude that the exercise had somehow been shielded from it, or in other words, that it was rigged.

 

[Piggy]

If you flipped a coin 1,000,000,000,000,000,000,000,000,000,000,000 times, there would be a run of 100 in there somewhere.

And why is that?

(Please do not answer with a purely mathematical model that is not a completely accurate description of the physical system.)

 

[Alan]

I suspect you'll get a Nobel prize in physics or a Fields Medal or something if you can prove what you are arguing for, that the laws of physics makes things of this likelihood impossible.

 

[Piggy]

I was making your monkey typing analogy fit the chances of getting a streak with a coin.

What an odd thing to do.

 

[Alan]

It is possible to get 10 heads in a row, right?

Why is it literally impossible to get 10 heads in a row, ten times in a row?

Or one heads in a row, 100 times in a row.

Each flip is approximately 50/50. It doesn't mysteriously change at some point to 0% tails, 100% heads.

 

[Piggy]

I suspect you'll get a Nobel prize in physics or a Fields Medal or something if you can prove what you are arguing for, that the laws of physics makes things of this likelihood impossible.

Oh, I'm sure I would,too. I would love to be able to prove it.

But I guess the short answer to your earlier question is that any combination of 2 results is consistent with the randomness inherent in the physical setup, while a series of 100 identical results is not.

By the way, can you prove that a string of fair flips of fair coins is adequately described by your math, that it is in fact a system which will eventually result in all possible full-length combinations?

 

[Alan]

What an odd thing to do.

We were talking about coins and then you started talking about the chance of a monkey typing out Shakespeare! It is entirely reasonable to bring the odds in your analogy to more closely fit the topic being discussed.

 

[Piggy]

It is possible to get 10 heads in a row, right?

Why is it literally impossible to get 10 heads in a row, ten times in a row?

According to the cited paper, we wouldn't expect a string of 10 unless we were performing more than 2,000 flips.

When it comes to flips, the terrain of results is bumpy on a large scale.

When the terrain we observe is, instead, smooth on a large scale, we conclude that we're no longer looking at a series of fair tosses.

Similarly, if we're looking at a large body of water and it lacks turbulence, currents, and tides, we conclude we're not observing the ocean.

 

[Piggy]

We were talking about coins and then you started talking about the chance of a monkey typing out Shakespeare! It is entirely reasonable to bring the odds in your analogy to more closely fit the topic being discussed.

Not if you understand the approach I'm taking to answering the question.

 

[Piggy]

You're still not making any sense. No mechanism is needed to ensure extended runs don't happen, they're simply rare enough that we don't expect to see them anyway. Reality predicts exactly the results we see without needing your bizarre contortions, so why do you insist on making them? Yes, flipping 100 heads is possible. No, you shouldn't hold your breath waiting for it to happen.

Reality shows a rugged terrain of results at large expanses of coin-toss space.

Why is this? I don't know if there's a satisfactory answer to be had at this point.

But nevertheless, that's what we see.

If we view a coin-toss space with a perfectly smooth terrain at large expanses, it is at odds with all experience for fair tosses of fair coins; however, it is perfectly consistent with coin-toss space for rigged coins or rigged tosses.

 

[Piggy]

Why does this "volatility" only affect flip sequences that humans happen to consider special, like runs of 100 heads, and not other sequences? Magic?

You might as well ask why only a perfectly calm state is considered special when observing the behavior of water in the ocean, while a virtually infinite number of dynamic states are considered nothing worth noting.

 

[Piggy]

If extended runs don't happen in practice, it's only because vast populations don't spend entire lifetimes flipping coins.

The mathematics has already been provided: 2¹⁰⁰ (1.2676506 × 10³⁰) tosses would, on average, be expected to yield one sequence which was either all heads or all tails. If you multiply that number of tosses by 100 (1.2676506 × 10³²), the result is virtually guaranteed.

If you have never seen the result, it's because no one has actually run the experiment.

Since the experiment hasn't been run in actual conditions, you have no grounds to insist that your mathematics actually describes the system.

 

[Piggy]

Why? I think that's what we're disagreeing about. I don't understand why you think this.

See post 154.

A large number of highly variable patterns are consistent with our observation of how fair tosses behave at a large scale.

A flatline of heads or tails is not one of those patterns, however.

 

[Alan]

Oh, I'm sure I would,too. I would love to be able to prove it.

But I guess the short answer to your earlier question is that any combination of 2 results is consistent with the randomness inherent in the physical setup, while a series of 100 identical results is not.

By the way, can you prove that a string of fair flips of fair coins is adequately described by your math, that it is in fact a system which will eventually result in all possible full-length combinations?

Randomness can include clumping, of 100 or of any other number. So yes, 100 can be consistent with randomness.

Something does not need to eventually result in order to be possible.

And fair flips of fair coins? A computer system is the way to test that out. But I know that's not what you mean to ask. I have to go soon so I won't respond in more detail at the moment.

 

[sol invictus]

Reality shows a rugged terrain of results at large expanses of coin-toss space.

I hope you realize that the above is utter nonsense. There is nothing about real sequences of coin flips that indicates anything other than that they behave as expected based on the standard laws of physics and probability. Long sequences of heads are rare, because that's what standard theory tells you. But it also tells you that they are not impossible, and it also explains why you've never seen one.

Not to put to fine a point on it, the position you are arguing for is unscientific, magical thinking based on your failure to comprehend basic statistics.

 

[Simon Bridge]

I undestand why you're frustrated, but I think the discussion is relevant to the content of your paper.

My interpretation of your paper is that it's a more rigourous version of "extraordinary claims require extraordinary evidence" - a claim with a high prior probability input is easier to accept with a lower confidence interval in the test protocol.[\quote]Hurray!

That was one of the things I'd hoped the curves illustrate - except that the confidence interval does not change with the prior.

In the prediction example, a believer in precognition will reject chance as an explanation for their results too soon. We could technically work the system backwards and produce the effective prior the person is using (unconsciously). For example - this skeptical audience will intuitively reject chance after 5 or so tosses ... which corresponds to a prior of 0.5. Probably this is due to experience of paranormal testing. 95% of the time we see someone saying "look at me I can predict 5 tosses in a row" they are cheating - the other 5% we'd expect them to repeat the performance what: about one time in 20?

We have to be careful about this though - our intuitive estimate of the prior kinda depends, amongst other things, on our assumptions about the context of the experiment.

I think this is part of the general discussion in other threads about prior probability and its relationship to what constitutes extraordinary claims requiring extraordinary evidence.

I'll have to disagree there - it is only slightly related. Those other threads are arguing over how one goes about determining the prior ... is a claim really all that extraordinary and how extraordinary is it? Is the extraordinariness of the evidence required justified by the extraordinariness of the claim? It is an important discussion which is not part of the paper - I have made no attempt to justfy the size of the prior.

Cearly you will need to have some justification, and the A Fair Coin Revisted has an example of how to go about it. But the main point of A Fair Coin? is to examine how assumptions about the prior affect the overall outcome. This can short-cut a lot of the debate in those other threads because, once you quantify the effect, you may find it is not such a big gap.

Using the model as an illustration: a mundane claim would have a low prior ... at p=0.5 you'd reject the hypothesis after only 5 or so contrary results. We don't need much evidence. Perhaps it fits well with everything else we feel we can prove?

This is why pseudoscience exists - to give the appearance that an odd claim is actually quite mundane. Then you can be tricked into giving it less scrutiny than it deserves.

An extraordinary claim is at the other end, see the graph for p=0.9999. We would want to see a lot of careful testing before we start to accept it but once those tests have been done, the acceptance gets high quite fast with subsequent supporting evidence. The claim is never completely accepted ... but at the top end of the curve it is very flat.

If, in the coin-toss thing, we have rigged the experiment so that we have eliminated every possible way of cheating that has ever been used, then we are going to need longer to conclude that something we don't already know about is going on with the coin. You'll see that emphasis on controlling variables in the experiment has already been mentioned.

And I think you're correct that it's something skeptics should be aware of - the take-away from this is that paranormal advocates will use exactly the same formulas (some have) to explain why skeptics are wrong. In their opinion, we have misinterpreted the body of literature and are undervaluing prior probability. They accuse us of insisting on tests with unjustifiably small confidence intervals to 'pass' our challenges.

Actually, they may be doing this because their estimate of the prior is very small. See how a quantitative example clears up this sort of discussion?

You still have the argument about what counts as a reasonable prior... fortunately, it is very unusual for a claimant to actually perform so far over odds as to make this sort of thing a problem in an actual test... but showing them the curves will help some of them understand why we expect longer tests.

You still have suspicion of cheating the other way though ... a believer may come to the conclusion that the test is biased in some underhand way because we are part of a cover up or something ... the model does not do much for irrational fears. But it may help guide us to an assessment of what counts as irrational.

Which is where I started.

It is not conclusive - it is not intended to be. Just one more discussion tool. Help me make it more useful... maybe I can turn it into a book later: I'll give you all credit

 

[sol invictus]

You might as well ask why only a perfectly calm state is considered special when observing the behavior of water in the ocean, while a virtually infinite number of dynamic states are considered nothing worth noting.

Indeed, and the question is just as pertinent there. Your position is analogous to asserting that such a perfectly calm state is physically impossible rather than simply very unlikely.

You have no basis for that claim, just as you have no basis for your claim about coin flips. We, on the other hand, have a very strong basis for ours (that it is possible).