Do you believe in Luck?

[stanfr]

I'm not a statistician, but I am a poker player, and the methodology here seems seriously flawed to me. I haven't read through the preceding 8 pages, since I think the first response to the OP was adequate, so forgive me if im missing something, but for starters,

1) how does the winning percentage of .56 constitute 'bad luck'. That's greater than even, so it is actually good luck if you are starting with a situation where the odds are close to even (and presumably, that situation was chosen for that very reason!)

2) The results are highly dependent on the whether ALL the cards are known, including those cards discarded by the other players. For example, If you hold AK but every other A and K has been discarded by the other players, than the odds of you pairing your hand is exactly zero. In that case, the other player holding a low pair has a huge advantage! Poker stats calculators generally don't account for the hands discarded (because from the relative standpoint of the player using those odds, the discarded cards are irrelevant, but from an absolute statistical view, they're crucial!) so your analysis is flawed from the start. The results you stated could be highly dependent on your husbands playing style etc. For example, it is unclear (or perhaps I missed it buried in the 8 previous pages of responses...) what percentage of time your husband was playing the overcards vs the pairs in these case--this seems to me to be a crucial question, and yet I'm unable to find an answer with my cursory look at all the info here...

 

[Beth]

I'm not a statistician, but I am a poker player, and the methodology here seems seriously flawed to me. I haven't read through the preceding 8 pages, since I think the first response to the OP was adequate, so forgive me if im missing something, but for starters,

1) how does the winning percentage of .56 constitute 'bad luck'. That's greater than even, so it is actually good luck if you are starting with a situation where the odds are close to even (and presumably, that situation was chosen for that very reason!)

The winning percentage (0.5562) is being compared to an expected average probability of winning (0.5737) which was computed using the known cards at the time.

2) The results are highly dependent on the whether ALL the cards are known, including those cards discarded by the other players.

This was discussed earlier in the thread. If ALL the cards are known, there is no probability to compute. The outcome is a certainty. The effect of unknown discarded cards is not relevant to computing the probability of a win from the perspective of the players still in the game.

 

[dudalb]

Yes. I think he is going to be a great Quarterback for The Indy Colts.

 

[jt512]

known, there is no probability to compute. The outcome is a certainty. The effect of unknown discarded cards is not relevant to computing the probability of a win from the perspective of the players still in the game.

I think the folded cards are relevant, but I don't know if their effect is big enough to worry about. The computer that calculates the expected odds is almost certainly considering the folded cards to be random, but they aren't. Consider the following two situations in a 7-handed game.

Situation 1: Player 1 is on first base and goes all-in. Player 2 is next to act and calls. Everybody else folds.

Situation 2: Player 1 is on first base and goes all-in. Everybody folds, except Player 2, who is last to act, and calls.

It seems to me that the expected hands that these players folded are not the same. In Situation 1, the players are facing a bet and at least one call, so they will fold stronger hands than in Situation 2, where they are facing just a bet. If so, then it is possible that Player 2, your husband, could still be making bad calls, which could cause his expected results to be less than those calculated by the poker computer. In other words, your procedure would not be completely eliminating skill (or a lack of it).

I'm a statistician, not a poker player, so I could be completely wrong about this.

 

[Beth]

I think the folded cards are relevant, but I don't know if their effect is big enough to worry about. The computer that calculates the expected odds is almost certainly considering the folded cards to be random, but they aren't. Consider the following two situations in a 7-handed game.

It seems to me that the expected hands that these players folded are not the same.

This may be true. It was discussed earlier on. There's no way of including them in the computations. Basically, uncertainty due to that cause is folded into the general background noise. The size of the impact is unlikely to be large. More importantly, it's unlikely to bias the results in one direction versus the other.

 

[stanfr]

The winning percentage (0.5562) is being compared to an expected average probability of winning (0.5737) which was computed using the known cards at the time.

Yes, understood, but the probability of winning you are using may be overestimating the odds, because it does not factor in the exact scenario in the hand. If you use a scenario that is a 'true' even chance, then that .56% is actually good luck, not bad!

This was discussed earlier in the thread. If ALL the cards are known, there is no probability to compute. The outcome is a certainty. The effect of unknown discarded cards is not relevant to computing the probability of a win from the perspective of the players still in the game.

Incorrect. There is no certainty as long as there are cards to be drawn. The effect of unknown cards is not relevant if you are a poker player calculating your odds, but that's not the situation here--you are ex post facto trying to determine whether 'bad luck' existed. To do that, you need ALL of the cards to determine the correct odds--because those odds are partly determined by the play PRIOR to the heads-up situation. You are not factoring this in, which is the flaw in your methodology. So, again, what percentage of hands was your husband playing the overcards vs the pair? That's at least a starting point...

 

[stanfr]

This may be true. It was discussed earlier on. There's no way of including them in the computations. Basically, uncertainty due to that cause is folded into the general background noise. The size of the impact is unlikely to be large. More importantly, it's unlikely to bias the results in one direction versus the other.

We disagree here. I don't think it's 'noise'--it can be highly correlated to your subjects playing style, the types of tables he is playing, etc. You are underestimating this influence. As was mentioned earlier in the thread, a better test would be to look at all the starting hands and compare that with chance (I think you did this with AK, 52 etc--I will take a second look!)

 

[Beth]

We disagree here. I don't think it's 'noise'--it can be highly correlated to your subjects playing style, the types of tables he is playing, etc. You are underestimating this influence.

Perhaps. If you can think of better way to estimate or eliminate it, I wouldn't mind hearing it.

As was mentioned earlier in the thread, a better test would be to look at all the starting hands and compare that with chance (I think you did this with AK, 52 etc--I will take a second look!)

Yes, that is what was done with the AK, Q8, 52.

 

[stanfr]

Perhaps. If you can think of better way to estimate or eliminate it, I wouldn't mind hearing it.


Yes, that is what was done with the AK, Q8, 52.

Ok--I looked at those results--that is much more interesting and it is probably the way to go here. I personally think it would be basically impossible to properly analyze the odds using your heads up situation--there are just too many 'skill' variables that occur prior to the heads up situation that affect the odds (to give an extreme example--suppose you know that all your opponents play every A or K to showdown, and you go all-in with QQ as last-to-act...what are your odds??)
As I said, stats are not my forte, but the sample size seems kind of small here given that only three hands were considered (resulting in double digit deals). The better way to do it would be to consider several sets of starting hands (AK,AQ,AJ vs 25,26,27 for example....but that may be problematic from a practicality standpoint).

 

[jt512]

I'm not sure why you think a Bayes computation is suitable for this. Could you explain?

You have two competing hypotheses: H0, that your husband's "luck" is not bad, and H1, that your husband's luck is bad. p-values tell you the probability of your data (or more extreme data) assuming that H0 is true. However, we would like to quantify the strength of the evidence that the observed p-values provide against H0 vs. H1, at least that's what I would want to do. The relative strength of the evidence for H0 vs H1 is quantified by the Bayes factor, which is what Rouder's on-line calculators calculate.

Berger and Selke (1987) showed that for any reasonable (ie, unimodal symmetric) prior distribution, the Bayes factor in favor of the null hypothesis vs the alternative cannot be greater than about 2.4 for a p-value of .05, or 8.2 for a p-value of .01 (and these are best cases). Thus p-values in this range provide, at best, modest evidence against the null. Other analyses have shown that p-values in this range can actually favor the alternative over the null, so in the worst case, we reject the null hypothesis when the data actually favor it.

What is the subjective prior being used? What does the 'scale r on effect size' represent?

Rouder et al (2009) explains the role of the prior distribution in the computation of the Bayes factor, derives the specific priors used in the on-line calculator, and explains the role of the scaling factor (briefly, it can be used to incorporate prior knowledge of the expected effect size under H1).

Does the calculator you linked to have a method to include results from multiple experiments to adjust the final result to include all known information?

Not exactly. You'd have to consult the Bayesian literature on this. For truly independent experiments, the Bayes factors can be multiplied together. Otherwise, Bayesian meta-analysis methods should be used to combine the evidence.

The session data collection (which started in 2013) is ongoing.

Was this, then, an interim analysis? If so, what prompted you to do it?

I want to test some hypotheses about what he can change and whether or not it will have an impact on the results.

His future luck is independent of his past luck. What he can change is his level of skill.

But theoretically, these results have a low probability of occurring under the null while they are consistent with the alternative hypothesis.

That's correct, but they don't necessarily provide strong evidence against the null hypothesis or strong evidence for the alternative hypothesis. I strongly recommend that you compute Bayes factors.

How would you suggest he improve his data collection to test the hypothesis that his observation is correct, that he actually has results worse than random chance predicts?

I don't know that his data collection is bad in the first place. What I know (at least up to an extremely high probability) is that people don't have a "luck" attribute. It is simply superstition to say, using the present tense, that you "have" bad luck. Of course it's perfectly correct to say that you "had" bad luck. But to project your past bad luck into the future is silly.

As I've stated, I don't think you have compelling evidence that the null hypothesis is false. But if you eventually get p-values that indicate the data are overwhelmingly improbable under the null hypothesis, then this would suggest that there's something wrong with the experiment: that the expected win probabilities are wrong, the data collection unreliable, or something else I haven't thought of. Each of these alternative hypotheses is orders of magnitude more plausible than the existence of a supernatural "luck" phenomenon.

 

[Beth]

The relative strength of the evidence for H0 vs H1 is quantified by the Bayes factor, which is what Rouder's on-line calculators calculate.

Thanks for the summary. I'll look that paper over.

Was this, then, an interim analysis? If so, what prompted you to do it?

Yes. I have an ongoing analysis that I'm simply adding information in as he collects it. I wanted to add something to my blog and my husband's poker friends have expressed interest. Since I had posted this thread to get ideas about what data to collect, etc., I thought I'd post an update for those interested.

His future luck is independent of his past luck. What he can change is his level of skill.

His skill isn't so terrible. In fact, his particular poker group has been running a year long competition and he's currently in the lead. That's probably why they give him such a hard time about it when he complains about his bad luck.

That's correct, but they don't necessarily provide strong evidence against the null hypothesis or strong evidence for the alternative hypothesis. I strongly recommend that you compute Bayes factors.

I'll look into it. Thanks for the suggestion and pointers.

 

[JJM 777]

The data collect and analysis I've done is an attempt to do just that.

If you mean the analysis of your husband playing poker, it is not a meaningful basis for evaluating _luck_, because poker is a _skill game_ (well, there is a theoretical luck element in all games and sports, but the skill element usually matters more than the luck element). Your husband losing more than 50% of his poker games does not necessarily prove anything about luck, it only proves that more than 50% of his opponents are more skilled than he.

Evaluating luck would only be meaningful in circumstances where a skill element of any kind cannot apparently have any effect whatsoever on the lucky or unlucky outcome. For example, choosing the correct lottery numbers for next Saturday.

Besides, looking back at the success rate of your husband is just another example of retrospectively assessed luck. Luck certainly does exist, in a retrospective sense. Ask any lottery jackpot winner.

 

[Pup]

Wanted to say something like this. Then found out that something like this has been said already.

Documentable luck exists in retrospective sense, but we cannot very meaningfully document or evaluate luck in a predictive sense. Unless anyone can give an example of that.

That pretty well sums it up. If, retrospectively, certain people are "unlucky," not all will regress to the mean. Some will continue to be "unlucky" until the day they die, and can say, "Ha! See? I really am unlucky." But we still can't predict who those ones will be. If we could, that would indeed be evidence of some actual force of luck that clings to people.

Add in weird human biases--is an unlucky poker player who keeps finding money on the ground lucky, unlucky, or just uniquely observant/unobservant in different situations?--and people can sense there's such a thing as luck without anything paranormal happening.

 

[Beth]

If you mean the analysis of your husband playing poker, it is not a meaningful basis for evaluating _luck_, because poker is a _skill game_ (well, there is a theoretical luck element in all games and sports, but the skill element usually matters more than the luck element). Your husband losing more than 50% of his poker games does not necessarily prove anything about luck, it only proves that more than 50% of his opponents are more skilled than he.
Evaluating luck would only be meaningful in circumstances where a skill element of any kind cannot apparently have any effect whatsoever on the lucky or unlucky outcome. For example, choosing the correct lottery numbers for next Saturday.

Actually, we've given a great deal of thought to this problem. The statistics we are keeping are predictable by random chance alone so that we can, in fact, define luck as a statistically significant difference from expectation in a consistent direction. The reason I started this thread was to get additional opinions and suggestions on how to collect data separately from the skill portion of the game.

Besides, looking back at the success rate of your husband is just another example of retrospectively assessed luck. Luck certainly does exist, in a retrospective sense. Ask any lottery jackpot winner.

Er, no. That was the point of the 2013 data collection exercise. We set it up in 2012, so it is NOT a retrospective study.

That pretty well sums it up. If, retrospectively, certain people are "unlucky," not all will regress to the mean. Some will continue to be "unlucky" until the day they die, and can say, "Ha! See? I really am unlucky." But we still can't predict who those ones will be. If we could, that would indeed be evidence of some actual force of luck that clings to people.

It's a difficult area to study.
We could, however, test the hypothesis of the future being like the past versus the future being random and unpredictable. That what my dh's data collection is doing.

Add in weird human biases--is an unlucky poker player who keeps finding money on the ground lucky, unlucky, or just uniquely observant/unobservant in different situations?--and people can sense there's such a thing as luck without anything paranormal happening.

It is indeed hard to differentiate between observational bias, one's attitude towards life affecting one's actions, etc. That's why data collection is so important.

On the other hand, what do you tell someone when they collect data on a supposedly random outcome, using a number of different approaches, improving them over time and the results consistently show their outcomes being worse than expected by random chance?

 

[Pup]

On the other hand, what do you tell someone when they collect data on a supposedly random outcome, using a number of different approaches, improving them over time and the results consistently show their outcomes being worse than expected by random chance?

That would be expected in about half the cases, wouldn't it? Let's say we have twenty coin flippers who claim to be unlucky. We'll define heads as lucky. They offer as evidence their last ten coin flips, when they got mostly tails.

What are the odds that any randomly selected member of the twenty will continue to get more unlucky tails in their next flips? I think it would be about one in two, not rare at all.

In other words, a larger sample size is needed. If one claims that Joe Blow is unlucky and correctly predicts his bad luck will continue, there will be about a 1:2 chance of being right. If one claims that these twenty people are unlucky and correctly predicts they'll all do worse than chance, and they do, the improbability of being right is starting to indicate something.

 

[stanfr]

Beth, I have a question: is 'luck' strategic in that it only applies to live poker games as opposed to anything else? In other words, is there any logical reason to suppose one's 'luck' or lack thereof only manifests itself in a live poker setting? If not (and I see no logical reason why it would), then why waste all that time collecting stats in live games? Why not get it over quickly--deal several thousand rounds of 2 cards and compare it to a control. If you can repeat this several hundred times and show a consistent obvious bias (and it wouldn't take long since you don't need a live poker game to do it!) then you've demonstrated something. But of course, there's no reason to think that the two cards you randomly (some decent shuffling!) deal to your husband are going to be any different statistically than if you were to deal two cards repeatedly to a mannequin. How does the person (or thing) you are dealing to somehow affect the outcome? Are they assisted by some divine being that enjoys playing cards? The whole idea is preposterous when put in those terms--yet that's essentially what people who believe in 'luck' are saying, if you ask me.

 

[ufology]

You have two competing hypotheses: H0, that your husband's "luck" is not bad, and H1, that your husband's luck is bad. p-values tell you the probability of your data (or more extreme data) ...

Impressive. Although I might argue that Luck isn't necessarily a supernatural concept. For example, if someone wins a random lottery more often than others, there may be nothing supernatural about it. Nevertheless few would deny that they are lucky. In other words, it can be seen a mere description of circumstances rather than a supernatural event.

 

[JJM 777]

what do you tell someone when (...) the results consistently show their outcomes being worse than expected by random chance?

You should make it clear to yourself that random chance _does not expect_ that one hand-picked individual will get the middle score of the gaussian curve. Not even if you repeat the test a thousand times. Random chance expects that a given individual gets a place _somewhere_ on the gaussian curve. Your husband has found his place on the gaussian curve, slightly on the unluckier side from the middle point. This is one of the billion different expected results of random chance.

Take a lottery jackpot as an example. By random chance, an average person should not win the jackpot. The chances are strongly against you winning. But when someone eventually does win the lottery jackpot, it is not against random chance. Random chance expects that all positions of the gaussian curve will eventually become occupied by someone. It is the expected result of random chance, when someone eventually gets the very unlikely correct numbers.

 

[jt512]

We could, however, test the hypothesis of the future being like the past versus the future being random and unpredictable. That what my dh's data collection is doing.

It's never been clear to me, Beth, whether you understand that the hypothesis that past luck can predict future luck is a paranormal hypothesis, or equivalently, nonsense. Do you agree? And if so, then why are you doing this "study"? The null hypothesis is true by the laws of physics, and so any rejection of the null must be due to random or systematic error.

On the other hand, if you disagree, can you explain why it is not a paranormal/nonsensical hypothesis?

Or, on the third hand, do you agree that your hypothesis is paranormal, but that the paranormal might exist?

 

[stanfr]

Impressive. Although I might argue that Luck isn't necessarily a supernatural concept. For example, if someone wins a random lottery more often than others, there may be nothing supernatural about it. Nevertheless few would deny that they are lucky. In other words, it can be seen a mere description of circumstances rather than a supernatural event.

I agree. For example, if one hundred people play a lottery in which the odds of winning (by picking a series of numbers, say) is one in a million--if someone wins I think few people would disagree that this was a 'lucky' win, even though, statistically speaking, that person's picks were just as likely to come up as any other pick. But what if the same person wins week after week? At some point, I think you have to concede that there is more than statistics playing out...and what is that point? I think that the relevance to this thread is that you had better have a really STRONG bias shown before you even begin to consider that something 'supernatural' is going on.