So you claimed in essence, "I am an authority and anyone who is will agree with me." I respond by saying, "So and so is an authority and disagrees with you." This is an ad hominem argument on both of our sides.
You have it backwards. You stated that authorities agree with you, and
then I disagreed with you,
No, I merely didn't trace the conversation back that far. You didn't just disagree, you disagreed, and claimed to be an authority. I responded by pointing out other authorities.
Which you'll note I have not been conducting through "argument from authority", but rather by presenting detailed examples and calculations.
You most certainly have been conducting an argument from authority. The authority is what you claim Bayes' Theorem says.
See
http://www.internationalskeptics.com/forums/showthread.php?postid=2239982#post2239982 and you will see a worked out example where I take an actual set of prior expectations and work out how they are modified in light of the experiment. You will note that there is no room in that calculation for the difference in experimental design to be a factor at all. You'll also note that (as I said), I presented a detailed example and calculation.
Let's back this up one step farther. I claim that Bayes' Theorem says that P(X given Y) is P(X and Y)/P(Y). (Which theorem says that we can use the latter formula, so in some posts I have referred to a theorem and in others to a formula. I'm talking about the same thing.) This claim is readily verified and you haven't disputed it.
I further claim that the correct way to apply Bayes' Theorem after observing either experiment A or B is with calculations similar to what I did in the post linked above. You have not commented on whether this method is correct. If you think it is incorrect then I would request that you show the calculation method that you think
is correct and explain why you think that is the right calculation method.
Note that this calculation only involves the odds the experimenter gives to various possible theories, and the odds of the observed outcomes under those theories. In particular there is absolutely no way for the differences in experimental design between A and B to enter the calculation, and therefore those differences cannot affect the conclusions drawn.
You can continue to claim that I am presenting an argument from authority here, but if you do you'll look silly. I am starting with a commonly accepted and easily verified fact (the statement of Bayes' Theorem), applying it to a concrete situation, and pointing out fairly obvious features of the calculation. If you believe that any step in this procedure is wrong, you are free to point it out.
Not only did they not, but it was one of them who first lead me through that calculations.
I don't believe you. How did it come up?
Are you always this much of a jerk?
I was a graduate student in mathematics at Dartmouth College from 1992 to 1997. Dr. Laurie Snell was a professor there. He has a habit of engaging people in conversations about interesting problems and odd facts. He brought this example up to me in one of those conversations.
Now since you think I'm a liar, you probably don't believe me. So I'll give some evidence that key people were in the right place at the right time.
That Laurie Snell is a professor at Dartmouth is easily verified - google his name and follow the top link. That he likes to pose odd and interesting problems to people is harder to verify, but google some of the material in his Chance newsletters and you should get an impression of the person that is consistent with his doing that. That there was probably a graduate student there at that time with my name is easily verified. I was very active on usenet, just look at the posting history of
benjamin.j.tilly@dartmouth.edu in sci.math. That this graduate student was me is somewhat harder to verify. However if you wish to look at my posts you'll find that I use the same name, have a similar set of interests, and a similar writing style as that graduate student. Plus I claim that I am he. A reasonable person should quickly be convinced that I probably am telling the truth about being that person.
That the conversation actually happened I cannot demonstrate. If you want to continue to be a jerk and claim that I am lying when I am not, you are free to do so. However continuing to do so at this point is more likely to cause anyone reading this to think badly of you than me.
I have no idea what your bona fides are to back up your self-identification as a statistician, but if you claim that anyone who disagrees is not a statistician, then you've made a claim that is very much on the outrageous side.
It's a rather basic principle of statistics. Without it, you're not doing statistics.
Who is arguing from authority now?
Not me.
I had to go back and add more context to this particular quote. I'll let people draw their own conclusions here.
Incidentally I still have no idea what your bona fides are to back up your self-identification as a statistician. But given that I've done a pretty good job of identifying who I am, I would appreciate it if you tried to do the same.
Would you mind explaining why it is a basic principle of statistics?
Because probabilities apply to random varaibles, not data that've already been collected.
Due to the way that the quoting in these forums lose context, that is a statement that just sits there in isolation. Nobody knows what you are talking about unless they wade back to the beginning of the thread.
However if anyone cared to, what they would find is that your statement is a non-sequitor. I applied probability theory to random variables that were described by the experimental design. After an observation was made, those a priori probabilities (note, probabilities that were worked out about random variables and
not about observed data) then feed into statistical methods and result in statistical conclusions.
If this kind of reasoning is always invalid, then we would find ourselves unable to use standard statistical methods to analyze the outcome of
any concrete experiment. However we can and do do that. So my overall form of reasoning is
not forbidden by a basic principle of statistics.
Whether my actual reasoning is forbidden by statistics is, of course, another question. You've asserted many times that it is, but you haven't yet bothered to support that assertion.
My claims are a matter of easily established fact.
Argument by assertion. You seem to be doing that a lot.
I do get tired of repeatedly demonstrating the same things over and over again and resort to assertion. I admit it. However all of the things that I've asserted I have elsewhere backed up with reasoning. Reasoning that you have yet to acknowledge, let alone address.
If you wish to convince me that I am wrong, all that you need to do is produce a set of prior beliefs which would lead to a different set of posterior beliefs after observing case A and B. I am quite confident that you will fail.
You are the one with the burden of proof.
If you read my posts, this one in particular, you should find that I have fully discharged that burden. At some point it becomes reasonable to request an effort from you. I have specifically requested two efforts that are of key importance. They are:
1. Describe how you would analyze the outcomes of experiments A and B using what you consider to be standard statistical methods.
2. Provide a concrete set of prior expectations which would result in different conclusions after observing A and B.
You'll find my answer to #1 at
http://www.internationalskeptics.com/forums/showthread.php?postid=2239245#post2239245. You'll find my understanding of the correct method of calculation to use for #2 at
http://www.internationalskeptics.com/forums/showthread.php?postid=2239982#post2239982.
Furthermore, you keep equivocating between Bayes' Theorem and statements that are beyond the theorems. You haven't addressed any my points:
I have not? Then I'll address them right now, and hopefully you'll bother addressing mine in turn.
Theorems don't speak of "should".
Short answer: When I say, "Bayes Theorem says we should..." what I actually mean is close to, "Bayes Theorem implies a result. Given that result I think we should..." But let me give a longer answer.
Bayes' Theorem indeed does not speak of should. It speaks of how to calculate the probability of one event given that another has happened. Which means that if we start with a person who has a set of expectations of the probabilities of various outcomes, we can say how that person's expectations will differ after an observation if that person's beliefs update according to the rules of probability theory. (This insight was made, incidentally, in Thomas Bayes original paper where he introduced the theorem.)
However Bayes' Theorem does
not address whether people actually
will use this procedure to update their beliefs. And it cannot answer normative questions like whether they
should. Answers to those questions involve an extra layer of non-mathematical interpretation.
However it is my belief that a person
should update their beliefs according to the rules of probability if they are in a position to do so. Admittedly they
might not do this. So I don't say they
will do this. Only that they
should.
Theorems only speak of mathematical concepts, therefore any declaration that Bayes' Theorem makes a statement about a nonmathematical concept is clearly wrong.
See my explanation above of how a purely mathematical concept leads me to a normative conclusion.
You are claiming that hypothesis testing is unreasonable because it's not allowed by Bayes' Theorem, and you say that it's not allowed by Bayes' Theorem because it's unreasonable.
This is a blatantly false statement of my claims.
I claim that hypothesis testing is unreasonable because it distinguishes cases that are indistinguishable when drawing conclusions according to Bayes' Theorem. Since I believe that Bayes' Theorem is the gold standard for how to draw inferences (see explanation above for why), the conclusion follows if my claim about Bayes' Theorem is correct. See much farther above for an explanation of why my claim about Bayes' Theorem is correct. If that explanation is unclear, see (again)
http://www.internationalskeptics.com/forums/showthread.php?postid=2239982#post2239982 for a worked out specific example using Bayes' Theorem in which you can see that the mathematics leaves no room for the difference in experimental design to make a difference.
Now, you said "According to Bayes' Theorem, under no prior set of beliefs should the difference in design of the experiments make any difference in your conclusions."
Then, you said "If you wish to convince me that I am wrong, all that you need to do is produce a set of prior beliefs which would lead to a different set of posterior beliefs after observing case A and B."
Well, that's just silly. To show that your statement is wrong, I just have to show that your statement is wrong, not show that some completely diffferent statement is wrong. Now, hypothesis testing is a case where design of experiments can make a difference in the conclusion. Therefore, you are wrong. Or else Bayes' Theorem is.
Where do I begin sorting this one out?
First of all, you're wrong to think that I was asking you to show that a completely different statement was wrong. The correct way to prove that a statement of the form, "There is no X such that Y" is wrong is to produce an X under which Y. Therefore the second quoted statement is
exactly asking you to prove that the first quoted statement is wrong.
Now let's proceed to the rest of your statement. I spent a long time trying to figure out how you thought it made sense. And then it hit me where the miscommunication was.
When I said, "A prior set of beliefs", I meant, "The experimenter has a priori belief about the relative likelyhoods of different sets of relative odds of the couple having sons and daughters." In which case hypothesis testing is an invalid procedure for describing how this person's believes would be modified by observing the experiment (if they follow the laws of probability in updating their beliefs). The correct procedure to use is Bayes' Theorem. Therefore hypothesis testing is absolutely irrelevant to what conclusions will be drawn.
However it appears that you've interpreted the phrase to include belief systems like, "We have an experimenter who believes that hypothesis testing at a 95% confidence level is the right way to determine whether this couple has even odds of producing sons or daughters." Never mind the (in)correctness of the experimenter's beliefs, but this experimenter will indeed draw different conclusions from the two experiments. And Bayes' Theorem is inapplicable in this case because the experimenter's belief system is not in a form where Bayes' Theorem can be applied.
This is a reasonable interpretation. It is obviously not one that I meant, not one that I thought of, and not one that my wording had ruled out. So you can produce a belief system assumes the conclusion you want, and therefore behaves in the way you want. I was unclear. My bad. (Note that I may be overly generous to you here. I don't know that you were thinking this way. I just searched for a way in which you might not be saying something stupid, and, to my chagrin, found one.)
But now that you've been informed what kind of belief systems I was referring to, you should be able to verify that what I've been saying about Bayes' Theorem indeed applies for those belief systems. (See details of my reasoning earlier in this post and in the worked out example in a previous post.)
It provides no way for the difference in experimental design to matter.
Do you really not see how fallacious that is? You are denying the antecedent.
I really don't see what you are talking about. Denying the antecedent refers an argument where you assume "If p then q", demonstrate not p and conclude not q.
That's not what I'm doing.
I'm pointing out that for any set of prior expectations about the probabilities of the couple having sons versus daughters, Bayes' Theorem says that if we follow the rules of probability, the posterior conclusions will not differ after experiment A and B.
I am a reasonable person, and I draw a distinction. Therefore, you are wrong.
You left yourself open to snide remarks there, but I'll refrain. (Not that you refrained from implying that I am a liar...)
By that I mean give me a detailed set of prior beliefs which, when modified according to Bayes' Theorem in the light of these two experiments, leads to different conclusions.
Then you aren't speaking English. You see, among English speakers, when someone says "Give me an example of X", that means "Give me an example of X", not "Give me an example of Y".
Here's your argument: "I only accept arguments based on Bayes' Theorem. There is no way to establish a difference based on Bayes' Theorem. Therefore, I dismiss that there could possibly any difference."
You have now been informed that when I said, "a detailed set of prior beliefs" I meant, "A set of expectations about what the possible ratios of sons to daughters are." That is, I meant what Bayesians like to call "a prior".
When you re-read my words knowing what I meant by that phrase, does it turn into English?
I strongly suspect that if you try and fail to provide me with the requested example, the connection will become much clearer to you.
I strongly suspect that you have a huge mental blind spot. "If you would only think about it, you'd agree with me".
I strongly suspect that you are an *******. But that is neither here nor there.
I clearly had a blind spot. I was saying, "a detailed belief system" when I meant a belief system that was detailed in a particular way. And I had not realized that you were interpreting my words differently than what I was saying.
Looking back over the conversation I suspect that you have a similar blind spot. You think that I'm a moron who is trivially wrong and therefore never tried to figure out what I might be trying to say.
Because it shows that any set of prior expectations will lead to the exact same posterior beliefs after observing either experiment A or B.
Therefore the details of what might have happened but didn't should be irrelevant to the inferences we draw. All that should matter is that there were 8 children and 7 of them were boys.
The second doesn't follow from the first, and neither of them answer my question.
The second follows from the first if you are only considering drawing inferences from within belief systems described by prior sets of expectations of the form that I was thinking of. And the quotes were in response to different question, so I am not sure what question you think has not been answered.
Just because you don't understand the reason doesn't mean they are absurd.
Is there any reason for this not-so-subtle putdown?
I don't think that it's reasonable to call it a "putdown" to point out the flaw in your logic.
Ah, I see. If you think an insult worked the first time, then you hit that point again.
Of course you didn't actually point out a flaw in my logic. But you did imply that I did not understand something that I actually do. If you can't see how that is a putdown, then you are far less intelligent than I think you are.
Particularly when I've fairly conclusively demonstrated that I understand why hypothesis testing leads to a distinction being drawn in this case?
You're clearly begging the question, as this statement assumes that the understanding of hypothesis that you have demonstrated is, in fact, the correct understanding.
I've asked you several times to show us what you think the correct understanding is. You haven't done so yet.
But it is an increase in the strength of our conclusion in this particular case. Which is exactly what I was saying.
My point stands.
Then your point depends on cherry-picking.
You've gone in this case from denying my point to insulting it. That's progress of a sort.
I look forward to your elucidation of this point. Preferably with a calculated, worked out, example.
Suppose that we have three dice, a red die, a blue die, and a green die. The red die has the numbers 1-6. The blue die has the even numbers, each twice, and the green has the odd numbers, each twice.
Let's say that we start out with the belief that each die is equally likely. We roll a die, and note that it's a three, but don't note the color.
Red die: one side has a three, each side has probability 1/6, likelihood=1/6
Blue die: no threes, likelihood=0
Green die: two sides have threes, each side has probabilitiy 1/6, likelihood=1/3
So now we have confidences of: red, 1/3; blue, zero; green, 2/3.
But our calculations regarding the green die included the probability of
both threes, even though we only saw one. In fact, if it is indeed the green die, then, when we considered the probability of the red three, we were also including the probability of something that never happened. No matter what, two thirds of the possibilities that we considered in our analysis are possibilities that never happened.
Furthermore, the reason that we think that the red three has a probability of 1/6 is because there are six sides, and we assume that they are equally likely. But we didn't see any of those other sides! Why are we including those other possibilities that
didn't happen when calculating the probability of what
did happen?
All of the possibilities that are included are ones that
might have actually happened. We
don't include the ones that we know
didn't happen. Only the ones that possibly did.
So while it is true that only one of the 3 threes came up and the others didn't, we have to include in our calculation the possibility of all of them because we have no idea which one happened. If we knew which three came up we could change our calculation, but we don't and so we can't.
However things that we know didn't happen do not enter our calculation in any way, shape or form. For instance no considerations are made about the sides with, say, 2s on them. If you changed the experiment to be run with 8-sided dice rather than 6-sided dice, the reasoning and conclusions from the observations would be unchanged. Similarly if you had the dice numbered from 0 to 5 instead of 1 to 6.
By contrast in the experiments that I discussed, hypothesis testing distinguishes the experiments because of an enumeration of possibilities that we know (after the experiment ran) absolutely could not have happened.
Regards,
Ben
