JREF Challenge Statistics

[CFLarsen]

That's an interesting opinion.

It's not an opinion, Justin. Your posting history here, and elsewhere, shows only one thing: You want to bring down skeptics and JREF in particular.

Says you. I've never said I would or wouldn't.

Will you, yes or no?

I have not "demanded" anything.

Bull.

You should apply for the challenge and see if you can will a million. Really.

Let's see what happens.

You just have some odd need to say I've "demanded" things, that I won't pay for things, and other fairy tales. You're very threatened to do that, IMO..

If that suits your little fantasy world.

Why do you hate skeptics so much, Justin? Don't say that you don't, because your posting history is evidence that you do.

 

[T'ai Chi]

I don't care a half cup of warm spit what you think your textbooks say, because it's obvious that you don't understand them, so it would not be a useful way to spend my time tracking down citations.

Here's an exact quote from one book I got:

In good statistical practice, alpha is specified in advance before any samples are drawn so that results will not influence the choice for the level of significance.
Understanding Basic Statistics, Brase and Brase, 1997

They don't talk about calculating alpha as you're claiming.

Looks like you're wrong here.

 

[T'ai Chi]

Your posting history here, and elsewhere, shows only one thing: You want to bring down skeptics and JREF in particular.

Your opinion is interesting. Wrong, but interesting.

Will you, yes or no?

It all depends on specifics.

Bull.

We disagree.

Why do you hate skeptics so much, Justin? Don't say that you don't, because your posting history is evidence that you do.

You have a very active imagination. Gotta wonder about those who see hate and attacks in everything, and why they get so threatened.

 

[rwguinn]

I'm not sure where you are getting lost here dr. I'm not interested in your math. I've been asking to see even one textbook that says one calculates alpha.

A numerical value that cannot be calculated. A mathematical result thatmcannot be determined mathematically. Amazing.
If it cannot be calculated, it isn't real anyway.
So why pick nits?

 

[drkitten]

Here's an exact quote from one book I got:

Like I said, you don't understand the textbook.

Here's you're quote again, suitably emphasized.

In good statistical practice, alpha is specified in advance before any samples are drawn so that results will not influence the choice for the level of significance.

Obviously, this raises the possibility that alpha is not specified in advance (which in this description -- and both I and Harlequin agree -- is poor practice, but poor practice is not the same as impossibility.) In the event that alpha is not specified in advance, then, as Harlequin put it,

If you have already performed an experiment without bothering to think about this ahead of time, you can calculate what exactly the alpha was for that experiment.

... but this is generally poor practice, because it permits the obtained "results [to] influence the choice of the level of significance," or in other words is an open invitation to cherry-picking.

The most common time that you will see this mistake made (other than in the hands of exceedingly incompetent researchers) is in so-called "post hoc" analysis where the alpha value is adjusted in secondary (unplanned) analyses subsequent to a main analysis, usually an ANOVA.

So even your own preferred citation implies that you can calculate alpha, but that it's preferable not to do so.

 

[ChristineR]

At the risk of making things worse, I believe that the way it works is the researcher chooses an alpha value to fit the circumstances, then calculates p-values for the various outcomes. Then one can match possible outcomes to the chosen alpha values. You could describe what you've done as "calculating the alpha values" that go with the various outcomes.

Conversely, if you have a very restricted experiment (i.e., Natasha Demikina) you might first calculate all the p-values (for seven possible outcomes), and then pick a de facto range of alpha.

 

[CFLarsen]

It all depends on specifics.

What specifics?

You have a very active imagination. Gotta wonder about those who see hate and attacks in everything, and why they get so threatened.

Just answer the question: Why do you hate skeptics so much?

 

[T'ai Chi]

Like I said, you don't understand the textbook.

So you say.

Say an average is theoretically distributed normally with a mean of 5 and a standard deviation of 1.

We take 20 samples and observe a mean of 4.7.

Test the hypothesis that mu = 5.

This book just sets alpha = .05. Then it goes on to calculate a test statistic, and a p-value, then compares the p-value to alpha, and ends up not rejecting the null hypothesis that mu = 5.

But you're saying you can calculate alpha. Can you do it here please to shut me up?

 

[drkitten]

At the risk of making things worse, I believe that the way it works is the researcher chooses an alpha value to fit the circumstances, then calculates p-values for the various outcomes.

That's standard practice and the preferred method of doing it, certainly. But in many cases -- usually involving either post hoc analysis or in cleaning up the mess left by a semi-competent researcher -- it's necessary to recreate alpha values from the experimental protocol in order to determine what the implicit rejection threshold would have been.

Conversely, if you have a very restricted experiment (i.e., Natasha Demikina) you might first calculate all the p-values (for seven possible outcomes), and then pick a de facto range of alpha.

It doesn't even need to be that restricted an experiment; this kind of argument comes up all the time in various reports of paranormal coincidences. Calculating the probabiliy of a "Type I error" (which is essentially, calculating an alpha value) is a routine assignment for first term statistics students.

 

[T'ai Chi]

What specifics?

It depends on what skeptical organizations do if anything.

Just answer the question: Why do you hate skeptics so much?

I answered your "question" with about as much respect as it warranted.. You may not have liked my answer though.

 

[drkitten]

Say an average is theoretically distributed normally with a mean of 5 and a standard deviation of 1.

We take 20 samples and observe a mean of 4.7.

Insufficient information given. What's the distribution of the samples? Was the null hypothesis accepted or rejected?

But you're saying you can calculate alpha. Can you do it here please to shut me up?

With sufficient information given, I can.

As a matter of fact, it's a routine problem. (See this test, problem 5 for an example, although they phrase it as "calculate the probability of a type I error.")

Similarly, this page here, problem 15 asks students to "calculate the probability that you could have gotten results as extreme as yours," again asking for them to calculate the alpha value of the described experiment.

As I said, I'm really sorry you don't understand your own textbooks.

 

[CFLarsen]

It depends on what skeptical organizations do if anything.

But in this case, you already know what JREF does. Will you support it financially, yes or no?

I answered your "question" with about as much respect as it warranted.. You may not have liked my answer though.

Whether I like it or not has nothing to do with it. What I am wondering is if you think that you are making progress here.

Do you honestly think that you are successful in getting your arguments through? This constant refusal to back up your own claims with evidence, and refusal to clarify points in your argumentation - what possible good can that do to whatever point you are trying to make?

Does your refusal to answer questions about your claims mean that you insist that we merely take your word for it?

Or is it because you think we are too stupid to understand your explanations?

Or perhaps you have no answers at all? You are nothing but a troll, eager for whatever attention you can get, be it ever so vapid as all attention is on Internet forums?

I really can't think of any other alternatives. If you feel like clarifying, it would be most helpful.

To your own argument, that is. You are not scoring any points by not clarifying. I hope you can see that.

 

[strathmeyer]

...although they phrase it as "calculate the probability of a type I error.")

Either provide an example, or admit that you were wrong. When you dance around and pretend to not be an idiot, everyone can still tell that you are an idiot.

 

[T'ai Chi]

As a matter of fact, it's a routine problem. (See this test, problem 5 for an example, although they phrase it as "calculate the probability of a type I error.")

Dr, alpha is still set in that problem. You've just been asked to solve backwards to find what it was set at.

As I said, I'm really sorry you don't understand your own textbooks.

Yes, you've mentioned that. It still reeks of ad hom and adds nothing to your argument.

Here's another book outlining the steps of testing hypotheses

1. state Ho and Ha just as in a test of significance
2. Think of the problem as a decision problem, so that the probabilities of Type I and Type II errors are relevant
3. Because of Step 1, Type I errors are more serious. So choose an alpha (significance level) and consider only tests with probability of Type I error no greater than alpha.
4. etc.
Introduction to the Practice of Statistics, Moore and McCabe, 2003

3. you choose an alpha, you don't calculate an alpha.

 

[T'ai Chi]

But in this case, you already know what JREF does. Will you support it financially, yes or no?

It all depends.

Do you honestly think that you are successful in getting your arguments through? This constant refusal to back up your own claims with evidence, and refusal to clarify points in your argumentation - what possible good can that do to whatever point you are trying to make?

Does your refusal to answer questions about your claims mean that you insist that we merely take your word for it?

Or is it because you think we are too stupid to understand your explanations?

Or perhaps you have no answers at all? You are nothing but a troll, eager for whatever attention you can get, be it ever so vapid as all attention is on Internet forums?

I'm not interested in your personal beefs. It is OK, I forgive you.

 

[CFLarsen]

It all depends.

You know what JREF does. Will you support them financially, yes or no?

I'm not interested in your personal beefs. It is OK, I forgive you.

I think it is a bit disturbing that you have taken on this air of supreme superiority. You do actually believe that you are above everyone else, even to the point where you can regally - or even divinely - "forgive" people for insisting that you back up your claims with evidence.

 

[petre]

Dr, alpha is still set in that problem. You've just been asked to solve backwards to find what it was set at.



Yes, you've mentioned that. It still reeks of ad hom and adds nothing to your argument.

Here's another book outlining the steps of testing hypotheses



3. you choose an alpha, you don't calculate an alpha.

You cannot calculate the value of a variable either, you choose the value of a variable.

 

[T'ai Chi]

You know what JREF does. Will you support them financially, yes or no?

It all depends.

I think it is a bit disturbing that you have taken on this air of supreme superiority. You do actually believe that you are above everyone else, even to the point where you can regally - or even divinely - "forgive" people for insisting that you back up your claims with evidence.

I'm sorry you feel (mistakengly) that way.

 

[Mercutio]

Wow. Tai, for someone who claims to love stats, you really do not understand them.

Do you have a copy of Dunn (2001)? He takes the time that most other stats authors do not, to explain how alpha is determined from the relative costs of type I and II errors. Unlike most authors, he takes pains to show that the conventions of .05, .01, or .001 are arbitrary (but traditional, and avoided "because authors would not want to appear capricious rather than careful.")

Glass & Hopkins (1984 is the version I have), treats alpha as the result of a bayesian process, as does Rosenberg (1990).

 

[T'ai Chi]

Wow. Tai, for someone who claims to love stats, you really do not understand them.

I understand the quotes from books I've presented perfectly well. If I don't, you're free to demonstrate where I don't.

Do you have a copy of Dunn (2001)?

No I don't. I'll check it out though. Thanks.

, to explain how alpha is determined from the relative costs of type I and II errors. Unlike most authors, he takes pains to show that the conventions of .05, .01, or .001 are arbitrary (but traditional, and avoided "because authors would not want to appear capricious rather than careful.")

I'ev seen discussions about the costs of making type 1 and type 2 errors before. But does that book have a formula that one can plug in cost(type I) and cost(type II), and possibly other things, and out comes an alpha?