• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Need Help with Randomizing for Experiment

Jeff Corey said:
I was suggesting number 3 as part of the procedure, maybe I should have been most explicit.
I don't get part about the guesser declaring he will guess in a certain way being any different from deciding he will guess in a certain way and not declaring it or having a bias to guess in a certain way and not realizing it.
Click to expand...

The difference is that if he declares it, then someone knows both pieces of information. Probability is a function of the information available to the one computing the probability, so if someone knows both the experimenter's randomization technique and the guesser's randomization technique, it's possible they will compute probabilities differently than if they didn't know one or the other. In scenario 1 the computed probabilities will be the same; in scenario 3 they will not.

Similarly if the guesser does this experiment (scenario 3) a hundred times and each time picks exactly 10 god-pots, there is very good grounds for re-doing the probabilities given that he will guess this way. To be thorough you might set up a different experiment (scenario 1) to see if he is psychically (or whatever) picking up on the fact that there are exactly 10 god-pots. Then if he keeps guessing 10, you know it's a matter of how he likes to guess, and if he doesn't, something fishy is going on. What's happened? Basically the experimenter has found out empirically that the guesser has a non-independent method of guessing, rather than the guesser straight-out just declaring it. Either way it's time to recalculate the probabilities as soon as you know.
 
Last edited:
I originally made my suggestion about assigning God-pots or not based on a die roll (rather than a draw without replacement) in order to avoid the argument that has occupied the last two pages. However, it has been enlightening to now discover who I can and can not trust when it comes to these issues. ;)

Linda
 
fls said:
I originally made my suggestion about assigning God-pots or not based on a die roll (rather than a draw without replacement) in order to avoid the argument that has occupied the last two pages. However, it has been enlightening to now discover who I can and can not trust when it comes to these issues. ;)

Linda
Click to expand...

Meanie :blush:

Given the discussion so far, then, would a better way to do it be to have a series of as many trial as you want where one of the plants is selected at random, and the guy guesses. The plant is replaced and another randomly selected plant appears (including the 1/20 chance that it's the same plant you just rated).

This then would make it binomial and one could go for as many trials as desired?

As it stands, it's non-binomial and apparently I've confused the definition of independent; but does my sense of it still hold: There's no way the guy can use knowledge that it's 10/10 to obtain any advantage, using the new and improved distribution that GA posted a few posts ago?

But, using the wrong distribution (binomial) to assess his performance in the 10/10 scenario would result in slightly over-estimating his ability relative to the null?
 
Addressing the question in the OP, what is the simplest way to answer the fricking question.
Twenty pots.
Randomly assign some to the prayed water, the rest to equivalent non-prayed water. I can see that non-replacement would handicap the guesser who would assume that there were the same numbers in each condition. When I do the ESP scam in class and ask students to write down a random series of heads and tails, they usually write 10 heads and 10 tails.
So, Linda, how about going back to your plan of tossing for the prayed watered plants, tell the prayer that it's random, and let the chips fall where they may. Then we can do the Sign test and be happy ever after.
 
Why not try to establish a dose-effect relationship? Make 4 groups of 10 plants each:

Group I gets only plain water
Group II gets 10% prayed-for water and 90% plain water
Group III gets 50% prayed-for water and 50% plain water
Group IV gets only prayer-for water.

That way you can demonstrate an effect (or lack thereof) by measuring growth in each group. If it fits a dose-effect curve, your friend is right. If not, it's woo.

This is a method commonly used in pharmaceutical testing. It is designed, among other things, to eliminate statistical flukes and outliers.
 
Too complicated. The question can be answered with two conditions.
 
Last edited:
Amapola said:
I don't see how his changing his mind would affect results but on the other hand some of this math is really over my head... I don't think it would matter, because he would finally have to make his choices (whatever they are) and the results will still (in my view) be pure guess work.
Click to expand...

You are correct. He doesn't have to make a decision about any of the plants until after he has seen all of them and had the opportunity to compare them. Then he should pick the ten that he thinks were most likely to have been given the prayed-for water.

Even if he picks them entirely at random, he'd still have about a 1-in-11 chance of getting seven or more right. So getting seven right wouldn't be especially impressive. But he'd have only about a 1-in-87 chance of getting eight or more right. Does he think he can get at least eight right?

I'm curious about his reasons for thinking that this will work. Has he actually seen plants that seemed to grow unusually well after being prayed for? Or is he relying on web pages like the one you linked to? (I just read that, by the way. Oh dear...)
 
We need to check with the guy to determine a couple of things: like, how will we know the prayed-for water is working, how long to run the experiment and how well he will be able to tell which pots are which. When we do I'll ask him why he believes this specifically, but as far as we can make out, he is relying on that web page plus his belief in god and prayer.

I really doubt he has grown some plants himself and tried this. He is more of an "idea" kind of guy. I tend to think that if he had actually tried growing some plants, watering some with prayed-for water and some with normal water and then seen the results, he would not now be insisting that we do it to "prove" to us that prayer works.

69Dodge, do you concur with Jeff Corey that I should not pick out 10 randomly, but randomize by coin flipping and let the group be as large or small as the coin flip dictates?
 
Amapola said:
I really doubt he has grown some plants himself and tried this. He is more of an "idea" kind of guy. I tend to think that if he had actually tried growing some plants, watering some with prayed-for water and some with normal water and then seen the results, he would not now be insisting that we do it to "prove" to us that prayer works.
Click to expand...

I see.

To be fair to him, I'm pretty sure about what the outcome will be, too, and I haven't tried growing any plants either. So... :D

Amapola said:
69Dodge, do you concur with Jeff Corey that I should not pick out 10 randomly, but randomize by coin flipping and let the group be as large or small as the coin flip dictates?
Click to expand...

Either way is really ok, if the guy is amenable. The probabilities will be somewhat different---that's all. bpesta22 gave them in post #7. You'd need to record all twenty guesses.

It seems like it would be easier for the guy simply to pick the ten best-looking plants, rather than to have to decide, for each plant separately, whether it looks like it was prayed for. I mean, I don't imagine he thinks that prayer makes a plant look qualitatively different, so that it could be recognized in isolation as having been prayed for. Presumably, it would just look better in some way, as compared to plants that haven't been prayed for.

So he might prefer to know that they're split exactly ten-and-ten, and there's no reason not to do it this way. But the other way is perfectly ok too, if he agrees.
 
bpesta22 said:
Meanie :blush:
Click to expand...

You've stood up to much worse. :)

Given the discussion so far, then, would a better way to do it be to have a series of as many trial as you want where one of the plants is selected at random, and the guy guesses. The plant is replaced and another randomly selected plant appears (including the 1/20 chance that it's the same plant you just rated).

This then would make it binomial and one could go for as many trials as desired?
Click to expand...

As Jeff pointed out, at this point we're making it too complicated and haven't answered poor Amapola's question. I officially retract any suggestions I made earlier - I didn't sufficiently think them through and didn't take into consideration the actual situation.

It seems reasonable for Amapola to perform the experiment and then have this guy come and pick out the God-pots. And realistically, he will do so by considering all the plants as a group and picking out the best. He is going to expect 10 in each group and that should be the set-up (no need to give him a reason to complain about unfair conditions) - adequate ways to do this have already been described. Instead of bringing him one plant at a time, he should simply to allowed to look at the group and pick out the ones in the God-pots (one at a time sampling is unrealistic and confusing). At least 8 have to be correct in order for it to be unexpected.

As it stands, it's non-binomial and apparently I've confused the definition of independent; but does my sense of it still hold: There's no way the guy can use knowledge that it's 10/10 to obtain any advantage, using the new and improved distribution that GA posted a few posts ago?

But, using the wrong distribution (binomial) to assess his performance in the 10/10 scenario would result in slightly over-estimating his ability relative to the null?
Click to expand...

Now that I think about this some more, I realize that I'm not sure just what claims were being made in regards to the effect of knowledge. I can see that knowledge would influence the size of the sample (if he knows that 9 pots are God-pots, then his sample will be 9 not 10) which would influence the probability calculations. I need to re-read what y'all said again and get back to you.

Linda
 
69dodge said:
It seems like it would be easier for the guy simply to pick the ten best-looking plants, rather than to have to decide, for each plant separately, whether it looks like it was prayed for. I mean, I don't imagine he thinks that prayer makes a plant look qualitatively different, so that it could be recognized in isolation as having been prayed for. Presumably, it would just look better in some way, as compared to plants that haven't been prayed for.

So he might prefer to know that they're split exactly ten-and-ten, and there's no reason not to do it this way. But the other way is perfectly ok too, if he agrees.
Click to expand...

Whoa! Are we actually reaching consensus on this? :)

Linda
 
fls said:
Whoa! Are we actually reaching consensus on this? :)

Linda
Click to expand...

And it only took 3 pages! :D

I'm quite gratified, thanks to everyone who has participated. I thought I might get at most 3 replies. I ended up learning quite a lot which is great, I love to learn things, and maybe someone else reading this can learn too.

It demonstrated to me that most people don't sit and think of all the things that must be done to make an experiment like this fair and useful. They don't calculate what must be the correct number of guesses to make it significant. Our friend will probably be really impressed with himself if he gets 6 correct, and it will be our sad duty to point out that that is not all that exciting.

I'm going to take pictures as I go along in the experiment and when it is finally done I will write it up and report my findings. In the meantime it is great to have this thread here for a reference on the math.
 
What are you going to do with the beans when the experiment is over? I hope you don't just sacrifice them-- I'll call PETB. They might even raid your lab and liberate the beans.
 
This exercise trivializes the reason for prayer.
Prayer is for the benefit of someone who can be saved/cured, innit?
If the most sincere prayers ever fail to get answered, when there's human life involved, why would what's-His-name pay attention to the health of some vegetation?
What's-his-name is too busy counting the sparrows that fall, anyway.
 
I Ratant said:
This exercise trivializes the reason for prayer.
Prayer is for the benefit of someone who can be saved/cured, innit?
If the most sincere prayers ever fail to get answered, when there's human life involved, why would what's-His-name pay attention to the health of some vegetation?
What's-his-name is too busy counting the sparrows that fall, anyway.
Click to expand...

Hey, it's not your place to rationalize failure - that's the job of the claimant for after the test.
 
Maybe I missed something, but it seems to me this is a ten coin-flip problem. Religious heads and atheist tails with the friend calling heads ten times in succession. Here are the probabilities for each outcome.
Code: 
correct
 0            0.09%
 1            0.97%
 2            4.39%
 3            11.71%
 4            20.51%
 5            24.60%
 6            20.51%
 7            11.71%
 8            4.39%
 9            0.97%
 10           0.09%

ETA: Providing the results of individual choices are not revealed until all choices have been made.
 
Last edited:
BobK said:
Maybe I missed something, but it seems to me this is a ten coin-flip problem. Religious heads and atheist tails with the friend calling heads ten times in succession. Here are the probabilities for each outcome.
Code: 
correct
 0            0.09%
 1            0.97%
 2            4.39%
 3            11.71%
 4            20.51%
 5            24.60%
 6            20.51%
 7            11.71%
 8            4.39%
 9            0.97%
 10           0.09%

ETA: Providing the results of individual choices are not revealed until all choices have been made.
Click to expand...
There are 20 plants and 20 guesses, not 10.
 
Assuming there are 10 God-pots and 10 not-God-pots, and the guesser knows this, it is only worth while making 10 guesses. E.g. the guesser could be asked to sort the plants into God-pot and not-God-pot groups.

Under this scenario the probability of getting all 10 guesses (either picking out God-pots or not-God-pots) correct is:

(10/20)*(9/19)*(8/18)*...*(1/11) = 0.00000541.

The calculation of the probabilities for the other combinations is left as an exercise for the reader.:)
 
Last edited:
You must log in or register to reply here.

Back
Top Bottom