Ball-drawing experiments-Discussion with Dr. Ertel

[fls]

From Damien Broderick

As far as logistics go, Randi works with international groups so that the testing can be done locally. The difficulty may lie in the availability of
the psi-stars.

Linda

I don't wish to go on about this, but that would surely be one
factor. Another is that any anomaly with the small effect size that
psi shows under strict scrutiny requires (as you know, from studies
of aspirin etc) a *huge* number of trials. My sense is that the Randi
tests, when they actually happen, are aimed at idiots and frauds who
make preposterous claims for their effectiveness. I once read a BOTE
calc by Dean Radin where he estimated that the costs involved in
satisfying the Randi criteria (which are on the order of p =
0.000001) would require advance funding approximating the million
dollars prize--assuming it actually exists and would be paid without
equivocation.

Damien

 

[fls]

From Damien Broderick:

I don't wish to go on about this, but that would surely be one
factor. Another is that any anomaly with the small effect size that
psi shows under strict scrutiny requires (as you know, from studies
of aspirin etc) a *huge* number of trials. My sense is that the Randi
tests, when they actually happen, are aimed at idiots and frauds who
make preposterous claims for their effectiveness. I once read a BOTE
calc by Dean Radin where he estimated that the costs involved in
satisfying the Randi criteria (which are on the order of p =
0.000001) would require advance funding approximating the million
dollars prize--assuming it actually exists and would be paid without
equivocation.

Damien

First of all, Radin is talking about using the ganzfeld, which is quite
different from using a psi-star, and probably isn't really appropriate for
the Million Dollar Challenge. Second of all, a BOTE calc shows that the
number of ganzfeld trials needed to satisfy Randi's criteria is more like
37, which is very different from the 1000's of trials that Radin came up
with. But Radin did this by using grossly inappropriate parameters, such as
asking for odds against chance of 100,000,000 to one (Randi doesn't specify
the odds, but previous tests have required odds on the order of 100 to 1000
to one) and choosing a power level of 99%.

More about this here:
http://www.internationalskeptics.com/forums/showthread.php?postid=4420509#post4420509

From the description Suitbert has given of his psi-stars, it should be easy
to accumulate enough trials to surpass Randi's standards within an hour or
two.

Linda

 

[steenkh]

I wasn't sure whether this would work, or whether it would even be of interest, so I appreciate hearing from you and Moochie. I will keep going with this as long as it seems to be working.

Please record a "me too"!

 

[Ivor the Engineer]

Please record a "me too"!

And me.

 

[fls]

From Damien Broderick:

a BOTE calc shows that the
number of ganzfeld trials needed to satisfy Randi's criteria is more like
37, which is very different from the 1000's of trials that Radin came up
with. But Radin did this by using grossly inappropriate parameters, such as asking for odds against chance of 100,000,000 to one (Randi doesn't specify the odds, but previous tests have required odds on the order of 100 to 1000 to one)

I gather that at least two independent tests are required: a
preliminary screening and a Challenge test. If each requires p <0.001, there's the one in a million. But wait, tip of the iceberg:

http://www.skepticalinvestigations.org/exam/Dace_amazing3.htm

an article by Ted Dace :

SKEPTICAL OF THE SKEPTICS
The Amazing 3 Meeting - Las Vegas, January 13-16, 2005

"according to the rules of Randi's competition, if a psychic ability
is proven, he must pay up. Randi stated to me that a preliminary test
would have to yield a probability of one in a thousand that the
results were due to chance. After passing the preliminary, the
investigator could commence with the formal test, which would have to
yield a probability against chance of one in a million."

 

[fls]

From Damien Broderick:

I gather that at least two independent tests are required: a
preliminary screening and a Challenge test. If each requires p <0.001, there's the one in a million.

It depends. The preliminary test is not always required, especially when testing someone who already has a presence in the field. For example, his test of Jacques Benveniste and his homeopathy experiments was done without preliminary screening, as was one of his more recent televised tests, on Derek Ogilvie.

He doesn't really specify p<0.001 as a requirement, that has simply been the requirement on some of the Challenge tests. Other tests have been undertaken with lower and higher p-values. I don't know where you got "one in a million" from.

But wait, tip of the iceberg:

http://www.skepticalinvestigations.org/exam/Dace_amazing3.htm

an article by Ted Dace :

SKEPTICAL OF THE SKEPTICS
The Amazing 3 Meeting - Las Vegas, January 13-16, 2005

"according to the rules of Randi's competition, if a psychic ability
is proven, he must pay up. Randi stated to me that a preliminary test
would have to yield a probability of one in a thousand that the
results were due to chance. After passing the preliminary, the
investigator could commence with the formal test, which would have to
yield a probability against chance of one in a million."

I wouldn't give much weight to some off-hand remarks that may or may not have been reported correctly. I would look instead at what the practice has been.

I didn't mean to give the impression that one can definitely win the Challenge with a single test at p=0.01 or 0.001. I simply wished to point out that there have been circumstances where a single successful test would have won the Challenge, and that the p-values have varied around levels that are considerably larger than one in a million or the one in one hundred million that Radin used.

Linda

 

[fls]

From Suitbert Ertel:

Linda,

regarding your first query in your message yesterday, you argue that I
ignored the influence of normal looking. You did not ask me whether I did or
did not ignore that. Since you implied it without evidence it seems that you
want me to have ignored the influence of normal looking. Somewhat biased,
isn't it?

No doubt, in the filmed test session Kannan might have peeped into the bag
sometimes or even often in order to make ball selections aided by visual
support. Can we tell from observing him on the video whether he peeped into
the bag? Not with certainty.

I was aware at that time of the possibility that Kannan might have peeped
into the bag, because I once made a note between test sessions about the
possible peeping factor. I looked the note up and summarize here:

1. If Kannan had obtained hits by peeping into the bag he would have mixed
up similar numbers (2 and 5 for example) more often than dissimilar numbers
(1 and 5) (1 is written as a dash in India as in England). This I did not
find. The percentage of mixing up 2 with 5 when he called 2 was 14.01%, the
percentage of mixing up the two numbers was 15.3 % when he called 5. The
percentages of mixing up 1 with 5 which should be less (dissimilar numbers)
was 10.4% and 14.0%, which is somewhat less, but not significantly so. Let
me provide all mixing up cases (drawn wrong numbers in brackets). The
percent values are given because the differences among calls are evened out.
1 (2) 20.5%
1 (3) 09.4%
1 (4) 08.7%
1 (5) 10.4%

2 (1) 25.1%
2 (3) 13.5%
2 (4) 11.1%
2 (5) 14.0%

3 (1) 13.6%
3 (2) 15.3%
3 (4) 16.0%
3 (5) 16.7%

4 (1) 12.9%
4 (2) 15.7%
4 (3) 10.1%
4 (5) 16.7%

5 (1) 14.0%
5 (2) 15.3%
5 (3) 11.2%
5 (4) 17.1%

You can see that the frequencies of drawing 2 when 1 was called and drawing
1 when 2 was called are the only conspicuous cases. The reason cannot be
figural similarity. The reason is arithmetical "similarity". I am entitled
to say that, I think, because this result has been obtained on a larger
scale in my standard sample of 238 student participants, i.e. also with
calls of 3 (here 2 and 4 were more frequently drawn than 1 and 5) and with
calls of 4 (here 3 and 5 were more frequently drawn than 1 and 2). For calls
of 5 the rule of picking neighboring numbers did not apply (4 and 3 were
not drawn more frequently than 1 and 2). I am telling you this, Linda,
because this result with "good misses" which also shows up to some extent in
Kannan's data does not support the skeptics' assumption that hits above
chance can be explained by unintended or fraudulent sensory leakage.

2. I tested Kannan as experimenter in 1996, 1998, and 2001, the total number
of trials was 5880. It is unreasonable to assume that Kannan, if he cheated
by peeping into the bag, could have done this throughout without me noticing
this. Once I a while, he would have needed longer than cursory peeping
durations forgetting to keep his searching the numbers unnoticeable. I was
aware of this possibility as I said. Even Randi, I think, could not cheat
under my standard conditions for 20 hours (total testing time with Kannan).

3. All visitors to India are familiar with interruptions of current supply
once or twice almost every day. This occurred frequently when I tested
Kannan at night. We lit a candle when this occurred. Kannan could not see
the numbers during such periods which lasted for half an hour, an hour or
longer. He had to hold the drawn balls close to my eyes so that I could see
and put down the numbers. If Kannan had obtained high hit scores during the
day by fraudulent peeping, his scores would have dropped remarkably, I would
have noticed this drop of hits, I can be sure, because Kannan's hit rate was
remarkable (42.5% hits in 1996 and 44.3% in 1998 while 20% are expected. In
2001 I used the draw-from-two-bags condition which is dissimilar, but the
combined results in 2001 were also remarkable).

Linda, it is very easy for skeptics to put verbal obstacles on the road of
thoroughly working psi researchers and to attribute negligence on them by
quick conjectures. Do you think I have deserved to be suspected to have
ignored, for sure, the peeping factor? Much more do I have to endure in
discussions with other skeptics - and also with skeptical colleagues from
our research field, more about that later.

Suitbert

 

[fls]

From Suitbert Ertel:

Linda,

regarding your first query in your message yesterday, you argue that I
ignored the influence of normal looking. You did not ask me whether I did or
did not ignore that. Since you implied it without evidence it seems that you
want me to have ignored the influence of normal looking. Somewhat biased,
isn't it?

I didn't mean to imply that you had ignored it altogether. You even mention at the end of the video those postcaution analyses that you mention here. But in the papers that have been referenced, you not only fail to draw attention to this, you specifically seem to deny that it could occur - "attempts to peep into the box or other such fraudulent actions would hardly go unnoticed, apparently they have not occurred." And realistically, this is a very important issue which deserves at least as much attention as the four pages you devote to the idea of "good misses", and which, once recognized, should have led to a change in procedure so that the bag opening was screened from the drawer. Instead you make a few guesses about how this would show up in other ways, tested those ideas (without any indication of the probability of Type II error), and concluded that it couldn't be so. Yet an equally valid conclusion from your failure to confirm any of your guesses is that your guesses were simply wrong to begin with.

I don't think it is "without evidence" that I suggest you have not given this the attention it deserves. You have yet to assure me that the ball-drawing procedure in your other tests is substantially different from that used by Kannan.

No doubt, in the filmed test session Kannan might have peeped into the bag
sometimes or even often in order to make ball selections aided by visual
support. Can we tell from observing him on the video whether he peeped into
the bag? Not with certainty.

I was aware at that time of the possibility that Kannan might have peeped
into the bag, because I once made a note between test sessions about the
possible peeping factor. I looked the note up and summarize here:

1. If Kannan had obtained hits by peeping into the bag he would have mixed
up similar numbers (2 and 5 for example) more often than dissimilar numbers
(1 and 5) (1 is written as a dash in India as in England). This I did not
find. The percentage of mixing up 2 with 5 when he called 2 was 14.01%, the
percentage of mixing up the two numbers was 15.3 % when he called 5. The
percentages of mixing up 1 with 5 which should be less (dissimilar numbers)
was 10.4% and 14.0%, which is somewhat less, but not significantly so. Let
me provide all mixing up cases (drawn wrong numbers in brackets). The
percent values are given because the differences among calls are evened out.
1 (2) 20.5%
1 (3) 09.4%
1 (4) 08.7%
1 (5) 10.4%

2 (1) 25.1%
2 (3) 13.5%
2 (4) 11.1%
2 (5) 14.0%

3 (1) 13.6%
3 (2) 15.3%
3 (4) 16.0%
3 (5) 16.7%

4 (1) 12.9%
4 (2) 15.7%
4 (3) 10.1%
4 (5) 16.7%

5 (1) 14.0%
5 (2) 15.3%
5 (3) 11.2%
5 (4) 17.1%

Doesn't this data simply show that he didn't mix up 2's and 5's when looking at them? Do we usually expect people to do this?

You can see that the frequencies of drawing 2 when 1 was called and drawing 1 when 2 was called are the only conspicuous cases. The reason cannot be figural similarity. The reason is arithmetical "similarity". I am entitled
to say that, I think, because this result has been obtained on a larger
scale in my standard sample of 238 student participants, i.e. also with
calls of 3 (here 2 and 4 were more frequently drawn than 1 and 5) and with
calls of 4 (here 3 and 5 were more frequently drawn than 1 and 2). For calls
of 5 the rule of picking neighboring numbers did not apply (4 and 3 were
not drawn more frequently than 1 and 2). I am telling you this, Linda,
because this result with "good misses" which also shows up to some extent in
Kannan's data does not support the skeptics' assumption that hits above
chance can be explained by unintended or fraudulent sensory leakage.

Why not?

Your "good misses" analysis (from http://www.psych.uni-goettingen.de/home/ertel/ertel-dir/downloads/ertelchapterwithfigurespdf.pdf) depends upon a variable definition of what constitutes a "good miss". For some numbers it means the number above and the number below, for other numbers it means the number above and the next number after that, and for other numbers it means the number below and the next number after that. If you redo your analysis using a consistent definition for a "good miss", such as a miss for #2 is #3 and #4, and a miss for #3 is #4 and #5, the significant differences disappear. You cannot draw reliable conclusions from data-dredging after the fact whereby you keep grouping numbers together in various combinations until you find some groups that show a significant difference. At best, it can form the basis for exploratory research, but not if you can't even form a definition of a good miss a priori. How are you supposed to know, in advance, for each number, whether a good miss means the number on each side or the two numbers below or the two numbers above?

2. I tested Kannan as experimenter in 1996, 1998, and 2001, the total number
of trials was 5880. It is unreasonable to assume that Kannan, if he cheated
by peeping into the bag, could have done this throughout without me noticing
this. Once I a while, he would have needed longer than cursory peeping
durations forgetting to keep his searching the numbers unnoticeable. I was
aware of this possibility as I said. Even Randi, I think, could not cheat
under my standard conditions for 20 hours (total testing time with Kannan).

According to what you have stated and the information provided on the video, 94% of the trials were performed before you were aware that he could peep. It was only during the last 360 trials that you were aware of the possibility and were presumably on the alert for this. The last 360 trials also show a dramatic decline in his hit rate, from a previous effect size of 0.60 to an effect size of 0.10.

3. All visitors to India are familiar with interruptions of current supply
once or twice almost every day. This occurred frequently when I tested
Kannan at night. We lit a candle when this occurred. Kannan could not see
the numbers during such periods which lasted for half an hour, an hour or
longer. He had to hold the drawn balls close to my eyes so that I could see
and put down the numbers. If Kannan had obtained high hit scores during the
day by fraudulent peeping, his scores would have dropped remarkably, I would
have noticed this drop of hits, I can be sure, because Kannan's hit rate was
remarkable (42.5% hits in 1996 and 44.3% in 1998 while 20% are expected. In
2001 I used the draw-from-two-bags condition which is dissimilar, but the
combined results in 2001 were also remarkable).

Wouldn't the low light conditions simply make it harder to notice other ways of manipulating the draw, like simply palming balls drawn on prior rounds to serve as guesses on subsequent rounds? If you can't even see the number unless it is close to your eyes, how can you be sure he put the balls back in the bag?

Linda, it is very easy for skeptics to put verbal obstacles on the road of
thoroughly working psi researchers and to attribute negligence on them by
quick conjectures. Do you think I have deserved to be suspected to have
ignored, for sure, the peeping factor? Much more do I have to endure in
discussions with other skeptics - and also with skeptical colleagues from
our research field, more about that later.

Suitbert

I don't know what to say. I sincerely think that psi conducive conditions shouldn't be the same as "conditions under which manipulation can occur". There must be a way for these two things to be different.

Linda

 

[fls]

From Suitbert Ertel:

But again, this could be tested and controlled - at what point does changing the bag take away the effect of psi? Is it by using 5 bags rather than one? Is it by having someone else mix the bags? Is it by making the choice of bag random, rather than the choice of the participant?

I appreciate your idea to put hypotheses to a test. I did not claim that using 5 bags rather than one would take the effect of psi away. Among the long list of my experimental to-do's the changing of bags across trials has been included. If hit scores would decline under the change-bags condition, however, sensory leakage effects under the no-bags-change condition would not be strictly confirmed, that's what I wanted to say.

That is good to hear, because I have a solution to your lack of funds and the lack of interest from other scientists. If you have not already heard
of this, James Randi offers one million dollars to anyone who can
demonstrate psi in the way that you describe. All you have to do is have
your Ukrainian psi stars demonstrate their very significant results and you
not only have the funds you need for further research, but also the
attention of other scientists as well.

I am well aware of Randi's million dollar campaign. There are several hurdles to overcome. One is Randi's demand of so-called precautions which, if fulfilled, intimidate the participants by artificial and, partly non-sensical procedures. Since psi effects can be demonstrated only under appropriate and fair conditions the precautions demanded by skeptics tend to turn into the opposite of precaution regarding effect-producing conditions.

This I can say because I tried to run this test using a psi-gifted student (Silke) under the supervision of a German skeptic group (GWUP) in Würzburg. My test conditions were grossly altered by GWUP representatives, but I had to concur, otherwise this project would have been brought to a close. This has been documented in an article that I published in German:

Ertel, S. (2007). "Außersinnliche Wahrnehmung unter der Kontrolle organisierter Skeptiker." [Extrasensory perception under control of organized skeptics] Zeitschrift für Anomalistik 7(3): 236-269.

Despite authoritarian changes of the test conditions by GWUP (money is power!) and despite intimidations of participant Silke by additional subtle means, the observed hit score obtained in roughly two hours was very significant (p = .002). This, however, did not reach the skeptics' (Randi's and the GWUP's) effect size. Thus, Randi, a person uneducated in matters of statistics, and GWUP, who organized themselves without institutional control by science organizations, define which p-values are to be considered as highly significant irrespective of what the science community considers as highly significant.

In the year following the year of my first attempt with GWUP (pre-test for JREF’s million dollar test) I intended to make another attempt in Würzburg. But unfortunately Silke did not want to continue this project because of her bad first impressions the year before.

I hoped another very psi-gifted student would cooperate, but he refused to serve as participant in Würzburg at the last moment. He was afraid of endangering his scientific career if his extraordinary psi performance would become public.

One of the three Ukrainean participants (a boy of 10 years, Vanya is his nickname, he needed to travel with his mother) failed to get a visa in time. There were two other psi-gifted students left and I persuaded them, with much effort, to cooperate. Their mental conditions, however, were awful at that time (one was in divorce and the other was overstressed physically, overslept and almost missed the train). For this reason, most probably, and due to those unnatural suppressive GWUP conditions, they did not obtain hit surpluses above chance.

In future, I would like to let my Ukrainean boy Vanya be tested under my standard conditions because Vanya had shown extraordinary psi power under a skeptic's control (not GWUP, but also skeptical) and subsequently under my own control in Germany where I had invited him. But his increasing time problems (high school achievement goals etc.) his mother's chronical health problems together with visa problems (Germany raised obstacles because of masses of Ukraineans who were flooding Germany) have thwarted this idea. I asked GWUP whether they are aware of any skeptical organization in Ukraine so that Vanya could be tested by some skeptic in his home town, but GWUP could not help me in this regard. I myself will travel to Ukraine (Kiev) in August to test Vanya. For the first time in my psi research career, I obtained some financial support (from an open-minded non-scientific company (4,000 $ has been awarded). I will conduct various tests with Vanya, also with his sick mother and his handicapped grandmother. But I would prefer as experimenter some skeptic.

In order to avoid violations of precaution regarding fair conditions I would attempt to convince the skeptic to first agree with the ball test conditions as I define them. The skeptic should merely act as observer, a mini-video camera might also be used. My hypothesis is that the members of Vanya’s family would obtain highly significant hit scores, say p = .0001 within roughly 1-2 hours testing time and that the observer would not be able to explain the result.

Later on, the skeptical observer or some conjurer of his choice should take the ball test under the same condition. I would again act as experimenter. My hypothesis is that the conjurer’s hit score would not deviate from chance. If the score would exceed significantly the level of chance expectation, though, or, I expect to notice, within 1-2 hours testing time, a trick that the applies. If I would not discover a trick the conjurer would have to disclose, when the test is done, which trick he applied. He would also have to make plausible that the members of the Kiev family used his own or some other trick. If the conjurer confessed under oath that he did not apply any trick then I would engage him as another psi-gifted participant for further research.

These many words, Linda, that I have to use in order to give you an idea of what I did for that project and what I am willing to do in the future despite lack of funds (except 2000 US on one occasion which is 3% of my total expenditures of a decade of psi research, those 4,000 USD that I mentioned will not cover all my future expenses) are somehow apt to discourage my endeavour to find out whether psi exists and whether the positive results of 10 years of research that I obtained are genuine or not. My main obstacles are not empirical givens, but dogged presuppositions of disbelievers. In your case, however, I am beginning to feel that I am communicating with a sufficiently open-minded person who might become skeptical, by challenging empirical and trustful information, to her own beliefs. This would show that you are truly skeptical.

I am also still curious as to how you view those parapsychologists who prefer methodological rigour rather than the use of postcaution? Do you also consider their attitude detrimental to the study of psi, or a demonstration of the power of human disbelief?

A good question. Yes, I consider the behavior of those who reject discussions of my approach in public (in professional Journals) as detrimental to the study of psi. These seemingly critical members of the parapsychological community forego the chance to demonstrate or to let readers of their Journals demonstrate, by empirical evidence, that Ertel results are due to errors, sensory factors or even fraud. Ertel-contradicting results would even raise the reputation of the Parapsychological Association.

How do I view the parapsychologists who prefer methodological rigor rather than the use of post-caution. My impression, as a newcomer to the parapsychological community, is that in this community the fear of criticism by disbelievers, above all of organized skeptics, is a strong motivating force. Members of the Parapsychological Association tend to succumb to often irrational demands and contentions of their opponents and to ignore demands and insights of common sense and common logic.

An example: Psychologists are aware of the fact that some participants of their questionnaire studies do not adequately respond to their questionnaire items (response sets etc), some participants even cheat. Never has any scientist doubted that the Big 5 factorial dimensions of personality are real, that they are not due to fraudulent or other result-spoiling factors. If fraud or other result-spoiling influence occurred, with empirical evidence, in my ball test this test would have to be abolished at one. Even without such evidence which I have been searching in vain for a decade, the mere speculation that fraud and sensory leakage might occur leads “rational” scientists to keep my research at bay. Damien Broderick is an exception.

...if you had understood the relevance of Type II error to my criticisms, I could simply move on instead of bringing it up again below.

I think I understand the relevance of the distinction between Type I and Type II error in this context. I hope you agree that both errors must be considered. Excluding one of these two errors by all means increases the probability of an occurrence of the opposite error. We have to aim at a compromise between the two probabilities. Linda, don’t you disregard Type I errors?

In parapsychological research, type I errors are diminished by looking at possible artifacts so as to avoid attributing effects to psi if artifacts are effective – a dangerous tendency of incautious believers.

Type II errors, on the other hand, are diminished by looking at possible genuine psi factors that might have caused effects while incautious psi-disbelievers tend to attribute them to ordinary (sensory or motor) factors.

Both parties, a priori believers and a priori disbelievers, are committed to support their beliefs, by evidence. An important point here is that prevailing beliefs of other members of one’s community are no safeguard. Prevailing beliefs in science may be wrong. Linda, you disbeliever need not exert much effort to gain acclaim from the majority of the science community. I myself and other researchers who disregard the sociological power of majorities are in a less happy situation.

Suitbert

 

[Wowbagger]

From Damien Broderick:

"according to the rules of Randi's competition, if a psychic ability
is proven, he must pay up. Randi stated to me that a preliminary test
would have to yield a probability of one in a thousand that the
results were due to chance. After passing the preliminary, the
investigator could commence with the formal test, which would have to
yield a probability against chance of one in a million."

Hey, you know what? If psi phenomena was really as real as you think it is, then this should not be a problem! Real phenomena have a way of being reliably reproduced.



One more quote from that article I want to briefly mention, because it gets a lot of attention:

Randi agreed he might have to pay up someday. But Dawkins had a trick up his sleeve. If a “psychic” phenomenon turns out to be real, then by definition it is physical and therefore not really psychic after all, and thus Randi still shouldn’t have to pay.

This line, though well balley-hooed by paranormalists, is simply not relevant:
Once the testing protocol contract is signed, they will get the Million Dollars if they actually do what was required to pass the test; even if it was later discovered that their abilities could be understood by science. Randi indicated as much, during his interview with Dawkins, right after Dawkins' presentation. It's a legally binding contract, and if Randi (or his advisors) missed something, that lead to such an opening, then Randi would just have to eat it.

So far, he has not needed to.

 

[Wowbagger]

From Suitbert Ertel:

I am well aware of Randi's million dollar campaign. There are several hurdles to overcome. One is Randi's demand of so-called precautions which, if fulfilled, intimidate the participants by artificial and, partly non-sensical procedures. Since psi effects can be demonstrated only under appropriate and fair conditions the precautions demanded by skeptics tend to turn into the opposite of precaution regarding effect-producing conditions.

Instead of complaining that the test is "too hard", you should focus on how you can overcome its hurdles.

What can you do, to make psi phenomena more reliable, up to the point of "self-evident"?

 

[fls]

From Suitbert Ertel:

I am well aware of Randi's million dollar campaign. There are several hurdles to overcome. One is Randi's demand of so-called precautions which, if fulfilled, intimidate the participants by artificial and, partly non-sensical procedures. Since psi effects can be demonstrated only under appropriate and fair conditions the precautions demanded by skeptics tend to turn into the opposite of precaution regarding effect-producing conditions.

This I can say because I tried to run this test using a psi-gifted student (Silke) under the supervision of a German skeptic group (GWUP) in Würzburg. My test conditions were grossly altered by GWUP representatives, but I had to concur, otherwise this project would have been brought to a close. This has been documented in an article that I published in German:

Ertel, S. (2007). "Außersinnliche Wahrnehmung unter der Kontrolle organisierter Skeptiker." [Extrasensory perception under control of organized skeptics] Zeitschrift für Anomalistik 7(3): 236-269.

Despite authoritarian changes of the test conditions by GWUP (money is power!) and despite intimidations of participant Silke by additional subtle means, the observed hit score obtained in roughly two hours was very significant (p = .002). This, however, did not reach the skeptics' (Randi's and the GWUP's) effect size. Thus, Randi, a person uneducated in matters of statistics, and GWUP, who organized themselves without institutional control by science organizations, define which p-values are to be considered as highly significant irrespective of what the science community considers as highly significant.

In the year following the year of my first attempt with GWUP (pre-test for JREF’s million dollar test) I intended to make another attempt in Würzburg. But unfortunately Silke did not want to continue this project because of her bad first impressions the year before.

I hoped another very psi-gifted student would cooperate, but he refused to serve as participant in Würzburg at the last moment. He was afraid of endangering his scientific career if his extraordinary psi performance would become public.

One of the three Ukrainean participants (a boy of 10 years, Vanya is his nickname, he needed to travel with his mother) failed to get a visa in time. There were two other psi-gifted students left and I persuaded them, with much effort, to cooperate. Their mental conditions, however, were awful at that time (one was in divorce and the other was overstressed physically, overslept and almost missed the train). For this reason, most probably, and due to those unnatural suppressive GWUP conditions, they did not obtain hit surpluses above chance.

In future, I would like to let my Ukrainean boy Vanya be tested under my standard conditions because Vanya had shown extraordinary psi power under a skeptic's control (not GWUP, but also skeptical) and subsequently under my own control in Germany where I had invited him. But his increasing time problems (high school achievement goals etc.) his mother's chronical health problems together with visa problems (Germany raised obstacles because of masses of Ukraineans who were flooding Germany) have thwarted this idea. I asked GWUP whether they are aware of any skeptical organization in Ukraine so that Vanya could be tested by some skeptic in his home town, but GWUP could not help me in this regard. I myself will travel to Ukraine (Kiev) in August to test Vanya. For the first time in my psi research career, I obtained some financial support (from an open-minded non-scientific company (4,000 $ has been awarded). I will conduct various tests with Vanya, also with his sick mother and his handicapped grandmother. But I would prefer as experimenter some skeptic.

I'm sorry. I did not realize that you had been through all this already or I would not have suggested it. I don't know if it would have helped, but it is always advised that if the procedure is changed (and it always seems to be to add controls) that you try it out before coming to the preliminary test. This may have told you that Silke was going to be affected and may have given you the opportunity to figure out how to overcome this. The other thing that I have noticed with applicants is that they don't given themselves enough of a chance to succeed - they choose a number of trials with a low power. It sounds like if you had chosen a larger number of trials, the requisite p-value may have been reached. But maybe you ended up with not much say in the matter.

However, I have said many times on the JREF forum that the Challenge cannot be thought of as a scientific test. The standards used are different, and it's not suitable as a form of investigation/exploration.

There may be people on the JREF forum who know of an organization or even an individual in the Ukraine that could test Vanya.

In order to avoid violations of precaution regarding fair conditions I would attempt to convince the skeptic to first agree with the ball test conditions as I define them. The skeptic should merely act as observer, a mini-video camera might also be used. My hypothesis is that the members of Vanya’s family would obtain highly significant hit scores, say p = .0001 within roughly 1-2 hours testing time and that the observer would not be able to explain the result.

Later on, the skeptical observer or some conjurer of his choice should take the ball test under the same condition. I would again act as experimenter. My hypothesis is that the conjurer’s hit score would not deviate from chance. If the score would exceed significantly the level of chance expectation, though, or, I expect to notice, within 1-2 hours testing time, a trick that the applies. If I would not discover a trick the conjurer would have to disclose, when the test is done, which trick he applied. He would also have to make plausible that the members of the Kiev family used his own or some other trick. If the conjurer confessed under oath that he did not apply any trick then I would engage him as another psi-gifted participant for further research.

I don't really understand the idea of asking someone to duplicate an effect through trickery. It shows that the effect could be done through trickery, but we already know these things. It makes more sense to me to set up conditions (and this is where a conjurer comes in handy) so that any forms of deception (deliberate or not) are difficult to impossible and simply allow an opportunity for psi to act. Also, one of the other JREF members had some suggestions about running a test with gradually stricter controls - if you did not see this, I will copy it and send it to you. If this has the effect of not allowing an opportunity for psi to act, then you will have found your control group conditions (i.e. conditions under which psi is absent) for further research. If psi is still present under strictly controlled conditions, then you have found a way to win the Challenge.

This comes back to a point I made earlier, though. Control groups should not consist of non-psi-gifted people making an attempt to accomplish what psi-gifted people do. Control groups should consist of psi-gifted people who are working under conditions where psi does not have an opportunity to act. For example, you could start with your psi-gifted students and randomly divide them into two groups. You would give both groups your sealed psi-pods and tell them to draw out one at a time, labelling them with the order in which they were drawn. You would choose an order to draw out the psi-pod numbers (preferably randomly from a few choices, like 1234512345... or 1111122222...) and tell one group the order and tell the other group nothing. Then you could see if there was any difference between the two groups.

A good question. Yes, I consider the behavior of those who reject discussions of my approach in public (in professional Journals) as detrimental to the study of psi. These seemingly critical members of the parapsychological community forego the chance to demonstrate or to let readers of their Journals demonstrate, by empirical evidence, that Ertel results are due to errors, sensory factors or even fraud. Ertel-contradicting results would even raise the reputation of the Parapsychological Association.

In what way would publishing your results be beneficial? Would the demonstration that your results are due to errors, sensory factors or even fraud teach us something new about psi research? Or is it that you expect your results to be replicated under more rigorous conditions? Would you consider performing that work yourself (replicating your results under more rigorous conditions) in order to engage the interests of others?

How do I view the parapsychologists who prefer methodological rigor rather than the use of post-caution. My impression, as a newcomer to the parapsychological community, is that in this community the fear of criticism by disbelievers, above all of organized skeptics, is a strong motivating force. Members of the Parapsychological Association tend to succumb to often irrational demands and contentions of their opponents and to ignore demands and insights of common sense and common logic.

An example: Psychologists are aware of the fact that some participants of their questionnaire studies do not adequately respond to their questionnaire items (response sets etc), some participants even cheat. Never has any scientist doubted that the Big 5 factorial dimensions of personality are real, that they are not due to fraudulent or other result-spoiling factors. If fraud or other result-spoiling influence occurred, with empirical evidence, in my ball test this test would have to be abolished at one. Even without such evidence which I have been searching in vain for a decade, the mere speculation that fraud and sensory leakage might occur leads “rational” scientists to keep my research at bay. Damien Broderick is an exception.

This is because the Big 5 has already gone through the type of testing that I earlier encouraged you to try. Tools such as the NEO PI-R have been tested to determine whether or not they are actually able to identify characteristics such as Agreeableness a priori compared to other ways of identifying these characteristics. Construct, content and criterion validity have been established. No such testing is available for psi - it can only be determined a posteriori as a deviation from chance expectation. And it lacks independent confirmation as to validity. For example, has it ever been established that telepathy would manifest itself in highly local card-guessing scenarios?

I think I understand the relevance of the distinction between Type I and Type II error in this context. I hope you agree that both errors must be considered. Excluding one of these two errors by all means increases the probability of an occurrence of the opposite error. We have to aim at a compromise between the two probabilities. Linda, don’t you disregard Type I errors?

I don't consider an alpha of 0.05 and a beta of 0.40 a "compromise". Also, if you are asking a question where the avoidance of Type II error is critical and that of Type I error is almost irrelevant, your alpha and beta really should reflect that question.

In parapsychological research, type I errors are diminished by looking at possible artifacts so as to avoid attributing effects to psi if artifacts are effective – a dangerous tendency of incautious believers.

Type II errors, on the other hand, are diminished by looking at possible genuine psi factors that might have caused effects while incautious psi-disbelievers tend to attribute them to ordinary (sensory or motor) factors.

Now you are talking about something else. You are talking about preventing the possibility of giving up a fruitful area of investigation too early. Yet, power analyses of parapsychological research show that the probability of Type II error is low when it comes to missing an obvious effect (and we are looking for something that should be obvious since our ideas about psi come from informal observation). However, that is not relevant to why I brought up Type II error. I was talking about missing the effect of bias (i.e. missing a small to very small effect, corresponding to the effect sizes you have measured), not missing the effect of psi.

Both parties, a priori believers and a priori disbelievers, are committed to support their beliefs, by evidence. An important point here is that prevailing beliefs of other members of one’s community are no safeguard. Prevailing beliefs in science may be wrong. Linda, you disbeliever need not exert much effort to gain acclaim from the majority of the science community. I myself and other researchers who disregard the sociological power of majorities are in a less happy situation.

Suitbert

Yet many other scientists have overcome this less happy situation simply by providing evidence. Instead of investing time and effort into repeating tests which non-believers do not find persuasive and then trying to find ways to manipulate the data a posteriori, why not simply invest your time and effort into developing adequate precautions which address the concerns of other scientists?

Linda

 

[fls]

From Suitbert Ertel:

And realistically, this is a very important issue which deserves at least as much attention as the four pages you devote to the idea of "good misses", and which, once recognized, should have led to a change in procedure so that the bag opening was screened from the drawer. Instead you make a few guesses about how this would show up in other ways, tested those ideas (without any indication of the probability of Type II error), and concluded that it couldn't be so. Yet an equally valid conclusion from your failure to confirm any of your guesses is that your guesses were simply wrong to begin with.

I never noticed as experimenter, in thousands of trials, that participants
peeped into the bag before they picked a ball from the bag. Peeping into the
bag is cheating. I claim that I would have noticed in at least some such
cases that a participant cheated that way and I also claim that students are
not inclined to cheat in the lab and not motivated to cheat. Even if some
pathological or criminal participant existed who was clever enough to cheat
in a way that I did not notice, the bulk of hit surpluses in my data base
cannot overall be explained by cheating (peeping into the bag).

This I can say because I also tested some high scoring participants using
balls with ill-written numbers on them (very small, in disorder etc). Their
hit scores did not decline.

I tested high-scoring participants, high-scoring with Arabic numbers written
on the balls, with number words written on the balls. The increased
difficulty of reading what was written on the balls had no effects on their
hit scores.

I also asked some very successful participants to wear blinds, their hit
scores did not drop.

I asked some participants to try to cheat at home in such a way that I, as
experimenter, might gain the impression that they are psi-gifted. But they
should be careful, I told them, so that their fraud would not be noticed. In
the great majority of these cases (roughly ten cases, as far as I remember)
were the hit scores excessively high or showed typical indications of
non-randomness of their seemingly "drawn" numbers thus revealing their
manipulation.

In other words, since I did not find indications of such fraudulent
manipulation in the students' home test data I think I am entitled to
conclude that the bulk of even home test data of my participants is clean.

I mentioned some studies which you, Linda, would probably call control
group studies. I should remind you also of the recent psi pod studies that I
have started with written numbers under lids of small containers so that the
numbers cannot be seen even by attentively looking into the bag. With using
psi pods the hit scores of my Ukrainean participants did not drop.

Linda, you demand to change the procedure so that the bag opening is
screened from the drawer. In view of the just reported results which clarify
the peeping question I am not inclined to screen the bag opening from the
drawer. This would render the situation artificial, the participants would
feel that they are distrusted. We have to be careful to avoid type 2 errors.

On that occasion, Linda, I would like to ask you to be more careful with
your critical claims as long as you are unaware of the many control
experiments that I actually conducted. I am prepared to ask further
questions, but I think the time needed for refuting unnecessary and
unsubstantiated claims by true skeptical believers should be diminished.

If you redo your analysis using a consistent definition for a "good miss", such as a miss for #2 is #3 and #4, and a miss for #3 is #4 and #5, the significant differences disappear.

A good miss of one's choice is a choice which was "almost" a hit. When you
reach the desired goal a little too early or a little too late, when you
throw the ball a little too near or a little too far etc. you had "almost" a
hit. Your objection is arbitrary: "You cannot draw reliable conclusions from
data-dredging after the fact whereby you keep grouping numbers together in
various combinations until you find some groups that show a significant
difference."

At best, it can form the basis for exploratory research, but not if you can't even form a definition of a good miss a priori. How are you supposed to know, in advance, for each number, whether a good miss means the number on each side or the two numbers below or the two numbers above?

Among the possible numerical differences between hits (1,2,3,4,5) and four
misses two are smaller and two are larger. This I consider as a clear-cut
definition of "good misses". Why do you want to restrict good misses to
drawing the larger adjacent number? Even if you do that the effect does not
disappear, it is even stronger. By restricting good misses to drawing the
smaller adjacent number - why should one do that? - the effect is still
apparent, but less so. The good misses result is destroyed by your arbitrary
and untenable redefinition of a "good miss" ("almost hit").

Wouldn't the low light conditions simply make it harder to notice other ways of manipulating the draw, like simply palming balls drawn on prior rounds to serve as guesses on subsequent rounds? If you can't even see the number unless it is close to your eyes, how can you be sure he put the balls back in the bag?

You seem to deem it likely that Kannan had two tricks at his disposal. Did
you notice any such "palming balls" behavior in the filmed test session?
Certainly not. Participants are told by the instruction anyway to let the
just drawn ball bounce back into the bag.

Let me generalize: The general strategy of convinced skeptics is to explain
surprising observations by unsurprising processes or mechanisms of ordinary
science (psychology, physics) even if the processes or mechanisms claimed in
their objections are extremely improbable and sometimes even inconceivable.
Rarely feel skeptics challenged to put their improbable ordinary
explanations to empirical tests. In this regard, Linda, I occasionally
noticed in your comments hopeful deviations, at least insofar as you
suggested to me or to the parapsychological community to test your
conjectures. Why don't you do parapsychological experiments? Or are you
conducting such project?

Suitbert

 

[fls]

From Suitbert Ertel:

I never noticed as experimenter, in thousands of trials, that participants
peeped into the bag before they picked a ball from the bag. Peeping into the
bag is cheating. I claim that I would have noticed in at least some such
cases that a participant cheated that way and I also claim that students are
not inclined to cheat in the lab and not motivated to cheat. Even if some
pathological or criminal participant existed who was clever enough to cheat
in a way that I did not notice, the bulk of hit surpluses in my data base
cannot overall be explained by cheating (peeping into the bag).

This I can say because I also tested some high scoring participants using
balls with ill-written numbers on them (very small, in disorder etc). Their
hit scores did not decline.

I tested high-scoring participants, high-scoring with Arabic numbers written
on the balls, with number words written on the balls. The increased
difficulty of reading what was written on the balls had no effects on their
hit scores.

I also asked some very successful participants to wear blinds, their hit
scores did not drop.

I asked some participants to try to cheat at home in such a way that I, as
experimenter, might gain the impression that they are psi-gifted. But they
should be careful, I told them, so that their fraud would not be noticed. In
the great majority of these cases (roughly ten cases, as far as I remember)
were the hit scores excessively high or showed typical indications of
non-randomness of their seemingly "drawn" numbers thus revealing their
manipulation.

In other words, since I did not find indications of such fraudulent
manipulation in the students' home test data I think I am entitled to
conclude that the bulk of even home test data of my participants is clean.

Thank you for recognizing that this is an important issue. It may be useful to include details of this information in any subsequent articles you write, rather than providing this information in bits and pieces after much discussion.

I think that you can conclude that when asked to deliberately cheat, and when told in advance that they will not suffer consequences from their cheating, that they will cheat in ways that may or may not look different from less deliberate cheating. Your description does not sound like you were blind as to where to look for fraud (something which would be a necessary pre-condition in order to drawn valid conclusions).

Can you provide references for the research, or show me the results of your own research which shows the extent to which words or small numbers cannot be read by the average participant (necessary information before valid conclusions can be drawn)? I hope that you are aware that blind-folds, unless applied in a specific manner, do not prevent people from seeing (especially at the angle which would be used for the ball-drawing). Did you tightly tape the bottom of the blindfolds around the nose and cheeks?

I attempted the following analysis from the results you have provided. It is possible that I misunderstood the information presented in the tables, since it was not designed for this type of analysis, so I apologize in advance if I am missing some key information.

It seems that the hit surplus (in the uncontrolled setting) is mostly accounted for by a small number of people. Using your data from table 1 (http://www.psych.uni-goettingen.de/home/ertel/ertel-dir/downloads/ertelchapterwithfigurespdf.pdf) only 3 people are responsible for almost all of the hit surplus (79% of the hit surplus). Of those 3, 2 were retested and one was not. Retesting of one of them (Katarina H.) led to chance results. Only after testing her over and over again, under conditions where some of the controls were dropped (e.g. video-taping), did her hits begin to climb past chance level. This may be consistent with her becoming comfortable with the test conditions, but it is also consistent with learning deception (not necessarily deliberate). We don't know how the one person responsible for more of the surplus hits than anyone else (Barbara F. was responsible for 33% of the hit surplus) would have done under controlled conditions. And you don't give the details of the retesting on the third person (Gabriela G.). So while you reference extensive testing for cheating, it is only relevant if that testing was performed on those three people. It didn't matter if you couldn't detect cheating in anyone else, because they didn't really contribute to the significance of your results.

You don't provide the same sort of details for the results under controlled conditions, however you do give the details of three people (http://www.parapsych.org/papers/38.pdf). And we can see that 2 of the people (Amelie J. and Silk T.) account for almost half (47%) of the hit surplus under controlled conditions for one set of experiments. And one person (Ahmed K.) accounts for 38% of the hit surplus under controlled conditions for the other set of experiments. Again, it looks like the bulk of your hit surplus is accounted for by a few people, so tests designed to look for cheating cannot show a difference if applied to anyone other than those few people. Can you provide those details for us? Is that the same Silke that was unable to perform above chance under the strictly controlled conditions for the GWUP prize?

I mentioned some studies which you, Linda, would probably call controlgroup studies. I should remind you also of the recent psi pod studies that I have started with written numbers under lids of small containers so that the numbers cannot be seen even by attentively looking into the bag. With using psi pods the hit scores of my Ukrainean participants did not drop.

Linda, you demand to change the procedure so that the bag opening is
screened from the drawer. In view of the just reported results which clarify
the peeping question I am not inclined to screen the bag opening from the
drawer. This would render the situation artificial, the participants would
feel that they are distrusted. We have to be careful to avoid type 2 errors.

On that occasion, Linda, I would like to ask you to be more careful with
your critical claims as long as you are unaware of the many control
experiments that I actually conducted. I am prepared to ask further
questions, but I think the time needed for refuting unnecessary and
unsubstantiated claims by true skeptical believers should be diminished.

You agree that this is important information if you went to the trouble to look for these sorts of things. Therefore, I don't understand why you are chastizing me for wasting your time by asking for this same information. And I am sorry that I remain unconvinced until I have been provided with this information, but would you really be any different? If someone made a claim that had not been validated independently, would you not also ask for details and for the answers to critical questions before allowing yourself to be persuaded?

A good miss of one's choice is a choice which was "almost" a hit. When you reach the desired goal a little too early or a little too late, when you throw the ball a little too near or a little too far etc. you had "almost" a
hit. Your objection is arbitrary: "You cannot draw reliable conclusions from
data-dredging after the fact whereby you keep grouping numbers together in
various combinations until you find some groups that show a significant
difference."

I understand what you mean by a "good miss", but what you have done is sometimes counted when the ball was a little too near as a "good miss" and the ball was a little too far as an "other miss" and sometimes when the ball was a little too far as a "good miss" and the ball was little too near as an "other miss", and it is this uneven counting that accounts for the "significant difference". If you always count the ball that was a little too far as a "good miss", that significant difference disappears. And you can't tell beforehand (i.e. before you start throwing the ball), whether you will be better off counting near or far balls as a "good miss". It is only by looking at the pattern after you have thrown the balls that you can pick out which particular combination of misses (near on trial 1,2,5 and far on trial 3, 4, for example) will give you a "statistically significant difference".

Among the possible numerical differences between hits (1,2,3,4,5) and four misses two are smaller and two are larger. This I consider as a clear-cut definition of "good misses". Why do you want to restrict good misses to
drawing the larger adjacent number? Even if you do that the effect does not
disappear, it is even stronger. By restricting good misses to drawing the
smaller adjacent number - why should one do that? - the effect is still
apparent, but less so. The good misses result is destroyed by your arbitrary
and untenable redefinition of a "good miss" ("almost hit").

Why is a good miss the larger adjacent number for one number, but not a good miss for another number? How can a good miss be the number on either side for one number, but it isn't a good miss for another number? My complaint isn't that I think there is only one possible definition for a good miss. My complaint is that there are many possible definitions for a good miss. And that not only do you not tell us how to choose between these various possibilities *a priori*, your choice isn't even consistent from one number to the next. How can I tell what is a "good miss" if it can be changed at will? Because if I change it, then your idea of "good misses" as a secondary effect disappears when it is discovered that a different combination of "good misses" does not lead to a statistically significant difference (and subsequently, one of your arguments against other explanations of the effects - forged data - disappears).

You seem to deem it likely that Kannan had two tricks at his disposal. Did you notice any such "palming balls" behavior in the filmed test session? Certainly not. Participants are told by the instruction anyway to let the just drawn ball bounce back into the bag.

Are you serious? Why couldn't Kannan have a dozen tricks at his disposal? Why would he palm balls when peeking would do the trick? I've watched magicians very carefully up close and not detected palming when it clearly had to have happened. How could I detect palming from that video even if it were present?

Let me generalize: The general strategy of convinced skeptics is to explain surprising observations by unsurprising processes or mechanisms of ordinary science (psychology, physics) even if the processes or mechanisms claimed in their objections are extremely improbable and sometimes even inconceivable. Rarely feel skeptics challenged to put their improbable ordinary
explanations to empirical tests. In this regard, Linda, I occasionally
noticed in your comments hopeful deviations, at least insofar as you
suggested to me or to the parapsychological community to test your
conjectures. Why don't you do parapsychological experiments? Or are you
conducting such project?

Suitbert

Why do you characterize my objections as extremely improbable and sometimes even inconceivable *when you also admit to the possibility of peeking or other methods of cheating*?

When I am used to working with people who take care to address issues of validity or reliability in their own research, it irks me to see people asking others to do it for them.

Linda

 

[fls]

From Damien Broderick:

Instead of investing time and effort into repeating tests which non-believers do not find persuasive and then trying to find ways to manipulate the data a posteriori, why not simply invest your time and effort into developing adequate precautions which address the concerns of other scientists?

This is my sentiment also, as an onlooker. But I can see the value,
for those who already feel convinced that psi is real, of
process-oriented research that seeks ways to enhance psychic
effectiveness. Obviously if that risks compromising the results with
unreliable or fraudulent data points, it might prove misleading or
time-wasting. But I can understand why using methods that don't
alienate or bore the purportedly "psi-gifted" would be worth trying,
then gradually tightening the conditions as the parameters come into
focus. Granted, the more slack there is in such methodologies, the
less an experimenter can complain if skeptics refuse to accept the
results as evidence of psi--but then that's the trade-off. I've never
met Suitbert nor seen his ball-drawing method applied (except on a
video) but I've always found him open to serious non-captious
critique, ready to close loopholes if possible or try variants that
seem more secure or more interesting. And it's obvious to me that his
test is more likely to elicit whatever evolved competences we have
for psi--the haptic character of feeling, touching, seizing, in
situations that are at least somewhat more engaging than the dreary
task of hitting keys or guessing cards, or even (for some) lying in
the half-dark trying to visualize a distant scene.

Damien

 

[Ivor the Engineer]

And it's obvious to me that his test is more likely to elicit whatever evolved competences we have for psi--the haptic character of feeling, touching, seizing, in situations that are at least somewhat more engaging than the dreary task of hitting keys or guessing cards, or even (for some) lying in the half-dark trying to visualize a distant scene.

Yet casinos have not had to worry about vast swathes of gamblers having their psi-abilities fine-tuned and amplified because of the interest they have in the game they are playing.

 

[Uncayimmy]

Thanks for the effort taken in this thread.

I finally got around to watching the video. The first thing that popped into my mind was that the balls were not shuffled adequately. He just kind of bounces the bag up and down. It doesn't appear that it was sufficient to cause the balls to switch positions very much. It's hard to imagine that a ball all the way on one side would make it all the way to the other.

If I wanted to cheat on that test, I would attempt to get the balls loaded into the bag so that the numbers were grouped. By that I mean the 1-balls would be in the left corner, the 2-balls next to them and so forth with the 5-balls being on the right hand side. When preparing the experiment, it would only be natural to group the balls by number first to make sure you have the right counts. Putting the balls in the bag as I describe would not be difficult at all if that happened. We didn't see the balls being loaded. And as we have seen, the "shuffling" was trivial at best.

Under this scenario, I would expect that when I call a number, I would draw that number or an adjacent number more frequently than numbers farther away because I would be reaching into the right general area. More importantly, I would also expect that 1-balls and 5-balls would be more likely to stay in place due to friction against the bag. The three remaining balls would probably shuffle around more. I haven't verified this through experiment, but it seems reasonable.

Thus, my prediction is that highest accuracy will be 1-balls and 5-balls. I also predict that when a 1-ball or 5-ball is called, I will be least likely to draw a ball from the opposite end. In other words I call a 1-ball and reach into the left side of the bag. Very few of the 5-balls on the right side of the bag will have migrated to the left due to poor shuffling. This means that when calling a 1-ball and reaching to the left, I would expect more 1-balls. I also expect the distribution of 2, 3, and 4-balls to show a slight bias in that order. And most importantly, I expect to get relatively few 5-balls. Of course, when calling 5-balls, I expect the same distribution in reverse. When calling the three balls in the middle, I would expect a slight bias for the adjacent balls.

I made this predictions without looking at the data. Let's see if I am right. Looking at the chart on page 17 of http://www.psych.uni-goettingen.de/home/ertel/ertel-dir/downloads/ertelchapterwithfigurespdf.pdf it looks like the results match my ad hoc prediction.







It looks like my predictions for the 1-ball and the 5-ball were spot on. The middle bias show a slight bias, but I don't think there's enough data to conclude either way.

 

[jasonpatterson]

It looks like my predictions for the 1-ball and the 5-ball were spot on. The middle bias show a slight bias, but I don't think there's enough data to conclude either way.

I'm not sure that that is what the data show at all. The scales on which you have graphed the data are deceptive to say the least. If you shrink any of the scales to the level that you have shrunk the 1 and 5 balls, they would look much more impressive, and the 'correct' choice would appear to have been chosen much more strongly. Basically, in each case the called ball was chosen about 400 times more often than the next higher ball, and all of the remaining 4 balls were chosen about equally often. You've graphed 2,3, and 4 on a scale that is 4000 high and 1 and 5 on a scale that is only 1000 high. Any differences would be visually magnified by 4 times. Sorry, this has always been a pet peeve of mine... Just looking at the data it would appear that people were most successful at calling the 2.

 

[Uncayimmy]

I'm not sure that that is what the data show at all. The scales on which you have graphed the data are deceptive to say the least. If you shrink any of the scales to the level that you have shrunk the 1 and 5 balls, they would look much more impressive, and the 'correct' choice would appear to have been chosen much more strongly. Basically, in each case the called ball was chosen about 400 times more often than the next higher ball, and all of the remaining 4 balls were chosen about equally often. You've graphed 2,3, and 4 on a scale that is 4000 high and 1 and 5 on a scale that is only 1000 high. Any differences would be visually magnified by 4 times. Sorry, this has always been a pet peeve of mine... Just looking at the data it would appear that people were most successful at calling the 2.

Mea culpa. It's a peeve of mine too, yet when I use Excel to make a chart, I always forget to check their automatic scaling. I will post revised charts in a little bit.

 

[Uncayimmy]

These are the revised charts with the scale corrrected along with a chart showing the percentage each ball was judged correct versus the percentage each ball was called. Also, I didn't make it clear that these numbers were for all the tests, not just the one in question. We have no idea what pattern, if any, that the kid in the video used.