Ball-drawing experiments-Discussion with Dr. Ertel

[steenkh]

It is interesting to read Linda's analysis of the experiments and the problems she has found with them. However...

Unfortunately I think what we will learn is it is no more effective than being polite for a while, then being blunt.

So there is really no advantage in stopping being polite?

 

[Ivor the Engineer]

So there is really no advantage in stopping being polite?

Depends if you've got other stuff to do. Being blunt tends to get to the inevitable conclusion faster than being unendingly polite.

 

[steenkh]

Depends if you've got other stuff to do. Being blunt tends to get to the inevitable conclusion faster than being unendingly polite.

If it is a question of time wasted, it is even faster not to engage in the discussion in the first place!

 

[fls]

Linda:
As I said before, it is not consistent. If it were a consistent representation of your idea, you would be comparing numbers which have distance 1 with numbers which have distance 2 or more for all cases.

Suitbert:
For each target I compared two misses with less distance from the target with two misses with more distance from the target. All four misses were thus equally considered for the five targets. This is consistent and balanced since all misses are considered.

Your definition may be called "inconsistent" insofar as you consider two misses with distance 1 for targets 2, 3, and 4, and only one miss with distance 1 for targets 1 and 5. In addition, you get three misses with distance "2 or more" for targets 1 and 5 and only two misses with distance 2 for targets 2, 3, and 4. Furthermore, for distances "2 or more" distance 2 occurs six times, distance 3 twice and distance 4 also twice. Finally, for an analysis with your definition you have to take different routes for calculating expectancies. This is confusing and objectionable. If you maintain your point, please tell me how you calculate expectancies. For my way of comparing two frequencies I take proportion .5 as expectancy for misses with less distance and .5 as expectancy for misses with more distance throughout for all five target cases. This is consistent, simple, transparent.

As you point out, the opportunity to miss and to miss at a greater distance
varies from number to number. It is inconvenient to provide a proper model
which takes this into consideration, so instead you chose a model which does
not take this into consideration but is convenient. It is consistent with
convenience, but it is not consistent with your own description of your
ideas about distance.

Linda:
"Secondary" is the term that you used. I cannot speak to the issue of whether you are now unhappy with your choice of name. I thought secondary was a reasonable choice on your part, but if you wish to use a different name, that is of course up to you.

Suitbert:
Please tell me where I used "secondary pattern" in my correspondence with you. I did NOT use the term "secondary pattern" or even "secondary" with some other noun.

You use the term "secondary effect" or "secondary psi-effect" six times in
the paper in which you discuss the "good misses" effect.

Linda:
The only parallel that I am drawing between my example and your study is that they both involve post-hoc analyses.

Suitbert:
The term "post-hoc analysis" with its negative connotation is used here, as often in criticisms of skeptics, to cast doubt on unwelcome results. I did not claim that I expected the "good miss" phenomenon at the outset. If I had, one might blame me for pretending to have expected it instead of discovering it by inspecting the data. But when I found the same pattern in several other data sets and I was prepared beforehand to find them again. I feel your attribution of "post hoc" to my analysis is another derogatory ploy.

I have two questions for you. If you had not found the "good miss"
phenomenon, would you have included that in your results? And, if the "good
misses" phenomenon had been absent, would you have considered that evidence that psi is absent, or rather would you have concluded that psi does not always miss in that manner?

Linda:
Can you provide the data where you replicate the effect?

Suitbert:
Yes, I can. Do you mean the original trial by trial data? Which format do you need? I would send them if you would really do an analysis and report the result, otherwise the extra time for that work might better be spared.

I misspoke. I meant whatever write-up you have available. It may be
interesting to look at the original data as well, but not if it requires
effort on your part.

Linda:
Also, your analysis of the effect of psi-power on "good misses" does not replicate the expected pattern. Instead, it shows that psi-power is unrelated to "good misses" (despite your post-hoc convoluted explanation).

Suitbert:
Again your derogatory "post-hoc"! All explanations of phenomena are "post hoc" because the phenomena always precede their explanations.

You provided a reasonable explanation of what you expected to see. When you didn't find what you expected to see, all of a sudden the explanation
changed to a convoluted pattern whereby psi-missing and psycho-analytic
defense mechanisms are brought into it. This was entirely post-hoc.

This shows up again in the write-up you sent me, where the data shows
patterns that are the opposite of those you expected or those you had
previously demonstrated. In that write-up you also show that this effect is
not found in some of your datasets. It looks like whatever you find, good,
bad, or indifferent, will be taken to be consistent with psi.

Linda:
According to that either "good misses" does not reflect psi, or "hit %" does not reflect psi. One or the other may have to be discarded.

Suitbert:
I do not understand your logic. A very good dart thrower might hit the centre of the target with concentric circles (12) very frequently, but when he misses 12, his dart will land on 11 more often than on 10, on 10 more often than on 9 etc. This is no case for "either or" separations, but for "as well as" inclusions.

And that is the logic you started with - it makes sense. But you discovered
that very good dart throwers don't land on 11 or 10 more often when they
miss 12 - they land all over the place. And very poor dart throwers, while
they can't hit the actual target, are better at hitting 11 or 10 than the
very good dart throwers. Sometimes. The more obvious conclusion is that
whatever it is that leads you to miss the target, it's different from dart
throwing accuracy. If there were a transparent shield in front of the
target, such that darts thrown directly at 11 or 10 were deflected away from
those numbers, while darts thrown further away were deflected towards those numbers, then the unexpected results would make sense. But the presence of a shield (i.e. a "secondary effect") doesn't really have anything to do with dart-throwing skill - i.e. you're not measuring the same thing.

Suitbert (earlier):
Regarding validity: You do not explain why you deem the pattern of drawn wrong numbers (almost-hits) not to be valid. I am entitled, I think, to maintain that the validity of this finding can hardly be overstated. How can this "pattern" of drawn wrong numbers be explained? Apparently, some "intelligent" and clearly "goal-oriented", but unconscious process becomes manifest.

Linda:
Most importantly, it lacks convergent validity - it fails to correlate with psi-power. It lacks criterion validity, mostly because it suffers from the same problem that all psi research suffers from - there is no independent measure of psi. It lacks construct validity - you haven't established how psi considers these categories (ordered, directional, symbolic, etc.). Simply put, all you have established is that in this particular set of data there is a pattern, but you haven't established that it is "psi" that explains the pattern.

Suitbert:
This reminds me of drumfire... with empty rounds, not dangerous.

Good misses does not lack "convergent validity" because good misses are correlated with hit percentage.

No they don't. In 2 of the 3 datasets you have provided information on,
they don't correlate, and in the 3rd, there is a correlation in the opposite
direction you predicted.

Good misses do correlate with psi-power, even though their psi power is less, otherwise they would not correlate with hit percentage.

The graph you presented does not show correlation with psi-power.

"Criterion validity"? This term does not apply here. The ball test does not assess good misses tendencies, good misses are an additional effect whose validity need not be based on other measured criteria just as, say, response sets affecting questionnaire results can be found without additional assessment.

Criterion validity in this case means that if we had a way of detecting
whether or not psi was present, we would be able to show that good misses
were present when psi was present and absent when psi was absent.

The terms "construct validity" and "categories" (ordered, directional, symbolic, etc.) remain unexplained.

Construct validity means that you have a theoretical/hypothetical basis by
which psi would miss in this particular manner - i.e. that psi considers
numbers as occurring in a particular order and that there is no direction to
that order. None of that has been established. Psi could treat them as
symbols (the claim of "good misses" for Zener symbols, for examples), it
could order them but there is a direction to that order, or it could order
them in a non-linear fashion.

I HAVE established that the good misses phenomenon is psi-related, because it is correlated with hits in the ball test. In addition, we find similar phenomena in earlier studies with the Zener symbol test, they were called "displacement effect" in the literature. Some symbols were more easily selected than other symbols when targets were missed. Displacements with numbers are easier to assess because "similarity" in this case is a matter of numerical difference, not of subjective judgment.

That psi is interested in a numerical difference is a subjective judgement
on your part. How do you know it's not something like even vs. odd numbers, straight vs. curvy lines, mirror symmetry vs. rotational symmetry vs. asymmetry, etc.?

Linda:
Here is the problem. You are analyzing an uncontrolled data set, which means that ordinary sensory leakage or fraud accounts for the hits. So why wouldn't ordinary sensory leakage or fraud, in addition to chance or other artefacts, account for this "good misses" pattern?

Suitbert:
Chance? No, the probability of chance has been calculated and must be considered, after several replications, as extremely low.

But in some of the datasets you have mentioned, the effect isn't replicated.
And I didn't mean *only* chance - I should have phrased it as "sensory
leakage and fraud and chance and other artefacts".

Artifact? This is merely a word in this context without any substance (can Linda construe any artifact here, I doubt it)

And example of an artefact would be some feature on the #3 balls which made it more likely that they would be drawn - someone accidentally added an extra set of #3 balls, they ran out of the regular balls and bought a new
batch which feel slightly different, a different marker was used which left
a different texture, etc. Since #3 draws were always a "good miss", an
excess of #3 draws would make it look like there was an excess of "good
misses".

Sensory leakage? If it occurred at all, it might cause mixing up graphically similar digits, not numerically adjacent numbers.

How do you know?

Fraud? A thief selects from a bunch of keys the key to the safe and, if he misses this key, he will not select preferably a key to a neighboring door.

How do you know?

Suitbert (earlier):
I did not yet check whether the good misses effect occurred in the 1234512345 data base also. The following Table shows the frequencies of good and bad misses for the five targets:

Good Bad Difference
---------------
1 634 614 20
2 652 601 51
3 615 610 5
4 626 586 40
5 598 617 -19
----------------
3125 3028 97
----------------

Good misses are more frequent than bad misses for numbers 1,2,3,and 4. The misses for number 5 are an exception, as they were in the main student sample.

Linda:
You are aware (I hope) that those slight differences in the frequency of #3 and in the "good misses" vs "other misses aren't statistically significant.

Suitbert:
Yes, I am aware of that. The point here is that the pattern of differences
does not differ significantly from the earlier pattern of other samples.

Linda:
It cannot be called a valid demonstration of psi just because you have failed to look for an explanation. It could equally be said to be a valid demonstration that invisible pink unicorns prefer to nudge the #3 ball into the ball-drawer's hand.

Suitbert:
This statement, if valid, would make all demonstrations of psi phenomena impossible because psi is defined, at present at least, as unexplainable phenomena of transfer of information and mental effectance on objects or living matter without motor and other tools.

Exactly. You have no positive definition of psi. The best you have is "I
can't really think of anything else". It's a "psi of the gaps" sort of
definition. The problem is that you haven't put any constraints on what the
gap can be filled with other than "has not yet been demonstrated to exist".

Linda

 

[Wowbagger]

So there is really no advantage in stopping being polite?

Depends if you've got other stuff to do. Being blunt tends to get to the inevitable conclusion faster than being unendingly polite.

It is quite possible to be both blunt and polite, at the same time. The two are not necessarily mutually exclusive.

Not that I claim to be good at doing that, yet. But, I have seen it done, and I am striving to improve my skills.

 

[fls]

What is the actual psi claim? That the person drawing the ball is able to predict what ball he will pull out of the bad?

Might we have also learned this: Given ample time to practice before being tested, people given control of the shuffling of a bag of balls and immediate feedback are able to locate a numbered ball of their choosing about 3% to 6% of the time?

Damien:
I wonder how often *the same balls* are drawn? I can't immediately think of any way to find out that wouldn't mark a ball and thus provide a possible cue for later draws, short of having a transponder inside each ball that triggers a (blinded) recorder when the ball is held, or taken outside the bag's opening, or the like. Alternatively, a fresh ball of the same number might be used as the replacement, although there's room for errors to creep in there. Better yet, as mentioned before, a large number of bags could be used in random order.

Suitbert:
I have no objection to using a larger number of bags, this I suggested myself to the German skeptics (GWUP) when they demanded more precaution. They were not satisfied with this precaution, they insisted that each ball should be used only once. The practical consequences were annoying for the participant, a skeptical assistant was running around busy with replacing the balls used, thus intimidating the innocent participant.

Instead of using possibly unnecessary precautions out of hand one should make sure, by appropriate experiments and/or data analysis, whether a precaution of using several bags successively is or is not necessary at all. An intent to merely calm prejudiced skeptics' down should NOT be regarded as sufficient grounds.

Several empirical attempts of mine did not confirm skeptical hypotheses:

1. The probability of hits for just drawn numbers, if called again after putting them back into the bag, is not increased compared with the probability of calling other numbers.

2. Hit rates of drawing assigned numbers 111111111111222222222222333333333333444444444444555555555555 that would facilitate an ordering of numbers in the bag, if such thing occurred, were not larger than drawing assigned numbers 123451234512345123451234512345123451234512345123451234512345 that hinders an ordering of numbers in the bag.

3. In one test conducted with my Ukrainian participants I had added alphabetical codes on each individual ball, the experimenter was instructed to also record the ball code, not only the drawn numbers. I counted the frequencies of draws in between for each repetition of individual ball draws and compared this with a computer simulation. It turned out that repetitions of draws of individual BALLS (not only numbers) were significantly delayed somewhat, for the three participants alike. This showed that the participants tended to put the just drawn balls deeper or farther away into the bag than was necessary for an objective ball randomization. The just drawn NUMBERS had no lesser chance to be drawn again, certainly because of the large number of balls used (50). If you would use, say, 250 balls and put just drawn balls on the bottom of the large bag containing 250 balls this would have an effect on ball repetition similar to using, say, five different bags with 50 balls successively.

This is an interesting discovery. This would have an effect similar to drawing balls without replacement and providing feedback after every draw - a strategy which increases the probability of correctly guessing the next ball as it changes the probabilities from combinations to permutations.

4. Since recognizing drawn balls and a helpful ordering of them in the bag would be a case of learning, hit rates should increase during runs of 60 trials. But they do not increase, neither low hitters nor high hitters showed an indication of a learning curve.

5. As I said earlier, hit rates with using "psi-pods", instead of ping pong balls, where numbers written on paper are put inside the pods and covered by a lid, were not smaller than hit rates with using balls.

Do you have a write-up on this data that you can send me? It sounds promising.

Using five different bags successively would certainly exclude an ordering the numbers in the bag subconsciously and exploiting cues of temperature and "unicorn stimulation". But you would never know whether this was necessary at all, and you would risk arousing psi-inhibiting psychological reactions.

It seems like it would be valuable to figure out how to make this a fairly psi-neutral procedure, as it would help relieve several concerns. It seems like the helpful passing of a fresh bag shouldn't be an anxiety-producing situation, especially if you presented it as the testing of additional psi-conducive ideas, rather than as an anti-sensory-leakage precaution. For example, you could tell the participants that it has been found that treating the bags with specific frequencies of electro-magnetic radiation is psi-conducive and that you are testing bags which have been treated with different frequencies.

Precaution by avoidance at any event is comparable to excessive irrational avoidance of patients suffering from chronic anxiety. They will never learn to know that their avoidance behavior is needless. My research may be compared with desensibilisation therapy, an adaptive benefit is unquestionable.

That's why I think that some attention should simply be paid to providing a non-anxiety-producing explanation for the various controls so there is no perception that psi will be inhibited. I do agree that interruptions should be avoided.

Linda

 

[Beth]

Sensory leakage? If it occurred at all, it might cause mixing up graphically similar digits, not numerically adjacent numbers.

How do you know?

I've been trying to follow along with this discussion, but haven't had time to participate. I think Linda has done an excellent job of bring up potential problems and I haven't thought of anything she hasn't already mentioned.

However, I would like to mention that I think this is going a bit too far. If sensory leakage occurs, it's going to be graphically similar digits getting confused because that's how they might have perceived the numbers with either vision or touch. Hearing, smelling or tasting seem an unlikely route for sensory leakage in this situation. If they are perceiving the number with some sense that isn't graphically based, why wouldn't that qualify as the psi-effect we're looking for?

 

[fls]

Linda:
Is it really reasonable to perform one-tailed testing when you clearly have a two-tailed hypothesis? You try to claim that you are only interested in
excess when it suits you (i.e. it enables you to claim "significant"
results), yet you take full advantage of two-tailed differences in your
explanations and in your use of Chi-square and Z-square statistics.

Suitbert:
This question does not have one simple answer. For my Ükrainian participants, I have to consider previous results. In eight years of psi test
participation, after conducting over 50 different tests with each one, I did not observe any negative deviation from chance expectancy. So I cannot expect negative deviations in any further such test. Hence, a one-tailed test is appropriate whenever a choice between one- and two-tailed tests is given. For Chi2 and Z2 tests, no such choice exists.

But you don't limit your one-tailed testing to your Ukrainian participants. You perform this testing on other datasets which do contain negative deviations. This issue is critical when attempting to study psi, because it creates a situation which increases the number of false-positive tests. And when the true-positive rate is probably very low, it aggravates the problem that any "significant" results are more likely to be false than true. If anything, the standard for statistical testing should be held higher, rather than lower than for other research, in order to increase your chances of finding true relationships.

See this article for an explanation of the problem.
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124

Chi-square and z-square tests take into account deviations in both directions. It is your *choice* of those particular tests/measures which dictates the inclusion of both tails.

Suitbert:
Yes, this psi test online does a poor job, and I assume that all psi tests online do a poor job. Why this is so? My tentative answer is it’s the amount of technological conditions, lack of naturalness, lack of visual and motor activity. I quote Damien: “Yes, this intuitively appeals me from my analysis of the likely evolved competences relevant to psi--that it's an adjunct to on-going sensorimotor processes of being-in-the-world.”

Linda:
How can you regard this as convergent? The effect size is so different, it
would be very difficult to persuade anyone that you are measuring the same
thing. It is, however, consistent with the IPU idea, since one would expect
that they wouldn't operate under these conditions.

Suitbert:
A low effect size of test X does not make the validity of test X different from test Y which has a larger effect size. Ski jumpers may show different jump distances depending on good or bad jump conditions.

I agree, but does that really apply here? You are claiming convergence - people who do well in one test do similarly well in another. But without testing the non-gifted ball-drawers at the same time for comparison, you don't know if this relationship holds.

Suitbert:
IPU? An acronym for what?

Sorry - Invisible Pink Unicorn

Suitbert:
In this case I mean “controlled” because all tests of the three Kiev participants were done under the supervision of experimenter Tanya. She was also her own experimenter. I myself did many experiments myself, so I was my own experimenter. In the parapsychological literature many cases of self-administered tests have been reported. Even Röntgen’s discovery of X-rays was the result of a test that he conducted on himself.

What do you think would have happened if it was only ever Rontgen's self-tests which demonstrated X-rays? Are you familiar with the story of N-rays? (http://en.wikipedia.org/wiki/N_ray) It isn't a matter of whether or not self-tests can serve as a starting point, it's a matter of whether or not it can ever progress beyond that point.

The question is why is the effect size much lower in this online psi-test than in the ball test? Since you are an expert in unicorns and prepared, I think, to admire achievements of the fraudulent subspecies of these invisible animals in particular, you might have an answer to this question. I don't have experience with unicorns and would rather conclude, tentatively, that the natural manual activity which the ball test requires is the main cause for larger hit percentages in the ball test.

Linda:
The most obvious conclusion would be that you are measuring two different things.

Suitbert:
I am reluctant to acknowledge two independent psi powers.

You don't have to. Since you have provided no constraints, other than "unexplained", this effect can be due to Hyperdimensional Dragons - another entity which is invoked to 'explain' the unexplained.

Linda:
One difference is that your ball-drawing test increases the amount of sensory information available.

Suitbert:
Yes, but I showed in various tests that the ball test condition does not provide USEFUL sensory information. Psi effects might be facilitated when many sensory channels are used with no avail. Psi might have a “compensatory” function in the first place. Who knows.

How do you know it does not provide USEFUL sensory information? I already referred to research which shows that when we are asked to explicitly describe our reasoning, we perform poorly compared to allowing for implicit processes. You tested for explicit reasoning - the kind of reasoning that we DO NOT think is taking place when determining whether or not a ball feels 'familiar'.

Linda:
Other than "increase the opportunities for sensory input", what other
psi-conducive conditions would you consider?

Suitbert:
As I just said, an increase of sensory input without useful information might be psi-conducive.

How would you go about achieving that state?

Suitbert:
Social factors have also been found to be either psi-conducive or psi-inhibitory. Remember my experiments with students wíth significant hit surplus at home whose hit scores were either lowered by my presence as experimenter or were considerably increased. This difference of social influence on psi processes I cannot yet explain.

How would you go about distinguishing this from chance a priori, as you would expect performance to vary due to chance as well?

Linda

 

[fls]

Linda:
For example, you could tell the participants that it has been found that treating the bags with specific frequencies of electro-magnetic radiation is psi-conducive and that you are testing bags which have been treated with different frequencies.

Damien:
Terrible idea. If psi is real, there is always an enhanced
possibility of what might loosely be called empathic entanglement
between ostensible Subject or Participant and ostensible
Experimenter. There is ample evidence of experimenter effects that
cannot be explained away as better scrutiny on the part of those who
get null results. Lying would seem to me an especially corrosive
basis for a relationship that needs to open and trustworthy on both
sides. I realizes this flies in the face of a century of deceitful
psych and medical protocols...

One doesn't even need to invoke "empathic entanglement" in order to know that lying doesn't work - what is the point of blinding, after all? I'm not suggesting that you need to lie. Simply make whatever it is that you use to implement a psi-conducive control true.

Linda

 

[fls]

Linda:
This is an interesting discovery. This would have an effect similar to drawing balls without replacement and providing feedback after every draw - a strategy which increases the probability of correctly guessing the next ball as it changes the probabilities from combinations to permutations.

Suitbert:
You are right, in principle, but the effect is much smaller than in studies without replacement, and for such studies it is already small. Even if you would make a just drawn number inaccessible and even if you would instruct the participant not to call the just drawn number, the expectancies for the four remaining numbers would be 10/49 (0.204) instead of 10/50 (0.20). For one total test series consisting of 360 trials the number of hits would be 73.4 instead of 72, i.e. 1.4 surplus hits by artifact – under unrealistically favorable conditions!

This is an advantage that would accumulate. Sure, after the first draw it would be 0.204, but after the 6th draw it might be 0.227, which is higher than your measured hit rates.

Suitbert:
I have not written up the results, I just received the first data from Kiev. I tabulate the observed hit rates from the ball test (conducted in 2003) and the psi-pods test (conducted in 2009). The total number of trials for each participant was 480 for the psi-pods test and 960 for the ball test. Expected hit rate .20.

Balls Pods Diff

Galina .385 .392 +.007

Tanya .310 .267 -.043

Vanya .458 .360 -.098

Galina’s pod results do hardly differ from her ball results, the pod results of Tanya and Vanya are lower, but still highly significant.

Were these experiments controlled or uncontrolled?

I keep forgetting to ask you...how did the 3 Ukrainian stars come to your attention?

Linda:
It seems like it would be valuable to figure out how to make this a fairly psi-neutral procedure, as it would help relieve several concerns. It seems like the helpful passing of a fresh bag shouldn't be an anxiety-producing situation, especially if you presented it as the testing of additional psi-conducive ideas, rather than as an anti-sensory-leakage precaution. For example, you could tell the participants that it has been found that treating the bags with specific frequencies of electro-magnetic radiation is psi-conducive and that you are testing bags which have been treated with different frequencies.

Suitbert:
I never lie in my experiments (see Damien’s comment), but changing bags would probably not have a strong negative effect on participants. One might ask participants to change bags, for each trial she should make a decision among four (?) bags so that she maintains some control, subjectively. One might also tell her that the bags should be selected equally often.

I agree that it's a good idea to not lie. One would not need to lie in order to implement my suggestion.

Letting the participants select the bags would dissipate the effect of the control. Why is it necessary to allow the participant the opportunity to gain an ordinary advantage in order for them to demonstrate what you are calling an unexplainable advantage?

Linda

 

[fls]

Suitbert:
Yes, I said „secondary effect“, or „secondary psi-effect”. When you speak of “secondary pattern” that the experimenter “picks” then you raise the idea that what is being analysed here is dependent on arbitrary decisions.

But isn't it? You dismiss that your model is inconsistent, you admit that only those tested relationships that are positive get any mention, and that any result (good, bad or indifferent) would be considered to be consistent with psi.

Linda:
I have two questions for you. If you had not found the "good miss"
phenomenon, would you have included that in your results?

Suitbert:
Of course not, I would not have searched it.

Testing multiple relationships and reporting only on those which are positive falsely increases the number apparent relationships. It makes associations discovered in this manner unreliable.

Linda:
And, if the "good misses" phenomenon had been absent, would you have considered that evidence that psi is absent, or rather would you have concluded that psi does not miss in that manner?

Suitbert:
My conclusion that psi is present is based on hit scores, not on miss scores. I would not have missed a secondary psi effect.

If any answer (good, bad or indifferent) about the presence of secondary effects leaves your conclusions about the presence of psi unchanged, how can a secondary psi effect be used as evidence for psi?

Linda:
You provided a reasonable explanation of what you expected to see. When you didn't find what you expected to see, all of a sudden the explanation changed to a convoluted pattern whereby psi-missing and psycho-analytic defense mechanisms are brought into it. This was entirely post-hoc.

This shows up again in the write-up you sent me, where the data shows patterns that are the opposite of those you expected or those you had previously demonstrated. In that write-up you also show that this effect is not found in some of your datasets. It looks like whatever you find, good, bad, or indifferent, will be taken to be consistent with psi.

Suitbert:
It is quite sensible to try to explain phenomena by using previous experience and existing knowledge, the phenomena should be consistent with one’s knowledge. That’s what you and other skeptics do as you try to explain results of parapsychological research by sensory leakage or fraud or artifact.

I myself am trying to interpret the observations of my research within a system of knowledge in which psi plays a role. So it is sensible to seek consistency with this knowledge.

Can you give me an example of something which would be inconsistent with this idea?

You are right, the results with bad misses do not replicate as I thought they would and a correlation had even a reverse direction. Anyway, there was a correlation which demands an explanation. I don’t think YOU can provide an explanation here. So why don’t you let others figure something out.

Can you give me an example of a situation where psi cannot be the explanation?

Suitbert:
The main point is to acknowledge that the good misses phenomenon, if it exists, shows that it is based on some rational tendency or intention, without being guided by the participants’ conscious intention.

Even if that were acknowledged, it doesn't tell you that "good misses" and psi are related.

Linda:
No they don't. In 2 of the 3 datasets you have provided information on, they don't correlate, and in the 3rd, there is a correlation in the opposite direction you predicted.

Suitbert:
There are correlations that should be explained. My explanation is tentative. Tentative explanations of observed phenomena are common in scientific discourse.

I was simply pointing out that your explanation didn't explain the observed results.

Linda:
The graph you presented does not show correlation with psi-power.

Suitbert:
The graph shows a correlation between double-hit count and good misses.

I was referring to your analysis of psi-power and good misses described in this paper:

http://www.psych.uni-goettingen.de/home/ertel/ertel-dir/downloads/ertelchapterwithfigurespdf.pdf

Suitbert:
Double hit count I regard as a measure of psi ability. You don’t?

What is the gold-standard measure of psi ability against which you measure double hit count? For example, if I want to regard spiral CT as a valid measure of the presence of pulmonary emboli, I measure it against pulmonary angiogram.

Suitbert:
What is, in your view, the validity of the double hit count? The significance tests show that you cannot take this variable as a random variable. So what are the crucial factors giving rise to surplus hit rates in the ball test?

If you wish for me to consider it a measure of psi, then you should be able to point me to research where it was compared to independent tests of psi.

Linda:
Criterion validity in this case means that if we had a way of detecting
whether or not psi was present, we would be able to show that good misses
were present when psi was present and absent when psi was absent.

Suitbert:
I consider the correlation (.40) shown in Figure 1 as an indication of criterion validity in your sense.

That is simply a correlation between double hit count and good misses. Are you suggesting that every time a double hit is obtained it indicates the presence of psi?

Linda:
Construct validity means that you have a theoretical/hypothetical basis by
which psi would miss in this particular manner - i.e. that psi considers
numbers as occurring in a particular order and that there is no direction to
that order. None of that has been established. Psi could treat them as
symbols (the claim of "good misses" for Zener symbols, for examples), it
could order them but there is a direction to that order, or it could order
them in a non-linear fashion.

Suitbert:
Numerical distance is the theoretical basis by which psi would prefer misses in this particular manner.

What is the name of the theory which establishes the basis by which psi misses in this particular manner?

Linda:
That psi is interested in a numerical difference is a subjective judgement
on your part. How do you know it's not something like even vs. odd numbers, straight vs. curvy lines, mirror symmetry vs. rotational symmetry vs. asymmetry, etc.?

Suitbert:
These factors are irrelevant because they cannot give rise to the pattern that was found. Even if any of your alternative factors was effective, this would have to be explained. Do you have an explanation for an effect by even vs. odd numbers, straight vs. curvy lines, mirror symmetry vs. rotational symmetry?

The question isn't which explanation is consistent with the pattern. The question is what happens in the presence of psi?

Linda:
And example of an artefact would be some feature on the #3 balls which made it more likely that they would be drawn - someone accidentally added an extra set of #3 balls, they ran out of the regular balls and bought a new batch which feel slightly different, a different marker was used which left a different texture, etc. Since #3 draws were always a "good miss", an excess of #3 draws would make it look like there was an excess of "good misses".

Suitbert:
This I consider as an example for skeptical strategies in general: Inventing post hoc alternative conditions without any evidence and even without showing any obligation to provide plausible indications for their existence.

This wasn't meant as a possible explanation for the results. I was simply providing you with an example of an artefact, as you indicated that you did not understand what I meant.

Suitbert:
Sensory leakage? If it occurred at all, it might cause mixing up graphically similar digits, not numerically adjacent numbers.

Linda:
How do you know?

Suitbert:
Because numerical distance has nothing to do with graphical similarity.

How do you know sensory leakage mixes up graphically similar digits? This should be something which is determined empirically. For example, in memory tests, people mix up words which sound similar or they mix up words which have similar meaning, depending upon whether you are talking about short or long-term memory.

Linda:
How do you know?

Suitbert:
Because this is an either laughable idea or a pathological case. The thief avoiding the safe in favor of a neighboring door might have to be transferred to a psychiatric hospital.

How do you know it isn't simply your choice of analogy which is laughable or pathological?

Suitbert:
Lack of explanation of a phenomenon does not diminish available evidence of its existence.

But that's entirely irrelevant, isn't it? The issue isn't whether or not these observations exist, but whether any particular idea serves as a roughly necessary, sufficient and useful explanation for those observations. An explanation which doesn't constrain the results, i.e. any result will be taken as consistent with the idea, is not useful. Contrast this with the idea of Gravity which explains only a very specific set of results. If we observed a different set of results, then Gravity would need to be changed or discarded.

Linda

 

[fls]

Linda:
This is an advantage that would accumulate. Sure, after the first draw it would be 0.204, but after the 6th draw it might be 0.227, which is higher than your measured hit rates.

Suitbert:
Your calculation is wrong since you presuppose what is improbable with astronomical degrees that the participant would draw the same number six times successively, with decreasing numbers of balls carrying number 6, in any event.

After six draws it is easily possible that one of the numbers has not yet been drawn. The probability for that number would then be 10/44, or 0.227.

Linda:
Were these experiments controlled or uncontrolled?

I keep forgetting to ask you...how did the 3 Ukrainian stars come to your attention?

Suitbert:
Tanya (Tatyana) Kolesnik, one of the three members of the Ukrainian family, was the experimenter. She obtained a degree in informatics, I learned to know her and her family during a stay in Kiev preparing for a scientific conference in Ukraine. I myself and independent skeptical experimenters tested her own and the other family’s psi powers during a visit of the family in Germany.

It seems that this method was more successful at identifying psi stars than your ball-drawing at home method? How did they come to realize their own psi abilities?

Suitbert:
“Dissipate the effect of the control” and “gain ordinary advantage” are ways of opposing to a prolific concrete idea by abstract semantics. As long as you do not spell out what you mean by “effect of control” and a “gain of ordinary advantage” I can hardly consider this seeming objection as worth any reply.

"Effect of control" would be the use of methods which reduce bias, provide an empirical basis for comparisons, and/or reduce the chance of intentional manipulation. Letting the participants select the bags can re-introduce a pattern to the selections and can allow for them to take advantage of sensory leakage - something which the controls are meant to reduce or eliminate.

"Gain of ordinary advantage" would be the gaining of information or knowledge (conscious or not) through ordinary means - vision, touch, hearing, memory, etc.

Linda

 

[Uncayimmy]

Linda forwarded one of my posts to Suitbert, who requested this be posted on the forum.

"UncaYimmy:
If I wanted to cheat on that test, I would attempt to get the balls
loaded into the bag so that the numbers were grouped. By that I mean
the 1-balls would be in the left corner, the 2-balls next to them and
so forth with the 5-balls being on the right hand side. When
preparing the experiment, it would only be natural to group the balls
by number first to make sure you have the right counts. Putting the
balls in the bag as I describe would not be difficult at all if that
happened. We didn't see the balls being loaded. And as we have seen,
the "shuffling" was trivial at best.

Suitbert replies:
UncaYimmy's ball ordering idea would indeed explain the "good misses"
effect, if such ball ordering in the bag would take place. A priori this is
very improbable, shuffling effects are NOT trivial. At each trial the bags
must be turned around like a pancake when its sides on the pan are changed.
This can hardly be done with keeping the balls in place. Supposing the balls
would not get mixed, then the participant must always remember the changing
positions of 1 and 5 in the bag because they are reversed at each trial.

Another point is that at the beginning of each test series the numbers are
not in numerical order, UncaYimmy's ordering of 50 balls in the bag needs
time. An increase of hit numbers would be observable only towards middle or
end of one test session. But my analysis showed that individual hit rate
levels do not increase within test sessions. Hit probabilities of the last
10 trials of one run of 60 trials are not larger than for the first 10
trials.

We disagree on how effective the shuffling is. I can't see how flipping the bag like a pancake will do any shuffling at all considering that centrifugal force and friction will keep the balls from moving much. As for remembering which side is which, that's not hard at all.

But in the video in the OP, the subject never flips the bag like a pancake. He picks up the bag by the top. He shakes it up and down. There's no room for much shuffling. Had this been a box with lots of room for the balls to bounce around, I would consider that adequate shuffling. The sides of the bag keeps the balls in place. I find it hard to imagine that a ball would migrate from bottom to top or vice-versa.

As for the order of the balls in the bag at the start, I didn't see that happen. My conjecture is that the experimenter will arrange the balls in order before the test to make sure none are missing. It seems only prudent to ensure there is a full set. Perhaps the experimenter only counted to ensure there were 50 balls, which would presume that the balls were always kept secure from tampering.

UncaYimmy continues:
"I would also expect that 1-balls and 5-balls would be more likely to stay
in place due to friction against
the bag. The three remaining balls would probably shuffle around more. I
haven't verified this through experiment, but it seems reasonable. Thus, my
prediction is that highest accuracy will be 1-balls and
5-balls."

Suitbert's reply:
Thus, UncaYimmy predicts more hits when 1 or 5 is called compared with hits
when 2, 3, and 4 are called. This is not what occurs. Here are the
percentages of hits for 1 through 5 (from my standard sample of 238
participants):
1: 22.7% hits of 15,370 calls
2: 23.2% hits of 19,613 calls
3: 22.7% hits of 19,156 calls
4: 23.2% hits of 17,008 calls
5: 22.4% hits of 14,053 calls

I made a mistake in my spreadsheet. His numbers are correct. Mine are incorrect. I apologize for the mistake. Below are the correct numbers:

Ball|Correct|Called|Drawn
1|22.7|18.0|19.7
2|23.2|23.0|20.6
3|22.7|22.5|20.2
4|23.1|20.0|19.9
5|22.4|16.5|19.6

However, it's important to note that we are looking at aggregate numbers, not just the subjects with high scores, which is what my theory is based on. I suggested a possible way of cheating based with *one* particular way of loading the balls into the bag. A subject looking to cheat would not have to put the balls in the bag in the order 1 through 5. They could be placed in any order so long as decent semblance of grouping remained.

The "trick" is that if the balls are loaded with any type of grouping, the subject need only remember that grouping and call/draw accordingly. The shuffling appears woefully inadequate, and it hasn't been demonstrated to be effective. It is only assumed.

* How are the balls determined to be a full set before testing?
* How are the balls shuffled before being placed into the bag?
* Can the subject see the balls as they are being loaded into the bag?
* Did the subject ever load the bag?
* What happens between runs? Are the balls removed from the bag?
* Is this information recorded with each trial? If so, then it might be possible to see a correlation between these actions and the accuracy.

UncaYimmy also predicts: "I also predict that when a 1-ball or 5-ball is
called, I will be least likely to draw a ball from the opposite end. In
other words I call a 1-ball and reach into the left side of the bag. Very
few of the 5-balls on the right side of the bag will have migrated to the
left due to poor shuffling. This means that when calling a 1-ball and
reaching to the left, I would expect more 1-balls. ... And most
importantly, I expect to get relatively few 5-balls. Of course, when calling
5-balls, I expect the same distribution in reverse."

Suitbert: The results do not confirm UncaYimmy's idea who expected more
number 2, 3, and 4 than number 5 draws when number 1 is called and more
number 4, 3, and 2 than number 1 draws when number 5 is called.

In my post with the charts I said that it didn't appear to be statistically significant. Again, we are looking at aggregate numbers, not the numbers for high scorers. And my predictions presumed just one order in which they could be loaded. Any order would work.

UncaYimmy's explanation of the good misses effect thus does not hold. Can
anyone out there propose another explanation? I like tentative explanations
that can be tested empirically. Mere speculation is less productive.

My speculation error was compounded by errors in my spreadsheet that coincidentally reinforced my theory. Mea culpa.

Looking at the numbers, it appears there is consistent accuracy across all 5 balls. By my reckoning the effect size was 3% to 4% accuracy. The only thing unusual is that the 1 and 5 balls were called much less frequently than balls 2 and 3. Ball 4 was called about 20% of the time. Then again, humans are not known for being good random number generators.

All that said, there is insufficient data available to me (as far as I know) to form any theories or even to reject my theory about ball placement. What we're looking at is all of the data at once. What I want to see is the data for the people with effect sizes greater than the average.

For example, in the video the subject got 23 out of 60. That effect size is around 20% to 25%. That subject was not shuffling the balls adequately at all - he merely bounced the bag up and down a couple of times. I would want to see the results of his performance and those with bigger effect sizes in order to form a theory. Not only that, I'd like to see the actual data recorded (trial #, called, drawn).

Is that data available?

In the one video I saw with the inadequate shuffling, it looks to me like the subject is trying to peer into the bag. See the animated GIF below. If he only got a glimpse on occasion, that could explain the effect size.

 

[fls]

The pause in our discussion was entirely my fault. To continue:

Linda:
I have two questions for you. If you had not found the "good miss"
phenomenon, would you have included that in your results?

Suitbert:
Of course not, I would not have searched it. Why do you ask? You know the answer in advance.

Linda:
Testing multiple relationships and reporting only on those which are positive falsely increases the number of apparent relationships. It makes associations discovered in this manner unreliable.

Suitbert: You imply that I do that. Is it fine to put one’s criticism in a seemingly harmless dressing ? I do report missing relationships , whenever I expected them, above all relationships which come out with an unexpected direction, as you know.

You mention 5 'secondary psi effects' including the preference paradox (higher hit proportions with less preferred calls), deviant succession of drawn numbers (repetition avoidance), deviant sequences of hits and misses, etc. But you don’t report on all of them. For example, the data that you perform your 'good misses' analysis on does not demonstrate the preference paradox, but you do not mention that.

Linda:
You provided a reasonable explanation of what you expected to see. When you didn't find what you expected to see, all of a sudden the explanation changed to a convoluted pattern whereby psi-missing and psycho-analytic
defense mechanisms are brought into it. This was entirely post-hoc.

Suitbert:
„all of a sudden“…? Why do you apply such rhetorical spices?

I am trying to communicate the seriousness of my concerns. Sometimes the use of rhetorical devices helps to convey mood.

Linda:
What is the gold-standard measure of psi ability against which you measure double hit count? For example, if I want to regard spiral CT as a valid measure of the presence of pulmonary emboli, I measure it against pulmonary angiogram.

That isn’t a good example. I used a diagnostic test because the example was more clear, but then it became less relevant. An example that is more relevant is 'health'. One can compare something like "self-perceived health status" with the mortality rate to determine the validity of that measurement - the two measures are otherwise quite independent. If you need an example from behavioural sciences, then it would be something like 'intelligence'.

What I am looking for are things which are determined a priori - a particular measurement or value of a variable predicts the presence or value of another variable. The former is an independent variable and the latter is dependent. I think that what you are saying is that a prior deviation from MCE predicts a subsequent deviation from MCE? In that case, what I am interested in is the data which demonstrates this association - a positive predictive value or likelihood ratio or a Cronbach alpha (it seems that reliability speaks to validity in this case).

Linda:
Criterion validity in this case means that if we had a way of detecting whether or not psi was present, we would be able to show that good misses were present when psi was present and absent when psi was absent.

Suitbert:
I consider the correlation (.40) shown in Figure 1 as an indication of criterion validity in your sense.

Linda:
That is simply a correlation between double hit count and good misses. Are you suggesting that every time a double hit is obtained it indicates the presence of psi?

Suitbert: Why “every time”? You know that this is not the case. We are dealing with hit rates, not with indicidual hits. Why do you ask such question. Do you want me to say “no” to your question which sounds as if I had to acknowledge the truth of some of your criticisms?

I say this to make it clear that your approach does not demonstrate criterion validity. You use the correlation as though the presence of a double hit indicates the presence of psi. But you have not established the reliability and validity of hit rates as a measure of psi.

Ideally, if you wished to use hit rates as a measure of psi, you would set about establishing the scale. Under different circumstances, one could form ROC (receiver operating characteristics) curves for different hit rates using a direct measure. But we have no direct measure of psi. However, I think your idea of using subsequent deviations from chance as a way to confirm your choice is a reasonable place to start. As I mentioned earlier, the idea of reliability overlaps considerably with validity, when it comes to psi.

I don't get the impression that you have developed a scale to measure psi based on the test-retest reliability of various hit rates. If you have, then it is that scale which would form the basis for your criterion validity when assessing the validity of 'good misses'. By simply comparing 'good misses' to hits, you are probably comparing the two over a range which has no potential to discriminate. I would suspect that you would discover that something like a deviation of 10% from chance has a poor test-retest reliability, whereas a deviation of 50% has a moderate amount of reliability

Linda:
Construct validity means that you have a theoretical/hypothetical basis by which psi would miss in this particular manner - i.e. that psi considers numbers as occurring in a particular order and that there is no direction to that order. None of that has been established. Psi could treat them as symbols (the claim of "good misses" for Zener symbols, for examples), it could order them but there is a direction to that order, or it could order them in a non-linear fashion.

Suitbert:
This could be, yes. Am I responsible that we do not observe such things? Do I have to deal with missing results that no one expected except Linda post hoc who does not like the real observation that she, as I myself, had not expected.

I ask you these questions because you have simply made an observation – that people sometimes miss in a particular pattern and sometimes do not. You haven’t provided any particular hypothetical or theoretical reason to connect that to psi. There are other patterns in the data. For example, numbers 1 and 5 are called and drawn less often than the other numbers. If I hypothesize that Invisible Pink Unicorns consider odd numbers unlucky, does the relative paucity of 1’s and 5’s confirm my idea that IPU’s are responsible for the results?

Linda:
That psi is interested in a numerical difference is a subjective judgement on your part. How do you know it's not something like even vs. odd numbers, straight vs. curvy lines, mirror symmetry vs. rotational symmetry vs. asymmetry, etc.?

Suitbert:
These factors are irrelevant because they cannot give rise to the pattern that was found. Even if any of your alternative factors was effective, this would have to be explained. Do you have an explanation for an effect by even vs. odd numbers, straight vs. curvy lines, mirror symmetry vs. rotational symmetry?

Linda:
The question isn't which explanation is consistent with the pattern. The question is what happens in the presence of psi?

Suitbert:
You replace my concrete question with some deteriorating abstract question.

Is that not the question though? You shouldn’t be wondering whether you can design experiments which easily generate patterns. You should be wondering whether you can design experiments which pick up on psi.

Linda:
How do you know sensory leakage mixes up graphically similar digits? This should be something which is determined empirically. For example, in memory tests, people mix up words which sound similar or they mix up words which have similar meaning, depending upon whether you are talking about short or long-term memory.

Suitbert:
If the ball drawing process were supported by sensory leakage sensory discriminations were taking place, not memory processes.

Again, this was just an example of the type of information or research that is available to answer a different question, to help clarify what kind of information I am asking for from you.

Linda