Natasha Demkina got a test, why can't I?
We might learn something from Natasha Demkina's test of a claim similar to mine. Her claim of having X-ray vision to see inside people's bodies and describe health problems was tested by CSICOP in May 2004. I apologize to the authors for liberal copying verbatim but I find that as a claimant anything I write in my own words is questioned.
Excerpts from: Natasha Demkina The Girl with Normal Eyes
http://www.csicop.org/specialarticles/demkina.html
by Andrew Skolnick
Seven volunteers were used for the test.
We wanted a test that would prevent Natasha from making diagnoses that could not be disproved
One or two kidneys can be disproved. At my expense of an ultrasound.
We wanted to conduct a blinded test. Natasha claims to see through people's clothing, yet she says she cannot see through a fabric screen, which we wanted to use to prevent her from seeing the test subjects.
I hear you.
We found this unexplained contradiction curious-and frustrating:
My Skeptics hear you.
Any test she would agree to would have to allow her to study the test subjects using her normal senses. As Hyman points out, this study flaw alone could provide an astute person powerful clues about a person's health problems.
About number of kidneys?
It would be less of a problem if we were able to recruit subjects who were physically and demographically similar. Unfortunately, we had great difficulty recruiting subjects for the test and had to settle for several people with characteristics that suggested their target conditions.
Lucky for us, physical and demographic similarity is not obvious just from looking at their clothed backs. And no one really has characteristics that suggest the number of kidneys. Provided we use healthy kidney donors and not persons who have had to have a kidney removed due to illness. The reason people donate a kidney is due to a choice, and past choices can not be discerned just by looking at a person.
it took her more than four hours to complete the test and, inexplicably, she took an hour to examine the seven subjects before deciding which one was missing a large part of her left lung! She guessed that one correctly, but why would anyone who claims to be able to see "every cell" inside a person take an hour to decide which person was missing a large portion of one lung?
Especially when one could receive clues just by looking at people face-on and at how they are breathing. Luckily, body movements such as breathing do not reveal how many kidneys they have.
Natasha matched only four of the conditions correctly-a score that everyone prior to the test had agreed would not justify further testing. Natasha's most dramatic misdiagnosis was her failure to see a large metal plate covering a missing section of skull in a man who had a large brain tumor removed. Instead, she indicated that she "saw" a metal plate and missing skull section in a man who had his appendix removed but a normal skull.
_____________________________________________________
Excerpts from: Testing Natasha
http://www.csicop.org/specialarticles/natasha.html
by Ray Hyman
Natasha can see through clothing, but not see what someone is holding behind their back. She cannot see inside people if she shuts her eyes. Daylight is better. She does not need to talk to them to diagnose.
Same as the conditions of my claim.
She says that she can examine the whole body, but it can give her a bad headache if she does too much.
Same as the conditions of my claim.
The evidence supporting Natasha's abilities comes from selected anecdotes of reactions to her readings.
Mine are just anecdotes and not other people's reactions to my readings.
No matter how subjectively compelling, the context of such readings makes it impossible to separate how much of the apparent success is due to such possibilities as: guessing; external clues from the client's physical appearance and observable behavior; feedback from the client's spoken and bodily reactions; or actual paranormal powers.
Same problem with my anecdotes.
Although psychological research has documented how humans frequently provide unconscious clues to their current thoughts and emotions, most people seem unaware of this possibility. The research also shows that subtle clues can influence us without our consciously realizing it.
We made sure to include in the protocol the statement that the "test is not in any way a definitive test. Deciding the truth of Natasha's claims with comfortable certainty is too simple and brief. It can only help to decide whether further studies of Natasha's claimed abilities are warranted."
And I would like such a preliminary test for myself too.
Even under ideal circumstances this test could not clearly decide if Natasha does or does not have X-ray vision. Any scientific hypothesis-especially a paranormal one-cannot be confirmed or disconfirmed by one test or one experiment. Scientific investigation requires a series of experiments. Each new experiment builds on the results of previous ones. The more we learn from the early experiments, the better we can understand what we need to control and what we can safely ignore. If the hypothesis is implausible and/or controversial-as Natasha's claim certainly is-then the original investigators must replicate their findings. In addition, independent investigators must also replicate the findings before they gain scientific credibility.
We knew that our test could not distinguish between two possibilities: (1) she can make correct matches using external clues; or (2) she can make correct matches using paranormal X-ray vision. The alternatives we could control or reduce were that she gets correct matches just by luck or that her correct matches are due to those factors that make vague statements seem like hits.
We used all our resources to obtain seven subjects. If we had been trying to test for a moderate or weak effect, we would have had to use many more subjects. Our test, then, was aimed at detecting a large effect. We reasoned that if she possessed the reliability of diagnosis that her proponents claimed, our test would reveal this. Such an effect would encourage us to investigate her abilities in more detail.
I would like to get started with something with this intent, too.
The outcome of the test could be from zero to seven correct matches. We set the criterion for success at five correct matches.
If we disregard any additional clues about health information that might throw off the probability of guessing, what were the odds that she get five out of seven correct by guessing?
Although Natasha's mother says that her daughter never makes a mistake, we did not want to demand that Natasha perform perfectly. We wanted to give her some margin for error.
Could I get a teensy-weensy margin of error too? Just a little one?
Keep in mind that if she got five or more correct this would be consistent with her having the X-ray power that she claims. Yet it would also be consistent with the possibility that she was matching the target condition by normal means such as the appearance and behavior of the subjects.
Yet she was given a test. So why would I not be?
Richard Wiseman, Andrew Skolnick, and I [Ray Hyman] collaborated in designing the test. We arrived at a mutually satisfactory plan after exchanging several e-mails. The task of finding appropriate subjects, and coordinating the many details was left to Andrew. He had less than one week to accomplish all this. He had to do this from Amherst, more than 350 miles from New York City.
I wonder if Mr. Skolnick would like the honor of finding some volunteers for my test?
During the test, we seated the seven subjects in a semicircle facing the chair where Natasha sat.
My protocol is much better already. The volunteers are not facing me but I see their backs. And I only see a small portion of their body, ie. part of the back. And I wasn't allowed to see all people at the same time but one at a time.
Each volunteer had an internal condition that should be easy to detect if Natasha's claim is correct. The target conditions were as follows: One patient had metal surgical staples in his chest from open heart surgery; one had a section of her esophagus surgically removed; one had a large section of one lung removed; one had an artificial hip replacement; one had a missing appendix (we discovered afterwards that another subject also had a missing appendix, which he didn't mention when we recruited him. Natasha chose neither of these two as the one with the missing appendix); one had a large brain tumor removed and now has a large hole in his skull covered by a metal plate; and the final subject had none of these target conditions.
Would seeing a portion of a person's back really reveal clues about their number of kidneys? Meanwhile Natasha was looking at the people and even though they didn't have mutual eyecontact, nobody's hearing was covered and all body language was available.
Metal surgical plates in the chest? Looking at the people from their front? Surely this has some clues. Esophagus? Clues, look at how they swallow. Lung? Clues, just look at the movement of the person's chest and how they breathe. Artificial hip? How they are standing and how they lean their weight on their hips. Missing appendix? Clues? How come missing a kidney would give off clues that we can't accept? And only from seeing their clothed back?
During the test, when Natasha was looking at the subjects, the subjects wore sunglasses whose lenses were covered with opaque tape. This prevented the subjects from knowing when Natasha was looking at them. This also prevented Natasha from picking up clues from their eye movements or pupillary dilations (which are a sign of emotional reaction).
And I will be facing their back and not even be able to see their heads.
Before the test, I instructed and rehearsed the subjects on how to behave. They were to sit as still as possible when Natasha was in the room. If Natasha needed to observe them in a standing position, I would tell Natasha to turn her back while they stood up and when they sat again. We used similar precautions if Natasha needed to look at them in profile. These precautions reduced the possibility of reactions by the subjects from knowing which target condition Natasha was currently studying. We also wanted to reduce external movements (for example, the subject with a hip replacement might give herself away from her efforts to stand or to change the position of her body).
I won't even be asking the volunteers to shift position.
Except for the subjects (and Austin Dacey [executive director of the Center for Inquiry-MetroNY]), everyone in the test room, including myself, was blind to the condition of each subject.
Because Andrew was in charge of recruiting the subjects and was not completely blind to their conditions, he stayed out of the testing room.
He remained in the briefing room during the entire test (which lasted more than four hours).
Four hours? I have now reduced to strictly 2 1/2.
We used this room to brief Natasha before each of the six required matches (once she had made six matches, the seventh was determined by default).
The test consisted of six trials. On each trial Andrew gave Natasha a test card that clearly described, in Russian and English, the condition she was to match to a subject.
After Natasha had studied the subjects for the given condition, she chose the subject she believed had the specified condition. She would circle the subject's number on the test card and both of us would sign the card. We then returned to the back room to prepare for the next condition and trial.
We wanted to make the test as comfortable and nonstressful for Natasha as possible. I made sure not to rush or pressure her in any way. I gave her all the time she wanted to make each match.
Oh if all Skeptics were like Ray Hyman, I would be so comfortable during the test, and with all the time in the world and nothing stressing me to make an answer.
She took one hour to make the first match-which was to find the subject who had a large section of the top of her left lung surgically removed. She required more than four hours to complete the matches of conditions to the seven subjects. Throughout this process I repeatedly asked her if she was comfortable and if we could do anything to make the process more agreeable to her.
Meanwhile I am anticipating a very uncomfortable and rushed test. We need Ray Hyman.
Natasha succeeded in correctly matching four target conditions out of a possible seven. Our protocol required that Natasha get five or more correct matches to "pass" our test.
Both inherent and unforeseen limitations of our test provided possible clues to the target conditions for some subjects. I already discussed the daunting task of finding seven appropriate subjects. We had to settle for a less than optimal set of subjects. These subjects differed sufficiently in outward appearance to provide possible clues about their conditions.
Meanwhile I fail to see how a person would give off signs that they have one or two kidneys. Especially if I am asked to diagnose all subjects with one or two, they should all be equally nervous and all feel like they are the 'target' and twitch in funny ways as some Skeptics here have suggested.
Our test included five subjects for whom external clues were available concerning their internal condition. The clues correctly pointed to the true target condition for four subjects.
Meanwhile nothing should be expected to point to the number of kidneys in a person.
The external clue for the fifth subject falsely pointed to the hole in the skull [the man was wearing a hat however he was not the one with the metal plate in his skull]. In each of these five cases Natasha made her choice consistent with how the external clue was pointing.
Nor would I expect anything to point to the false number of kidneys in a person.
Because a single test, even one done under ideal conditions, cannot settle a paranormal claim, we conceived our test as the first stage of a potential series.
I would love to begin my potential series now.
The first stage would not necessarily rule out nonparanormal alternatives. If Natasha could pass the first stage, this would justify continuing onto the next stage. If she passed that stage, then we would continue studying her claim. On the other hand, if she failed at any of the early stages, this would end our interest in her claim.
I wish I could go about it this way, rather than probably ending up with the underground bunker protocol.
Our test had its limitations. None of these limitations, however, worked against Natasha's claim. If anything, they may have artificially enhanced her score. Our task was not to prove that Natasha does not have X-ray vision. Rather, Natasha and her supporters had the responsibility to show us that she could perform well enough to deserve further scientific investigation. This they failed to do.
Why was Natasha given this convenient test and I am not? Is it because Natasha had been offering psychic medical readings in her country Russia and at a fee for many years? What does it take for me to get a test put together and test whether my medical perceptions depict real health information or whether they are a subjective experience? Would I need to offer medical advice to persons? Is that what it takes?