Remote healing

[saizai]

Why wasn't Pope John Paul II healed by the over 1 billion Catholics praying for his life?

A billion people, saizai? You can't ask for a better, stronger experimental setup: A 6th of the world's population, praying for just one person's health.

And yet, he croaked.

Oh, come now. A cute bit of hyperbole, but that argument is so flawed I'd feel wrong in tearing it apart. Too easy. Surely you see that.

I will if you insist, though.

 

[saizai]

Incidentally, my name (as mentioned at the top) is Sai. I'd prefer you use it.

 

[CFLarsen]

Oh, come now. A cute bit of hyperbole, but that argument is so flawed I'd feel wrong in tearing it apart. Too easy. Surely you see that.

I will if you insist, though.

Go ahead.

 

[saizai]

As you wish.

1. This is a retrospective, and thus entirely uncontrolled, hypothesis - as well as being liable to selection bias by definition.
2. There is no control case.
3. My claim is not to prove that prayer is additive, as you imply by hyperbole. A positive result of this study would solely prove that prayer has *some* effect on the recipient. It is easy to set up models that claim this is limited - viz. Zep's "binary" hypothesis - i.e. that there is only so much prayer can do.
4. The sample size is absurdly small.

There's more, but any of those by itself is enough to render your little throwaway completely untenable.

As a joke it's mildly amusing, but don't pretend to claim it's an actual rebuttal.


Please, if you can rebut the logic I have given so far, by all means do so. This doesn't.

[Edit:] It's also completely tangential to the topic at hand...

 

[CFLarsen]

1. This is a retrospective, and thus entirely uncontrolled, hypothesis - as well as being liable to selection bias by definition.

People were asked in advance to pray for the Pope's deteriorating health. Not retrospective at all.

Why was it uncontrolled, as opposed to your own hypothesis?

What do you mean by "selection bias"? He can't be healed because he's the Pope?

2. There is no control case.

Of course there was. All those that were not prayed for in the meantime.

3. My claim is not to prove that prayer is additive, as you imply by hyperbole. A positive result of this study would solely prove that prayer has *some* effect on the recipient. It is easy to set up models that claim this is limited - viz. Zep's "binary" hypothesis - i.e. that there is only so much prayer can do.

Regardless of numbers, the Pope was not healed.

How do you know that there is only so much prayer can do? How would you measure the "strength" of a single prayer?

4. The sample size is absurdly small.

Perhaps. But if not God's representative on Earth, why believe that anyone could be healed?

There's more, but any of those by itself is enough to render your little throwaway completely untenable.

As a joke it's mildly amusing, but don't pretend to claim it's an actual rebuttal.

It makes a very good case why prayer doesn't work. So far, we have seen nothing that indicates that it does.

Please, if you can rebut the logic I have given so far, by all means do so. This doesn't.

Considering that your hypothesis is filled with so many uncontrollable factors, I don't think you can speak of "logic" when it comes to that.

 

[saizai]

I believe your arguments are spurious, but I decline to be drawn into a debate about them. Your "Pope example" is completely irrelevant to the question of whether my study has sound methodological design or not.

Again, if you contest the logic I have given as to how your unspecified "uncontrollable factors" might affect the outcome of the experiment in such a way as to cause it to report an effect that is explainable by mundane means, please elaborate and point out the flaws in my logic.

(I would suggest you read this first: Wikipedia:Randomisation .)

If not, then admit that the design is sound and leave it be at that.

I have no desire to get into an argument about the hypothetical theologies that might underlie such an effect here; if you would like to raise that issue, do so in another thread. This one is solely about the two questions I raised in my first posting.

 

[CFLarsen]

For starters, you can begin by describing how you will guarantee that somebody not included in the experiment isn't praying for one of the sick participants.

 

[saizai]

No, you can begin by explaining how what you mention is not encompassed under, and controlled by, my previous comments re. randomization as a control for miscellaneous variables. Such as the one you describe.

I have already done my part.

 

[saizai]

While you're at it, please describe how your hypothetical confound would cause a *false* positive, given that you would need to assume that remote prayer works to be able to make the claim in the first place.

 

[Gayle]

Someone said, "God answers all prayers. Sometimes the answer is No."

Sai,

I have skimmed the posts this morning, but not read them thoroughly. I will try to do a better job when I have more time.

In answer to your question about Targ's studies. The AIDS distant healing study was published in the Western Journal of Medicine. It ceased publication a couple years ago and it is no longer on-line. You may be able to get a copy at a University Library. However, it would be of questionable assistance to you, considering that it was published with fatal flaws that were not revealed until much later.

The one thing that Targ's AIDS study did show, but did not publicly reveal, is that standard Western medical care -- the AIDS drug cocktail -- saved lives while her remote healing efforts had no effect at all. In fact, as you read in the article, on one measure, the prayer group did worse.

I don't know if the brain cancer study was continued after her death or if there was even funding available after it was revealed that her group had fudged the data.

You have convinced me in your posts that you are sincere in your efforts and that you want to do a well-designed study. The book "Healing Words" will not give you enough information. It's vitally important that you put yourself into a learning stance. That means accepting constructive criticism of your methodology and design, even if you don't like what you're hearing. It's much better to figure out the troublesome areas before you start, rather than after you're done.

These comments represent a fatal flaw in your thinking regarding design and methodology:

Re. possible falsification: sure, that's a possibility. But that's true with *all* studies, and I don't think it's something that should be considered a serious flaw. They report when they do, are encouraged to be honest, and asked to swear that the information they give is true. That's all you can ask.

Encouraging the healers to be honest is not enough, especially if you intend to apply for the Million Dollar Challenge. You need a tighter protocol.

However, I don't even know if prayer is eligible for the Challenge. There is a set of FAQs (written by Beleth.) Those answer many questions that are not in the rules. If you have not read them, I suggest that you do.

Then, if you want to apply for The Challenge, it would be important that you apply in the proper manner, with a notarized statement. From there, KRAMER and James Randi will help you find a group of skeptical associates who can help you design a proper protocol -- that is, if your application is accepted. Members of the forum will offer suggestions, some of which may be seriously considered in setting up your protocol.

The protocol has to be negotiated with the JREF. Both the applicant and the JREF have to find it agreeable. But as I said, I don't know if prayer is even eligible.

It would benefit you to read about other prayer studies. Some were published to much fanfare, only to be debunked later. That's certainly true of Targ's study. There is a prayer study from Columbia that is scandalous.

I am asking that your not only read the glowing reports of the distant healing studies, but that you also read the criticism. In that way, you can avoid making similar mistakes if you decide to go forward with your study.

Some of the suggestions you will receive in the forums may be of a rough and tumble nature. However, they can help you avoid making mistakes. Designing a tight protocol is not easy. When I was in school, I took three classes on experimental design and statistics. As a result, I can spot good design and bad design. But I'm still not good enough to come up with adequate design on my own.

If we want to be taken seriously, we have to take the rules of experimental design seriously. With help, I'm sure you will. Unfortunately, the book you have relied on is not good enough for your purposes.

Here's a couple links to critical articles. I'll try to find more when I have time. By studying failures, we can lean to avoid their mistakes.

The Columbia University 'Miracle' Study: Flawed and Fraud
http://www.csicop.org/si/2004-09/miracle-study.html


The Bizarre Columbia University 'Miracle' Saga Continues
http://www.csicop.org/si/2005-03/miracle-study.html


ETA: Speaking of the rules of the game: Sai, one of the rules here is that nobody has to explain themselves to you. It is the job of the person who makes a claim or who wants to be an applicant to the Challenge to provide evidence to back up their assertions. In a way, this is like a Ph.D candidate defending her dissertation to a hostile committee. You're the candidate and we're the committee. That's the way it is.


Gayle

 

[CFLarsen]

No, you can begin by explaining how what you mention is not encompassed under, and controlled by, my previous comments re. randomization as a control for miscellaneous variables. Such as the one you describe.

I have already done my part.

Actually, you haven't. It's called "controls", and by not ensuring that outsiders can influence the outcome, your experiment is worthless from the get-go.

It's your experiment. You present it. You defend it. You explain it.

While you're at it, please describe how your hypothetical confound would cause a *false* positive, given that you would need to assume that remote prayer works to be able to make the claim in the first place.

It's not hypothetical at all - which is the major problem. People pray all the time for those they know are sick. If just one not involved in the experiment prays for X while X is not supposed to be prayed for, it will invalidate the data completely.

The point is: You can't ensure that it won't happen. Ergo, your experiment will never give any results of any value whatsoever.

Do you understand this? It's like trying to measure the temperature of water, only you don't know if the water is being heated or not.

 

[saizai]

CFLarsen -

It's your experiment. You present it. You defend it. You explain it.

I did. I did. And I did.

And you, it seems, have completely ignored the logic I presented in favor of continuing blindly with your attack - without addressing my counterarguments at all. I believe I completely addressed the objection you still raise. You have not progressed past your initial argument, which I rebutted.

Please explain how the variable you claim to be a confound is not controlled by randomization.

Also, are you failing to notice that the experiment does indeed include controls - namely, the half of the recipients (determined randomly) who are not the subject of prayer as a result of the study?

It's not hypothetical at all - which is the major problem. People pray all the time for those they know are sick. If just one not involved in the experiment prays for X while X is not supposed to be prayed for, it will invalidate the data completely.

Sorry, but that's false. The experiment does not purport to control all prayer whatsoever for the subject. It merely adds one (controlled) variable: whether they are prayed for by the "healer" group. That is what is being tested. Everything else is randomized.

And you have not addressed my other counterargument: that to say that someone randomly (remotely) praying for some recipients - even if you could argue that this would happen more for one of the control groups than the other, which you haven't (and can't) - you would have to presuppose that prayer works. Which would itself be an admission of a positive result.

 

[CFLarsen]

I did. I did. And I did.

And you, it seems, have completely ignored the logic I presented in favor of continuing blindly with your attack - without addressing my counterarguments at all. I believe I completely addressed the objection you still raise. You have not progressed past your initial argument, which I rebutted.

Please explain how the variable you claim to be a confound is not controlled by randomization.

I did. I did. I did.

Also, are you failing to notice that the experiment does indeed include controls - namely, the half of the recipients (determined randomly) who are not the subject of prayer as a result of the study?

No, I have noticed that. Great. Problem is, that control is worthless, since you can't ensure that other people are influencing the outcome.

Sorry, but that's false. The experiment does not purport to control all prayer whatsoever for the subject. It merely adds one (controlled) variable: whether they are prayed for by the "healer" group. That is what is being tested. Everything else is randomized.

You clearly don't understand the problem. Here's what you are doing:

You want to investigate if heating up a pot of water increases the temperature of the water. So, you heat it up, and measure the temperature. Then, you don't heat it up, and measure the temperature. And so on.

Problem is, you can't see if someone else is heating the pot or not heating the pot, at any given moment during your testing. Therefore, whatever temperature you get out of it has nothing to do with you heating up the water - or not - because you can't control all input.

And you have not addressed my other counterargument: that to say that someone randomly (remotely) praying for some recipients - even if you could argue that this would happen more for one of the control groups than the other, which you haven't (and can't) - you would have to presuppose that prayer works. Which would itself be an admission of a positive result.

Not at all. You are testing if prayer has any influence on people's health. That's not the same as presupposing that prayer works.

How have you calculated how big a sample size is needed for your experiment?

 

[saizai]

Re: Someone said, "God answers all prayers. Sometimes the answer is No."

Gayle -

I have skimmed the posts this morning, but not read them thoroughly. I will try to do a better job when I have more time.

Please do. I believe I have completely rebutted all claims of methodological flaws so far.

The book "Healing Words" will not give you enough information.

Obviously not; most of the book is pap. Occasionally interesting, but not particularly useful. I read it primarily to see what logical flaws it presented (relatively few, in the parts that actually examine previous studies), and what ones there were in studies it cited as providing support.

I hope I have not given the false impression that I have derived the idea for this study, the methodology behind it, or the controls, from the book. I did not.

I take the design quite seriously. I have no personal stake in the result of the study; I am simply curious, and wish to do something that would obviate any of the usual objections. I do not expect a negative to convince believers of nonexistence; a positive would simply be justification to do a study one or two orders of magnitude larger and better-controlled.

I am quite open to criticism, so long as it is logical. I think I've been consistent in that. If you or anyone else can refute the logic I gave for defending any of the contested points of methodology, please do so. But I won't accept mere insistence in the existence of "uncontrolled variables" that does not bother to take into account my rebuttal - or such a basic design concept as randomization.

Encouraging the healers to be honest is not enough, especially if you intend to apply for the Million Dollar Challenge. You need a tighter protocol.

I disagree. Frankly, this step is superfluous; it is there solely as a check.

The strongest argument you can make is that every healer lies every time, and no prayer is done as a direct result of the study (or IOW the test group will not receive statistically more prayer than the controls). This, then, would produce a null case of difference between groups - i.e. solely their status within the study itself, since they would in no way be treated differently. This is not a confound.

Any weaker argument would have it that they *are* being prayed for by some of the healers some of the time. Which is what we are testing for. And since there is no mundane explanation for how this could have any effect on the recipient one way or the other, it is again not a confound - it could at most be argued (by a believer) to reduce the resulting effect.

Can you rebut that?

However, I don't even know if prayer is eligible for the Challenge.

I have made preliminary inquiry. I see nothing in the FAQ or challenge rules that prohibits it; there is a disqualification of things that *presuppose* something with religious basis (e.g. exorcism) as unprovable (for obvious reasons). This does not have that flaw, and in fact makes no religious claims whatsoever. It involves purely mundane, measureable end results.

As for the rest of your comments in re. protocol, I have actually read the FAQ and challenge rules, as well as a sampling of previous applications (highly amusing). Thank you for the intent, though. I don't intend to formally apply until I have the methodology fully worked out for my own sake, and preferrably run it by my university's IRB.

Thank you for the links; I will peruse them and post if I have any comment.

As for "justification": I expect you, the "attackers", to use logic. You are plenty welcome to disagree with me, or to point out any holes you see in my logic or my methodology.

But if you decline to engage me logically - by countering my rebuttals and not going into logical fallacies (viz. the Pope tangent) - then I don't feel obligation to consider you seriously desirous of arriving at a sound outcome.

Until someone counters them, or points out a flaw not encompassed by my response, I will not respond further to arguments I have already rebutted except to point to what I have already said. I think this is fair, even in a case where the burden of "proof" is upon me.

- Sai

 

[saizai]

I did. I did. I did.

Would you please quote me the passages in which you refute the validity of randomization as a control for "external" variables - such as third parties praying for participants? I do not see them.

Let me rephrase your analogy.

You are testing whether putting a pot on a lit stove increases the temperature of the water inside. So you put one on a lit stove, and one on an unlit stove. (As a simplification - obviously you'd do far more than one and one.)

You are claiming that I am unable to determine whether little fairies fly in and magically heat the water of the pot with blue sides. Or the one that's not being lit. Whatever.

The point is, this is utterly irrelevant. You do not claim to control all potential sources of heat; you cannot. All you claim to control is whether or not you light a fire under it, and that that choice will be random. If the ones that have fires lit are hotter, that is conclusive evidence that it is because you lit the fire - the interference of pixies is randomized out.

[Edit:] Also, this analogy misses the double-blind: in the more accurate version, it should be impossible for any third party (or even yourself) to determine which pots have a fire and which don't.

Not at all. You are testing if prayer has any influence on people's health. That's not the same as presupposing that prayer works.

Please explain the difference.

If you are claiming that a third party, praying for (somehow, magically) only the control group people, would be able to influence the results, then you are necessarily claiming that which is defined as a positive result: that you could cause a significant change via remote prayer.

How have you calculated how big a sample size is needed for your experiment?

As I indicated above, I have not - that is one of the questions I had for those on this board more mathematically inclined than I. I suspect that it would be roughly on the order of one or two hundred. The cutoff is that the results be to at least 95% certainty (i.e. p<0.05) and be able to detect a reasonably small effect.

 

[CFLarsen]

Would you please quote me the passages in which you refute the validity of randomization as a control for "external" variables - such as third parties praying for participants? I do not see them.

I've said this so many times now: You can't control for people outside the experiment praying - polluting the data, if you want.

Let me rephrase your analogy.

You are testing whether putting a pot on a lit stove increases the temperature of the water inside. So you put one on a lit stove, and one on an unlit stove. (As a simplification - obviously you'd do far more than one and one.)

You are claiming that I am unable to determine whether little fairies fly in and magically heat the water of the pot with blue sides. Or the one that's not being lit. Whatever.

No, I am claiming that people you can't see - because you can only see a small part of the pot - are heating up - or not - the pot. Not fairies. People outside the experiment doing exactly what those inside the experiment are doing: Praying for the sick.

The point is, this is utterly irrelevant. You do not claim to control all potential sources of heat; you cannot. All you claim to control is whether or not you light a fire under it, and that that choice will be random. If the ones that have fires lit are hotter, that is conclusive evidence that it is because you lit the fire - the interference of pixies is randomized out.

Of course you can control all potential sources of heat. I did this in science class in high school. It's how you test the 2nd law of thermodynamics, perhaps the most tested law of the universe.

Please explain the difference.

It's exactly the same as if you wanted to test if heat applied to water raises the temperature in the water. You don't presuppose that it does.

If you are claiming that a third party, praying for (somehow, magically) only the control group people, would be able to influence the results, then you are necessarily claiming that which is defined as a positive result: that you could cause a significant change via remote prayer.

No, I am pointing out that you can't control for people outside the experiment praying for the people inside your experiment.

As I indicated above, I have not - that is one of the questions I had for those on this board more mathematically inclined than I. I suspect that it would be roughly on the order of one or two hundred. The cutoff is that the results be to at least 95% certainty (i.e. p<0.05) and be able to detect a reasonably small effect.

You haven't even decided how many people are needed??

Back to the drawing board. Get back when you have something better, because this is one half-baked experiment.

 

[saizai]

QUESTIONS

So, there are three basic outstanding questions on the design.

#1: What is a sufficient number of test subjects?

Corollary / prereq: What is a sufficient percentage-certainty, and how small of an effect (on the assumption that any effect generated might be fairly small) could one detect?

#2: What would be an optimal disease to target?

Specs as initially stated - something common enough to have a large pool to draw from in multiple locations, serious enough to routinely have a poor prognosis; having easily ratable status (statii?); etc.

#3: What would be a good initial equation to compute a single, numeric result from thhe prognosis and progress? (Being able to incorporate ongoing progress in addition to just the endpoint would be good.)

As mentioned in a comment halfway down this page or so, this would be better done by having a divided study sample, and using the first half to determine an optimal measure to use for the second half.

 

[SezMe]

If you're talking about blind prayer, well, that's kinda hard to track. You'd have to assume it randomized out.

Precisely. An assumption that you have no basis for. Which negates the value of the protocol.

 

[saizai]

I've said this so many times now: You can't control for people outside the experiment praying - polluting the data, if you want.

Then I'll ask you a final time: PLEASE explain why this would not be controlled by the randomization.

Quite specifically: how would third parties' prayer - granted that this is a double-blind study and therefore third parties have no possible means of knowing whether a particular person is in the control or test groups - have an effect that is different on the control group than the test group in any way?

If you can't show that, then you can't claim that it's a confound.

I claim that there is no way for this to happen - that any effect of third-party prayer would be, statistically, randomly distributed between control and test groups. And that therefore there is no way for it to affect an overall outcome that measures the difference between these two groups.

If you can counter that argument, please do so. But you haven't - you've only reiterated your initial assertion that they somehow will.

To be quite clear: I make no pretense to control for whether any particular person is prayed for a specific amount (nor do I care). Solely for whether the test group as a whole will be prayed for more than the control, because that is what the "healers" are assigned to do.

Of course you can control all potential sources of heat. I did this in science class in high school. It's how you test the 2nd law of thermodynamics, perhaps the most tested law of the universe.

No: you can't control the "god made this hot because he felt like it" theory, on an individual sample level. Nor can you disprove it. But you can control it for the study overall - that is what randomization does.

You haven't even decided how many people are needed??

Back to the drawing board. Get back when you have something better, because this is one half-baked experiment.

That's a detail of implementation. It has little to do with the methodology per se, given that I am specifying the statistical minimum outcomes I want - they in turn would define what the number of people needed.

No need to suddenly get flabbergasted.

 

[CFLarsen]

Re: QUESTIONS

So, there are three basic outstanding questions on the design.

#1: What is a sufficient number of test subjects?

Corollary / prereq: What is a sufficient percentage-certainty, and how small of an effect (on the assumption that any effect generated might be fairly small) could one detect?

#2: What would be an optimal disease to target?

Specs as initially stated - something common enough to have a large pool to draw from in multiple locations, serious enough to routinely have a poor prognosis; having easily ratable status (statii?); etc.

#3: What would be a good initial equation to compute a single, numeric result from thhe prognosis and progress? (Being able to incorporate ongoing progress in addition to just the endpoint would be good.)

As mentioned in a comment halfway down this page or so, this would be better done by having a divided study sample, and using the first half to determine an optimal measure to use for the second half.

.....why are you asking us?