scientific review criticism

[rbairos]

Hi, Ive found myself embroiled in a long debate with a certain person
that claims the JREF is unscientific as the testing and results of
its investigations are not published in peer reviewed scientific literature.
Ie. that the JREF passes its MDC results as science when they should not.
I would appreciate any links to specific investigations publicized
in scientific peer-reviewed channels.

Thanks in advance,
Rob.

 

[CriticalThanking]

Please heve your acquaintance define terms a little differently. Peer-reviewed papers are attempts at scientific research. To my knowledge, the JREF has never claimed that the MDC is scientific research. It is a challenge to anyone claiming paranormal ability to be able to demonstrate the existence of such phenomena under controlled conditions. To date, no one has been able to do so depsite the fact that all such claimants helped develop the protocols and agreed to them in advance. Why would anyone publish a paper saying a particular unknown phenomoenon still has not been shown to exist? I still don't see leprechauns. Should I publish a paper on that?

Is the MDC scientific research? No. Is it helping to apply scientific reasoning to certain unscientific claims? Yes.

CT

 

[rbairos]

Is the MDC scientific research? No. Is it helping to apply scientific reasoning to certain unscientific claims? Yes.
CT

Unfortunately thats the source of the criticism.
If it is not scientific, it should not be used as scientific evidence against any paranormal claims.

I disagree however, I conclude it is scientific evidence, and the results are peer reviewed when informally presented at one of the many speaking engagements JREF does at scientific institutions.

Im hoping for something more tangible though..

 

[Cavemonster]

I think you're using the word "scientific" very broadly.
The test itself is made to be rigorous, made to be fair, but it isn't scientific evidence the way a peer reviewed article would be. It was never meant to be. It is meant to encourage critical thinking and to gain publicity to do so.

You may say that these informal presentations are like peer review, but peer review is a much more specific process. For one thing it occurs before publication of the results.

 

[Freethinker]

All of the challenge winners to date have had their results written up in peer reviewed journals.

 

[Lrrr]

Unfortunately thats the source of the criticism.
If it is not scientific, it should not be used as scientific evidence against any paranormal claims.

I agree with this in that the MDC is intended allow claimants to prove paranormal claims using a scientific method. It is not intended to disprove paranormal claims. When a claimant fails a test, the end result is not "the failure of the claimant to <insert claim here> proves that <insert paranormal activity> does not exist", only that the claimant has not proven that the paranormal activity does exist.

 

[drkitten]

Unfortunately thats the source of the criticism.
If it is not scientific, it should not be used as scientific evidence against any paranormal claims.

There's a difference between "this isn't science" and "this isn't science research."

Journals typically only publish research -- "novelty" is one of the key criteria for all of the journals and conferences for which I review.

Even conferences usually only publish research. The process of running controlled experiments is a key part of scientific practice and a key part of scientific training --- but not publishable.

 

[rbairos]

There's a difference between "this isn't science" and "this isn't science research."

Journals typically only publish research -- "novelty" is one of the key criteria for all of the journals and conferences for which I review.

Even conferences usually only publish research. The process of running controlled experiments is a key part of scientific practice and a key part of scientific training --- but not publishable.

But how would you justify the experiments themselves are controlled or even scientifically valid?

 

[Robert Oz]

But how would you justify the experiments themselves are controlled or even scientifically valid?

There would be no need to have the tests peer-reviewed until the results claim to prove something.

The tests are agreed to be fair by both parties and every person taking the test has failed to pass - this means nothing has changed. The person taking the test has failed to show they can do anything out of the ordinary. JREF has claimed nothing.

Imagine I claimed that gravity doesn't work. To prove my hypothesis, I devised a test where I would let go of an apple a metre from the ground and my hypothesis would be proven by the apple remaining suspended in mid-air. If, when I conduct the test, the apple drops to the ground, why would I need to have my test peer-reviewed. I haven't proven anything.

 

[athon]

Science isn't an objective term, unfortunately. There is no single, defining feature outside of our imagination that we can point at and say, 'hey, now it's science!'. Add to that the fact that the word has multiple meanings under different contexts, and you've got yourself a mire of confusion.

To get past this, you need to have a look at the context of the situation, as you would for any claim. Hell, not all research published in a journal is equal. Some journals have weak review panels consisting of people who all share the same biases. Deciding what constitutes science and science research on such a simplistic premise as whether it is in a journal or not is a poor way of thinking critically about information - understanding how the information was produced is far more important.

The challenge has scientific rigor for the following reasons - bias is spread across participants (on one side there are people who view the claim with doubt, while the other participants typically view it with preformed conclusions); the outcome is clearly articulated and made known prior to the experiment; and the data are recorded clearly and accurately, and checked by the party with an invested interest. As far as science goes, this is far more rigorous than a lot of peer reviewed papers that have snuck into journals in recent years.

So, ask your friend what difference it would make putting it into a journal. The process is more transparent, detailed and balanced in terms of bias than a lot of research. As far as I can see, that makes it pretty damn good science.

Athon

 

[drkitten]

But how would you justify the experiments themselves are controlled or even scientifically valid?

By looking at the experiments themselves. (Implicit "d'uh.")

The demonstrations Randi runs are, by design, the sort that can be evaluated by a lay person; he won't accept a protocol that is not transparent. Either you can tell me what's in the box or you can't; the apple either floats or it doesn't.

The flip side of that is that if you can't look at an accepted protocol and determine its validity, then the problem is with you, not with the protocol.

Yes, in theory he could mess up and accept a protocol/demonstration that wasn't valid; the Carina Landin experiment with the diaries might be an example. On the other hand, it's also immediately obvious that the protocol was violated (even if for what seemed a good reason) and the results weren't valid -- and he 'fessed up immediately when this was pointed out.

The best argument suggesting that the demonstrations are controlled is actually the fact that no one's been able to pass yet. Since the controls are a) common-sense, b) non-intrusive, and c) prevent anyone from passing, this seems to show that they're effective at doing their job, which in this case is preventing cheating.

 

[Doubt]

Well, having been involved in a couple of protocol negotiations, I can say that the preliminary test is not a "scientific" test. The preliminary tests are designed to make sure the person being tested can do the thing they say they can do before moving on to a real, "scientific" test.

The bar for the preliminary test is quite low. The controls are designed to eliminate chance, trickery and cheating and that is all. Yet so far nobody has passed the preliminary test.

The real scientific test cannot happen until a simple bit of evidence for an effect exists. So far that has not been the case.

 

[shadron]

You might also note that peer reviewed journals don't post $1M rewards for success.

This isn't peer reviewed science under the rules of the scientific establishment; it is testing of claims of the paranormal under the rules of the JREF. The establishment of the protocol is done in a scientific manner, because JREF believes that that is the best way to guarantee that the reality of the claim is tested. Better than that, though, is that it is done in a skeptical menner.

If someone should win the prize, then that will presumably open up a fertile field of scientific sudy suitable for peer reviewed science.

Or not. Depends.

 

[rbairos]

You might also note that peer reviewed journals don't post $1M rewards for success.

This isn't peer reviewed science under the rules of the scientific establishment; it is testing of claims of the paranormal under the rules of the JREF. The establishment of the protocol is done in a scientific manner, because JREF believes that that is the best way to guarantee that the reality of the claim is tested. Better than that, though, is that it is done in a skeptical menner.

If someone should win the prize, then that will presumably open up a fertile field of scientific sudy suitable for peer reviewed science.

Or not. Depends.

Try as I might all Ive seem to have done is empower him to gather
his flawed personal posts and my misrepresented rebuttals into a new youtube
post on Sheldrake of all people.

Its quite aggravating.

Rob.

 

[rbairos]

If anyone's interested to continue the sparring, its been moved to:

http://www.youtube.com/watch?v=YB3SAD-gHTc&feature=channel_page

The original argument can be found in my channel comments under 'rbairos1'

-Rob.