skeptisim of EVPs

[blutoski]

As if any of these guys actually ever do rule out natural explanations.

Well, they do, but we disagree with them. Ergo the different worldviews.

Just because these guys make this apology for not using proper scientific process while claiming they are using scientific process does not make their measurements any more valid.

? I think I was explaining that your proposal won't work because the assumption can't be confirmed - there is no baseline for calibration. I don't understand your response.

It doesn't stop these guys from having observers interpret static independent of each other to see if they even get standardized results.

What guys? I don't understand. I think the conversation is getting confused.

And you can't generate a number randomly rather than you choosing it?

That would be the correct way - that was my point.

I don't think you understand what I'm trying to say. It was a way to illustrate why getting independent people to listen to a recording does not eliminate bias and is not actually disconfirmatory and therefore not scientific.

No, it's likely to debunk this make believe science.

? I don't think so. These false positives common, and are a huge problem because some skeptics think this protocol would be disconfirmatory.

 

[blutoski]

You've done this where none of the three people had any contact with one another, and they all came to the same conclusion about the noise?

Not me personally: this protocol has been done many times by investigators over the decades.

Did you include any speakers of foreign language?

Depends on what you mean: do you mean are all the documented cases in English? (no)

Or do you mean did the experimenters get people of different languages to listen to the same recording? (I don't know of any done this way).

If you choose a number, it's not random.

Obviously. That's my point.

I really really think there's some confusion here.

In the situation of a ghost investigation, we are in the circular situation where we can't control the 'send' - my point was that cultural inertia will make 'reception' cluster around certain results more than you would expect from random chance.

Many magic tricks pivot on this fact.

I read most of the page on EVP as presented by psican. It was a joke, really. I see where you got the notion that analog is somehow superior to digital. They are wrong.

That said, setting up protocols for EVP would not be all that hard. It's not rocket surgery. Skeptigirl has the right idea, though she didn't include ABX testing as part of it. In short you play A, then B, then X where X is either A or B. The listener has to tell you whether X was A or B.

Of course, you'll have to eliminate any other sources of sounds in the room and use multiple recording devices. For good measure, include ones with no external mics. And synch the separate recordings. And ensure the chain of custody including preserving the originals and distributing digital copies.

Not that I would waste my time doing it. It's much more fun to fumble around in the dark holding little recorders and giggling.

I don't think it's fair to make fun of them: most don't care whether it's all that scientific. It's a hobby. Other people take up golfing.

Be cautious not to project.

 

[Uncayimmy]

That would be the correct way - that was my point.

I don't think you understand what I'm trying to say. It was a way to illustrate why getting independent people to listen to a recording does not eliminate bias and is not actually disconfirmatory and therefore not scientific.

If that was what you were trying to do, I totally missed it. I'm not at all clear what points you are trying to make. I'm not even clear if you believe EVP to be real or not.

As skeptics we need to be very strict about protocols. Every time we participate in a test without proper controls - even if we voice our dissent - we're doing a disservice to the ultimate goal of teaching critical thinking.

 

[Uncayimmy]

Not me personally: this protocol has been done many times by investigators over the decades.

Maybe I'm just not up on the pseudoscience literature.

Or do you mean did the experimenters get people of different languages to listen to the same recording? (I don't know of any done this way).

Don't you think this would be an important thing to test?

Obviously. That's my point.

You used "I" to say "I would choose" some number. If you meant they, say they. It's not rocket surgery to get a random number.

In the situation of a ghost investigation, we are in the circular situation where we can't control the 'send' - my point was that cultural inertia will make 'reception' cluster around certain results more than you would expect from random chance.

By "we" who do you mean? Certainly not the collective we that would include me. Are you talking about yourself? Are you saying you cannot eliminate cultural bias? Or are you saying you can't get others to agree to a protocol that would do it?

The confusion in this thread is coming from your end.

Many magic tricks pivot on this fact.

Because the magician sets the protocol. Let a rat design the maze, and he'll always find the cheese.

I don't think it's fair to make fun of them: most don't care whether it's all that scientific. It's a hobby. Other people take up golfing.

What do you mean you don't think it's fair? What forces are you trying to balance? They make claims, I refute claims. They post videos, I comment on the videos. It's not like I'm watching them on hidden camera. They put it out publicly, so expect public comments.

Be cautious not to project.

Don't know what you mean.

 

[blutoski]

Don't you think this would be an important thing to test?

I do think it's important to test, but I don't think the protocol that was suggested is sufficiently disconfirmatory, which is why I recommend against it. Too many false positives.

You used "I" to say "I would choose" some number. If you meant they, say they. It's not rocket surgery to get a random number.

It was in context of a suggested protocol, and I provided two other examples as illustrations of the problem.

By "we" who do you mean? Certainly not the collective we that would include me. Are you talking about yourself? Are you saying you cannot eliminate cultural bias? Or are you saying you can't get others to agree to a protocol that would do it?

'we' in my sentence in that post meant anybody interested in critical analysis of proposed protocols - basically, people in this thread

'we' cannot control against cultural bias with this specific protocol, so I recommend against using it

Because the magician sets the protocol. Let a rat design the maze, and he'll always find the cheese.

The point is that there is a cultural inertia. Because of this, I believe that the proposed protocol of having independent people listen to the same recording and see if they 'hear' the same words has too many false positives. They will often hear the same words because they speak the same language and neutral sounds that resemble words will be mistaken for actual words by many people.

What do you mean you don't think it's fair? What forces are you trying to balance? They make claims, I refute claims. They post videos, I comment on the videos. It's not like I'm watching them on hidden camera. They put it out publicly, so expect public comments.

I'm not talking about public comments - I'm talking about ad hominem. It's just as wrong when skeptics do it.

 

[blutoski]

If that was what you were trying to do, I totally missed it.

skeptigirl recommended a double-blinded technique that I felt would actually result in too many false positives, so I was recommending against it.

I'm not at all clear what points you are trying to make.

skeptigirl recommended a double-blinded technique that I felt would actually result in too many false positives, so I was recommending against it.

I'm not even clear if you believe EVP to be real or not.

What difference would that make? I thought we were evaluating protocols on their own merit, rather than based on whose idea they were?

As skeptics we need to be very strict about protocols. Every time we participate in a test without proper controls - even if we voice our dissent - we're doing a disservice to the ultimate goal of teaching critical thinking.

Absolutely. skeptigirl recommended a double-blinded technique that I felt would actually result in too many false positives, so I was recommending against it.

 

[Uncayimmy]

Blutoski, you're just making things more confused. Here is what Skeptigirl wrote:

Get a number of test observers. Train them in the same way. No test observer is to hear the interpretation of any other observer and the 3 types of sample recordings are of course, double blinded.

If you can provide evidence the true EVPs are even distinguishable from the controls you have at least the rudimentary elements of actual science. Anything else is make believe before you even hit the ground.

Her only conclusion was that you would end up with the "rudimentary elements of actual science." She's absolutely right. Just because double-blinding may not result in 100% elimination of false positives is not a reason to recommend against it. She offered an extremely important first step that will eliminate a significant portion of the false positives found in the same test done without the blinding.

I offered other steps. One was to introduce ABX testing to prove that people can even tell the clips apart. Another was to introduce people listening for voices in a foreign language in order to demonstrate how we hear what we want to hear. That's not to mention all the suggestions I made regarding making the recordings in the first place.

I have no idea why you are resisting these things. Is English not your primary language? I ask that not as in insult but to indicate to you that I am really struggling to understand you.

 

[blutoski]

Her only conclusion was that you would end up with the "rudimentary elements of actual science." She's absolutely right. Just because double-blinding may not result in 100% elimination of false positives is not a reason to recommend against it. She offered an extremely important first step that will eliminate a significant portion of the false positives found in the same test done without the blinding.

I offered other steps. One was to introduce ABX testing to prove that people can even tell the clips apart. Another was to introduce people listening for voices in a foreign language in order to demonstrate how we hear what we want to hear. That's not to mention all the suggestions I made regarding making the recordings in the first place.

I have no idea why you are resisting these things. Is English not your primary language? I ask that not as in insult but to indicate to you that I am really struggling to understand you.

I'm not sure what you mean by 'these things'.

I'm recommending against the types of testing proposed because of:

• my personal experience evaluating and designing scientific experiments for about 15 years (give or take)
• my personal experience as a paranormal investigator over the last 20 years (I am a director of BCSkeptics, which is loosely affiliated with - and shares founders with - CSI)
• the opinion of other paranormal investigators who have considered and critiqued these (or similar) approaches over the years (eg: Ray Hyman, Joe Nickell, Susan Blackmore, Paul Kurtz, &c)
• the specific concerns, as described in earlier posts

 

[Uncayimmy]

I'm not sure what you mean by 'these things'.

The recommendations I made in the preceding paragraphs.

I'm recommending against the types of testing proposed because of:
[*] my personal experience evaluating and designing scientific experiments for about 15 years (give or take)
[*] my personal experience as a paranormal investigator over the last 20 years (I am a director of BCSkeptics, which is loosely affiliated with - and shares founders with - CSI)
[*] the opinion of other paranormal investigators who have considered and critiqued these (or similar) approaches over the years (eg: Ray Hyman, Joe Nickell, Susan Blackmore, Paul Kurtz, &c)

All arguments from authority. They are not worth the electrons used to display them.

[*] the specific concerns, as described in earlier posts

What specific concerns? Compare and contrast the following:

Double-blinding vs No Blinding

ABX Testing vs No ABX Testing

Using Speakers of Foreign Language vs. Speakers of One Language

Handling the Mics vs Mounting Them

Staying in the Room vs Leaving the Room

Making Digital Copies vs Letting People Handle Originals

Using Digital Gear vs Using Analog Gear

Using Devices Without Mics vs Using Only Devices With Mics

I argue that these things will reduce the number of false positives.

 

[blutoski]

The recommendations I made in the preceding paragraphs.


All arguments from authority. They are not worth the electrons used to display them.

If you say so.

See: [Skeptical MythConceptions - part 1 - Authority]

What specific concerns? Compare and contrast the following:

Double-blinding vs No Blinding

ABX Testing vs No ABX Testing

Using Speakers of Foreign Language vs. Speakers of One Language

Handling the Mics vs Mounting Them

Staying in the Room vs Leaving the Room

Making Digital Copies vs Letting People Handle Originals

Using Digital Gear vs Using Analog Gear

Using Devices Without Mics vs Using Only Devices With Mics

I argue that these things will reduce the number of false positives.

I agree.

Still doesn't solve the problem of the remaining false positive rate being too high. My experience is that these protocols, when followed, very often produce positive results. My concern is that psi advocates will use these results to justify their claims.

Let me rewind a bit... my impression is that you haven't bounced these off other investigators to see if they would find the results persuasive one way or another. For example, when I produce a protocol for a clinical trial, I shop it around. In particular, I ask the people whom I'm trying to convince what they would consider convincing. The protocols you design must be designed to address specific questions.

The key challenge in paranormal investigation is mitigating the concept of agency.

 

[Uncayimmy]

If you say so.

See: [Skeptical MythConceptions - part 1 - Authority]

You wrote, "I'm recommending against the types of testing proposed because of:" and then listed a bunch of names. A bunch of names are not a reason. Had you explained what they said and why along with their credentials, you'd be on to something.

Still doesn't solve the problem of the remaining false positive rate being too high. My experience is that these protocols, when followed, very often produce positive results. My concern is that psi advocates will use these results to justify their claims.

You don't understand what argument from authority means. For example, among bass players there is a common belief that aluminum cone speakers sound "tinny" - something I consider a myth.

If I say they don't sound tinny, that's an argument from authority (long-time bass player).

If I say I ran a test and proved it, that is still essentially an argument from authority.

Now suppose I lay out my experiment. I recorded four different speaker cabinets (one was aluminum) using the same gear and conditions except for the cabinet. I normalized the sound levels. I presented the WAV files to a discussion board comprised of experienced bass players. Of the 15 people who participated, only one was able to pick out the aluminum cone speaker. Almost half the people picked the aluminum cone speaker as sounding the best.

Is that an argument from authority? Not really. Sure, you have to trust that I am not lying or omitting key details. You have to assume I was competent enough to make quality recordings or introduce bias. You have to assume that I played the riffs the same way for each recording.

So, if you're going to say that there are "too many false positives" I want to know more details. I want to know how the studies were done. How many is too many? How many false positives come up without the controls?

Let me rewind a bit... my impression is that you haven't bounced these off other investigators to see if they would find the results persuasive one way or another.

To be honest, I don't care what paranormal investigators think is satisfactory. I've seen what a mess most of their experiments are, so they have little credibility. I care what critical thinkers say about it.

For example, when I produce a protocol for a clinical trial, I shop it around. In particular, I ask the people whom I'm trying to convince what they would consider convincing. The protocols you design must be designed to address specific questions.

I prefer to argue with them regarding what scientists would find convincing. I'd say more is learned by building a good protocol than by actually performing the test.

The key challenge in paranormal investigation is mitigating the concept of agency.

On that we agree.

 

[Cuddles]

I agree.

Still doesn't solve the problem of the remaining false positive rate being too high. My experience is that these protocols, when followed, very often produce positive results. My concern is that psi advocates will use these results to justify their claims.

Let me rewind a bit... my impression is that you haven't bounced these off other investigators to see if they would find the results persuasive one way or another. For example, when I produce a protocol for a clinical trial, I shop it around. In particular, I ask the people whom I'm trying to convince what they would consider convincing. The protocols you design must be designed to address specific questions.

The key challenge in paranormal investigation is mitigating the concept of agency.

I'm with UncaYimmy here, I'm really not sure what your point is. It seems as though you are saying that because the various suggestions won't necessarily eliminate all sources of error there's no point implementing them at all, and that since most of the people involved are promoting pseudoscience as a hobby that we shouldn't bother criticising them. If that is what you're saying, then to be honest I find that a rather bizarre attitude, and if that's not what you're saying then I have no idea what you're actually trying to say.

 

[blutoski]

You wrote, "I'm recommending against the types of testing proposed because of:" and then listed a bunch of names. A bunch of names are not a reason. Had you explained what they said and why along with their credentials, you'd be on to something.

Well, at this point, I'm not trying to convince you of anything, really. There are other people who read the thread, they can evaluate this statement as they see fit. There's only so much I can do.

You don't understand what argument from authority means.

I'm pretty sure you're mistaken.

To be honest, I don't care what paranormal investigators think is satisfactory. I've seen what a mess most of their experiments are, so they have little credibility. I care what critical thinkers say about it.

I'm talking about critical thinkers who are paranormal investigators. The list of authors I cited are precisely these people. Are you unfamiliar with CSICOP?

I prefer to argue with them regarding what scientists would find convincing.

Why would you care what scientists think, if you believe authority is meaningless?

In any case, you appear to have dismissed my concerns, along with those of others I listed. So: as a scientist, I am under the impression that you're not interested in what scientists would find convincing.

I'd say more is learned by building a good protocol than by actually performing the test.

I think it's necessary to identify the hypothesis before designing the protocol. The hypothesis is derived from the claim.

Consider the way the MDC works: start with the claim, and the protocol will follow.

Engagement with the claimants is critical to avoid spinning wheels with a strawman.

 

[blutoski]

I'm with UncaYimmy here, I'm really not sure what your point is. It seems as though you are saying that because the various suggestions won't necessarily eliminate all sources of error there's no point implementing them at all, and that since most of the people involved are promoting pseudoscience as a hobby that we shouldn't bother criticising them. If that is what you're saying, then to be honest I find that a rather bizarre attitude, and if that's not what you're saying then I have no idea what you're actually trying to say.

Yes, I appreciate that at face value it seems like I'm recommending against improving the protocols, but this isn't what I'm saying: what I'm saying is that in this case better is still not very good.

I do think that it's better not to do an experiment at all, if it's not going to make an impact on resolving a specific hypothesis. What I look for is disconfirmatory power, which is especially sensitive to what hypothesis, exactly, is being tested.

While EVP per se is new, it's really just a current version of the time-honoured attempt to evaluate physical evidence offered in defense of paranormal claims. These types of experiments have been conducted for something like 150 years, and I see a lot of reinventing the wheel. I advise that those who are serious about designing protocols examine which have been discarded, and why, and decide if they accept the reasoning.


Regarding paranormalists, yes I'm saying we shouldn't criticize the paranormalists - we should criticize their experiments. I don't know why recommending against ad hominem attacks would be controversial among skeptics.

 

[Uncayimmy]

Yes, I appreciate that at face value it seems like I'm recommending against improving the protocols, but this isn't what I'm saying: what I'm saying is that in this case better is still not very good.

And I'm asking for evidence. Have you tried all of the things I suggested in prior posts all in experiment? If so, what was the result?

While EVP per se is new, it's really just a current version of the time-honoured attempt to evaluate physical evidence offered in defense of paranormal claims. These types of experiments have been conducted for something like 150 years, and I see a lot of reinventing the wheel. I advise that those who are serious about designing protocols examine which have been discarded, and why, and decide if they accept the reasoning.

And I have been asking which protocols have been attempted and why they were discarded.

Regarding paranormalists, yes I'm saying we shouldn't criticize the paranormalists - we should criticize their experiments. I don't know why recommending against ad hominem attacks would be controversial among skeptics.

I made no such personal attack. They were bumbling about performing a joke for an experiment. Ridiculing an experiment is not a personal attack.