Polygraphs: The evidence

[drkitten]

It's an old cynic's trick called raising the bar.

As opposed to the old woo's trick called lying through your teeth?

No one will be able to provide a good enough explanation or amount of evidence.

I suspect that no one will even be able to provide a primary source indicating that the alleged experiment with atheists was done. Given the highly inethical nature of the experiment (the experiment as described is so far outside the parameters of a valid polygraph experiment it couldn't even telephone validity), I suspect it was either manufactured out of whole cloth (more likely), or a simple analysis of the testing organization's credentials will show that they have no relevant expertise.

 

[Skeptic Ginger]

It's an old cynic's trick called raising the bar.

No one will be able to provide a good enough explanation or amount of evidence.

I purposefully asked Claus to take this to a new thread instead of hijacking yours. Do you think you could give him the same courtesy and keep the 100 atheists discussion in your thread?

 

[drkitten]

I asked you Claus, to give your estimate of the accuracy of a polygraph. You can even describe the circumstances.

Let's be fair. As posed, the question is not answerable. I can easily design a polygraph system that will have a 0% false negative rate -- i.e. it correctly detects ALL attempts at deception. It's a face-down piece of paper with "he's lying" written on one side, which I turn over at the end of the test. (Oddly enough, although it's got 100% sensitivity for lying, its ability to detect gender is only about 50%....)

Needless to say, under such circumstances, it would be very easy for me to get a group of 100 atheists and have them all test as liars. But not especially convincing.

In order to specify accuracy meaningfully, you need at least two estimates, one for the sensitivity and one for the specificity.

 

[Skeptic Ginger]

It isn't answerable as snipped. Perhaps I wasn't clear, I was asking for any hypothetical situation he cared to give that involved interpreting the results of a group of polygraph answers as opposed to a single answer. I am talking about what a polygraph can tell you about a group of tests and Claus is talking about what a polygraph can tell you about a single test. I agree with him on the single test. But his one track mind has stopped him from ever getting to the group of tests question.

I just want him to get to the same conversation as I'm having with whichever hypothetical numbers he wants to use. It's like pulling teeth to get him off his single minded discussions.

 

[Jimbo07]

Is the validity of the polygraph really the point?

With a skilled test administrator

*cough* interview theater *cough*

 

[Skeptic Ginger]

What's your point, Jim?

Let me sum up the point of the thread for you. I made a statement in another thread that even though you cannot use a polygraph result to determine if an individual is deceptive, you can use it to draw a conclusion about a group of people being tested. If the results are that 100 people are being deceptive per the results of the polygraph, and the polygraph is accurate 80% of the time, then you can say that at least 80% of those people are likely being deceptive.

I did not declare what the actual percentage of valid results were because we don't have the details of the 100 tests by which to determine that including knowledge of the skills of the test administrator. I used 80% hypothetically and provided a source which did a very thorough scientific meta-analysis (GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low. The intent of the person being tested to deceive the tester greatly decreased the reliability of the results among other factors.

For whatever reason, Lonewulf and Claus didn't get the concept of using a test that was 80% reliable to draw a conclusion about 80% of 100 test results. And I don't get the problem. If Claus or Lonewulf wanted to argue that 50% was a better average estimate, I wouldn't argue. We can't say unless we know more about the 100 tests. But to say, you cannot use any figure to draw a conclusion about the group test results because an individual test result is not reliable is not a correct statement. Claus may be trying to make the case that the polygraph is never better than chance. I don't think he has provided any source supporting that claim.

 

[CFLarsen]

And this is why I won't write for SkepticReport. The editors cherry-pick and misquote, out of context, nearly as badly as creationists do.

I don't think you read what I said. I specifically said that I would publish what you wrote, complete unedited.

What about Skeptic Magazine? TAM?

This misrepresents the NAS report at several levels.

Fine. Write that in your article/presentation.

Drkitten has done a better job addressing Claus that I could hope to.

I asked you Claus, to give your estimate of the accuracy of a polygraph. You can even describe the circumstances. Then we can get to the issue of how to interpret the mythical 100 atheists tests.

So with people trying to deceive the polygraph, the reliability appears abysmal.

With a skilled test administrator and a population not trying to fool the test, the results are better than chance and perhaps even as good as 80%. I am not going to attempt to offer an opinion as to how high that number is except to say the range is wide according to the extensive analysis by the GAO source I cited.

How do you know the polygraph is correct any of the time?

Will you present a paper at TAM?

 

[.13.]

CFLarsen, you need to step back and think about this. Statistical tests can tell you how well an imperfect test works. We do it all the time:

1) A self-proclaimed psychic says "I can psychically read Zener cards while blindfolded." We draw 100 known Zener cards and as the psychic to guess them. Then we compare each guess to the card. At the end of the test we can say "The psychic got 70% of these guesses right". In the future, if the psychic guesses at an unknown Zener card, we make an inference: "The psychic has a 70% chance of getting this guess right."

Really?

 

[Cuddles]

Really?

yes.

 

[Philip]

Sorry, skeptigirl and drkitten, if the polygraph is not 100% effective in detecting lies by individuals or if you're unwilling to write an article about the polygraph for Skeptic Magazine or make a presentation about it at TAM, your entire argument is worthless and the polygraph is useless.

 

[CFLarsen]

I did not declare what the actual percentage of valid results were because we don't have the details of the 100 tests by which to determine that including knowledge of the skills of the test administrator. I used 80% hypothetically and provided a source which did a very thorough scientific meta-analysis (GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low. The intent of the person being tested to deceive the tester greatly decreased the reliability of the results among other factors.

This is what you said:

I did but if you didn't catch it, here it is again, the error rate in polygraphs would not be 100%, in fact it would be close to zero. Polygraphs are not reliable enough to convict people in a court of law, nor should they be used to fire people. But if you asked 100 atheists if they believed in god and you got 100 results indicating they were lying, then most of them would be lying. Only a few wouldn't be.

Please provide the scientific evidence that the error rate in polygraphs is close to zero.

This is what this thread is entirely about.

 

[.13.]

yes.

How can you do that?

 

[drkitten]

I don't think you read what I said. I specifically said that I would publish what you wrote, complete unedited.

You did not, however, promise that you would not later distort, misrepresent, or lie about what I wrote.

And even if you made that promise, I have no reason to believe that you would keep it.

How do you know the polygraph is correct any of the time?

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

 

[Jimbo07]

What's your point, Jim?

Let me sum up the point of the thread for you. I made a statement in another thread that even though you cannot use a polygraph result to determine if an individual is deceptive, you can use it to draw a conclusion about a group of people being tested. If the results are that 100 people are being deceptive per the results of the polygraph, and the polygraph is accurate 80% of the time, then you can say that at least 80% of those people are likely being deceptive.

My point is that this is an interesting discussion. This isn't what polygraphs are typically used for, nor to my (albeit limited) understanding, how they're used.

(GAO report to Congress to determine the usefulness of using the polygraph to weed out spies). That source found a range of reliability from very high to very low.

The problem with elected leaders in democracies is that they may be inexpert in things other than how to get elected. Therefore, various branches have to come up with explanations for or against the (possibly random) ideas of those elected officials.

I have no idea how they might be used in a National Security setting. However, in law enforcement I believe that the effectiveness of the machine itself is somewhat irrelevant. I've been watching this thread, hoping that someone with expertise in law enforcement will show up to either confirm or deny this. There are many more factors at play than someone either simply trying to deceive the machine or not. What about those who believe they have done nothing wrong (apparently common)? Interviewing is both a science and an art.

Interestingly, around here, it's the police forces who are using polygraphs as part of their screening processes. I'd bet you a dollar (Canadian currency, now ) that the examiners are more interested in watching the face of the subject than the needle on the kerjigger.

Claus may be trying to make the case that the polygraph is never better than chance. I don't think he has provided any source supporting that claim.

I agree. I also think the claims for evidence at TAM are kinda weird. Is this some kind of controversial debate going on in the States right now? Is there a plan to take incompetent mouth-breathers off the street, put "lie detectors" in their hands and conduct national security screenings? I guess I could make a joke about not being surprised...

 

[Big Les]

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

 

[remirol]

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

I've seen a variant of it that could be called the "gambler's proof": where you ask someone to bet a very large amount of money about the truth/validity of a given issue. Generally, the attacker sets the amount so high such that he can be confident the opposing party either doesn't have that much money available, or won't be willing to risk it on something so (comparably) trivial. It also requires that the validity of the issue in question cannot be determined easily, such that neither party can be certain of the truth prior to accepting the bet.

(Example: "If you're so sure the Patriots will win the Super Bowl, you should be willing to bet your house on it. Let's bet your house against mine, OK?")

In some instances, it can be used effectively to demonstrate the depth of someone's conviction in their stated ideas, but has no bearing whatsoever on determining the validity of the ideas themselves. It is generally a tactic used to attempt to stop discussion of a topic and force a concession to the attacker's viewpoint.

The requests to write a paper and/or present at TAM in this thread are similar.

 

[CFLarsen]

You did not, however, promise that you would not later distort, misrepresent, or lie about what I wrote.

And even if you made that promise, I have no reason to believe that you would keep it.

That would only be evidence of my dishonesty. What have you got to lose?

What about Skeptic Magazine? TAM?

You keep evading this.

Any of the time? Be serious.

We know it's been correct at least three times, because it correctly identified Kari as telling the truth, and Tore and Grant as lying.

Because we know from other sources that they did lie, yes.

I agree. I also think the claims for evidence at TAM are kinda weird. Is this some kind of controversial debate going on in the States right now? Is there a plan to take incompetent mouth-breathers off the street, put "lie detectors" in their hands and conduct national security screenings? I guess I could make a joke about not being surprised...

No, the demands to present a TAM paper/write a skepticreport article are all rather unusual debate tactics that Claus uses in general. Some sort of implied appeal to authority - that the true big name sceptics would laugh the arguer out of the room I suppose. You'd have to ask him.

It's perfectly simple: That's where the world's best skeptics are. It's a unique opportunity to really have skeptics change their minds, based on scientific evidence. If the evidence is there, of course.

As you can see, both Shermer and Bob Carroll argue that polygraphs don't work. Randi does too, of course.

They will certainly be very interested in the evidence. As will the rest of the audience.

 

[drkitten]

Because we know from other sources that they did lie, yes.

Exactly. And that's how the various studies cited in the NAS report determined the accuracy and ROC curves for their polygraphs; they compared the results of the polygraph examination to the ground truths that they knew from other sources. (And, yes, they know about things like "blinding""; the polygraph examiner can't be the one who knows the ground truths.)

This is hardly rocket science; it's standard scientific practice and has been for literally decades. It's also how Randi knows that dowsers don't work -- he knows where the water is and he compares it to where the dowsers tell him that it is.

The difference is, in Randi's studies, the dowsers' performance is generally within the chance expectations --- it has never been beyond the rather stringent alpha cutoff of 0.001. The polygraph tests appear to be well beyond the threshhold of accuracy established by chance.

 

[rwguinn]

skeptigirl and drkitten (and you other supporters of statistical accuracy)

This is all very interesting, and a nice laboratory trick, but Polygraph testing is not done on groups--it is done on individuals.
A single false positive is one too many.

Our system on laws is based on the theory (but not always the practice) that is is better to allow 100 guilty to go free than to imprison 1 innocent.
Polygraph testing has just the opposite effect.

Statistical analysis is wonderful stuff, used properly. Statistically, the probability of you getting injured by a drunk driver after 2:00 AM is .0001 (a number I made up), or 1/10000. Good odds, yes? Unless you happen to be that 1.

 

[Jimbo07]

It's perfectly simple: That's where the world's best skeptics are. It's a unique opportunity to really have skeptics change their minds, based on scientific evidence. If the evidence is there, of course.

As you can see, both Shermer and Bob Carroll argue that polygraphs don't work. Randi does too, of course.

They will certainly be very interested in the evidence. As will the rest of the audience.

Change their minds about what?

Unfortunately, I can't view videos at this time. I'm not sure what position they are arguing, per se. Are they claiming that polygraphs are not useful in law-enforcement? Are they claiming that they are not magic lie detectors?

I guess I'm just coming too late to the issue. I'm still hoping that someone with a law enforcement background will show up and either confirm or deny my (what must seem like a random) claim that: polygraphs are a law-enforcement interview tool (or even technique), not magic lie detectors.