Polygraphs: The evidence

[digithead]

Doesn't that kind of mean that it works ?

Not with any degree of reliability...

 

[drkitten]

Doesn't that kind of mean that it works ?

Not at all. You see, it's neither perfect nor universal, and therefore it doesn't work at all, under any circumstances.

You should see the state that digithead's house is in. When someone breaks a window, it means his house no longer works, and so he just comes in with a wrecking ball and knocks the rest over.

 

[CFLarsen]

The exact sums are classified, as you might expect. However, the Department of Defense Polygraph Institute routinely gives grants to universities on the order of US $300,000 per year -- and is specificlaly focused, as you might expect, on polygraph technology. More broadly, the NSF/CMI has a specific program dedicated to the identification of non-verbal cues to deception (I could probably dig up the total fund for that, since NSF numbers are not classified, but I won't bother). I believe there there were several tens of millions of dollars of NIJ funding offered a few years back as well (2005?), but I can't find the solicitation as it is no longer active and they do not publish past solicitations as the NSF does.

I just called your bluff, didn't I?

 

[digithead]

The exact sums are classified, as you might expect. However, the Department of Defense Polygraph Institute routinely gives grants to universities on the order of US $300,000 per year -- and is specificlaly focused, as you might expect, on polygraph technology. More broadly, the NSF/CMI has a specific program dedicated to the identification of non-verbal cues to deception (I could probably dig up the total fund for that, since NSF numbers are not classified, but I won't bother). I believe there there were several tens of millions of dollars of NIJ funding offered a few years back as well (2005?), but I can't find the solicitation as it is no longer active and they do not publish past solicitations as the NSF does.

They're no longer called the Department of Defense Polygraph Institute. They changed their name to the Defense Academy for Credibility Assessment. I wonder why?

And NIJ grants rarely go into the millions...

And you can find all of NIJ awards at http://www.ojp.usdoj.gov/nij/funding.htm#post going back to 1996...

In 2005, one was awarded for "Use of Polygraphs to Combat Violence Against Women", BOTEC Analysis Corporation, $324,877, 2005–WG–BX–0010...

In 1997, one was awarded for :Sex Offenders in the Community: The Value of Polygraphs", Kim English, Colorado Division of Criminal Justice, $252,231, 1997-LB-VX-001...

That's it, just two in 11 years for little under $600k, hardly several millions of dollars...

 

[digithead]

Not at all. You see, it's neither perfect nor universal, and therefore it doesn't work at all, under any circumstances.

You should see the state that digithead's house is in. When someone breaks a window, it means his house no longer works, and so he just comes in with a wrecking ball and knocks the rest over.

So I point out that the NAS authors do not share your view that their report was positive or that they (the authors) believe that polygraph has little scientific support so rather than discuss this in an adult fashion, you engage in childish characterizations of my position...

There's a reason why I don't post here all that often and it's because of the childish behavior of some of members. Thanks for reminding me...

 

[drkitten]

So I point out that the NAS authors do not share your view that their report was positive or that they (the authors) believe that polygraph has little scientific support so rather than discuss this in an adult fashion, you engage in childish characterizations of my position...

Yes. You tell lies, and I ridicule you for it.

 

[digithead]

Yes. You tell lies, and I ridicule you for it.

Sorry, I told no lies. The NAS study and its authors do not support your assertion that the CQT polygraph is scientifically based or accurate except that it sometimes guesses correctly in specific incidents but not enough to be useful...

Again, to quote Fienberg et al (2005:22) who are the main authors of the NAS study:

The courts and the legal system should not act as if there is a scientific basis for many, if any, of these uses.

 

[CFLarsen]

Yes. You tell lies, and I ridicule you for it.

Lies?

Conclusions and Recommendations

We have reviewed the scientific evidence on the polygraph with the goal of assessing its validity for security uses, especially those involving the screening of substantial numbers of government employees. Overall, the evidence is scanty and scientifically weak. Our conclusions are necessarily based on the far from satisfactory body of evidence on polygraph accuracy, as well as basic knowledge about the physiological responses the polygraph measures. We separately present our conclusions about scientific knowledge on the validity of polygraph and other techniques of detecting deception, about policy for employee security screening in the context of the U.S. Department of Energy (DOE) laboratories, and about the future of detection and deterrence of deception, including a recommendation for research.
Source

It is a lie that the polygraph has little scientific support?

 

[digithead]

Lies?



It is a lie that the polygraph has little scientific support?

Looking back, I assume drkitten thinks I'm a liar because I made the Popperian mistake of saying "none" rather than "little" on a message board. I suppose at some point, evidence could arise that there is a scientific basis for emotionally-based lie detection. But then I suppose that evidence could also arise that dowsing or telepathy also have a scientific basis...

But in my paper I'm preparing, I said "The NRC (2003) found that the emotionally-based CQT polygraph lacked sufficient scientific validity and that the majority of the research conducted on the reliability and validity of CQT polygraph could not meet minimum standards for National Science Foundation and National Institute of Health funding. It also found that no sufficient estimate of polygraph accuracy could be determined beyond the appearance that the CQT polygraph seems to detect deception at rates greater than chance for incident-specific tests and that increases in the reliability and accuracy of CQT olygraph were unlikely. They also found no support for the use of CQT polygraph in screening situations. From these findings, the NRC (2003) concluded that overconfidence in the polygraph created a significant risk to national security. "

At least I got it correct there. Next time, I'll try to pass my declarative statements on this board through a falsifiablity test. I wonder what the error rate of that is?

 

[Lucky]

I'm not convinced that the polygraphs work.

You have been asked this question a number of times in this thread, and have not given a satisfactory answer:

What do you mean by 'work'?

If you are still claiming that the basic method (using blinded interpretation of the physiological measurements only) cannot be shown to produce significantly better than chance results in laboratory conditions, then that is plain wrong. There really is no debate here: the laboratory data and analysis in the NA report is conclusive on this point. Or are you saying that the accuracy rate is low enough that polygraphy is not (and perhaps never can be) safe and effective in real-world applications? If so, do you see that this is a completely different claim?

Reasonably enough, polygraphy is a matter of great public concern, especially in the US, and there's a lot of debate on the subject on a non-scientific level. Almost everything I have seen (on either side) that's not from scientists is of poor quality. Unfortunately, the above two questions – whether it is a sound theoretical concept, backed up by research data, and whether it produces results at anything like the levels claimed by the proponents (and marketers) - have got hopelessly mixed up.

A quick google turns up many references to organised campaigns to prevent the routine use of polygraphy in a wide variety of situations; for example this and this and this.

Now, I believe that on the whole these campaigns exist for the right reasons – that the technology is not sufficiently accurate for the purposes it is put to, and that (partly due to deliberate misinformation) government agencies, employers etc. have a grossly exaggerated notion of its value – and in general I support them. However, we should recognise that there is also a political element to the anti-polygraphy movement – it has an evident left/liberal tendency:

"Polygraphy is intimidation by state power, a degradation of employee and citizen rights."

"The Defense Academy for Credibility Assessment (the erstwhile DoD Polygraph Institute) suppressed a study suggesting that innocent blacks are more likely to fail the polygraph than innocent whites."

"The ACLU has long favored protective legislation against indiscriminate "lie detector" testing in the American workplace, not only because it is unreliable, but also because it is an extreme invasion of privacy."

Nothing at all wrong with that, but it underlines the point that the campaigners are not primarily interested in the scientific issues (I'm sure the large majority of them have little or no understanding of them).

Polygraphy is one of many policy matters that require the public and its representatives to have a good grounding in the basic scientific issues, and the ability to disentangle them from the social/political ones. You (and many of us here) wish to educate the public in this kind of thinking, but do you acknowledge that we can only be effective by taking every opportunity to discuss and explain the relevant science?

No matter how important the problem in a social or political sense, we should not be trying to convince with slogans, journalistic manipulation and appeals to authority. We must never fudge the argument or ignore unwelcome evidence.

Vague statements like 'polygraphs don't work' are just adding to the confusion.

 

[TV's Frank]

But in my paper I'm preparing, I said "The NRC (2003) found that the emotionally-based CQT polygraph lacked sufficient scientific validity and that the majority of the research conducted on the reliability and validity of CQT polygraph could not meet minimum standards for National Science Foundation and National Institute of Health funding. It also found that no sufficient estimate of polygraph accuracy could be determined beyond the appearance that the CQT polygraph seems to detect deception at rates greater than chance for incident-specific tests and that increases in the reliability and accuracy of CQT olygraph were unlikely. They also found no support for the use of CQT polygraph in screening situations. From these findings, the NRC (2003) concluded that overconfidence in the polygraph created a significant risk to national security. "

Oh, oh, now I'm a co-author!

 

[TV's Frank]

Hey folks, I think we're all having slightly different arguments here, and I'm getting thoroughly confused. Even the NAS authors appear to contradict themselves in different parts of the same paper (which is, unfortunately, not uncommon).

This is a question for digithead (which I'm sure you've answered already):

Is there any situation/scenario in which CQT polygraph has a valid scientific basis, and repeatedly performs better than chance? If so, can you explain why/how? If not, then what explains the strong better-than-chance results in the NAS report discussed earlier?

Thanks for your patience! I'm glad this conversation has become productive again..

 

[Eos of the Eons]

I'm just concerned because the RCMP subject new recruits to their versions of psychiatric testing and polygraph testing. If neither is that reliable, then are they just getting stuck with recruits that are just the best at blowing hot air and great at lying?

 

[CFLarsen]

I'm just concerned because the RCMP subject new recruits to their versions of psychiatric testing and polygraph testing. If neither is that reliable, then are they just getting stuck with recruits that are just the best at blowing hot air and great at lying?

In a word: Yes.

But that's OK. "Polygraphs perform better than chance", so you have nothing to worry about. Nothing!!

It could be interesting to hear from those who are so impressed with the evidence just what they think the polygraphs can be used for.

 

[qwert]

I'm confused.

CFLarsen and digithread seem to be arguing that polygraphs are not practical for use in the real world given that they cannot determine conclusively whether someone is lying or not.

I don't think anyone is arguing against that. I don't see anyone here arguing that polygraphs should be used in a court of law, for national security, employee screenings etc.

The OP dealt with the use of the polygraph on a large population of people. The NAS report indicates that the polygraph works "better than chance" in determining deception.

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

What am I missing here? Do people disagree with the methodology of the NAS report? Are there other reports that refute it? What evidence have I missed?

 

[.13.]

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

The polygraph doesn't show that 80% of the subjects were lying. It shows that 80% of the subjects had a deviation from the baseline. To repeat and rephrase my earlier question which hasn't been answered yet (not directed particularly at you but for the proponents in general):

How can you tell the difference between a lie and a nervous reaction to a question when you don't know what the truthfull answer is?

 

[qwert]

The polygraph doesn't show that 80% of the subjects were lying. It shows that 80% of the subjects had a deviation from the baseline. To repeat and rephrase my earlier question which hasn't been answered yet (not directed particularly at you but for the proponents in general):

How can you tell the difference between a lie and a nervous reaction to a question when you don't know what the truthfull answer is?

Then why do both the NAS report and the Office of technology report (here) indicate that for specific incident investigations the polygraph works better than pure chance.

The OP was provided in the context of a polygraph being used against 100 atheists asked the question "Do you believe there is a god?". The claim was that this resulted in over 80% of them failing the test.

Based on what we know of polygraphs, if this test were actually done appropriately with a competent examiner, we could safely conclude that more than half of those atheists did in fact believe in god.

Likely some of the 80% would be false positives, and some of the 20% would not have been correctly identified, but we could still conclude that more than half of those atheists do believe in god, even though we would be unable to tell specifically which of them did.

 

[digithead]

Therefore, if a large group of people take a polygraph test, and 80% are shown to by lying...we should be able to conclude that more than half of them are in fact lying, even if we can't tell which ones. This makes the polygraph useless in any practical sense, but interesting in that it does perform better than random guessing.

What am I missing here? Do people disagree with the methodology of the NAS report? Are there other reports that refute it? What evidence have I missed?

What you're missing about your conclusion is that you're ignoring the base rate of deception in the population you're testing. Within your hypothetical 80% deception indicated group, you could only conclude that more than half were lying if the base rate of deception was greater than 50%. If the base rate of deception is low, then the majority of your 80% deception indicated group would be false positives...

 

[digithead]

Hey folks, I think we're all having slightly different arguments here, and I'm getting thoroughly confused. Even the NAS authors appear to contradict themselves in different parts of the same paper (which is, unfortunately, not uncommon).

This is a question for digithead (which I'm sure you've answered already):

Is there any situation/scenario in which CQT polygraph has a valid scientific basis, and repeatedly performs better than chance? If so, can you explain why/how? If not, then what explains the strong better-than-chance results in the NAS report discussed earlier?

Thanks for your patience! I'm glad this conversation has become productive again..

No, I don't think that the CQT polygraph has a valid scientific basis but that does not mean it has no utility. It can scare people into confessing things that they otherwise would not admit to. It also can scare people into false confessions...

What we're really talking about here is that the CQT polygraph is simply an interrogation technique rather than any test with diagnostic validity. It has three phases. The first is the pre-test where the polygrapher tries to convince the subject that the method works (i.e., make them fearful). In the second phase, the actual test is given. Lastly, based on what the polygrapher observed both from the physiological responses recorded by the polygraph and the subject's demeanor within the test, the polygrapher further interrogates the subject with such questions as "you're having a little difficulty with question x, is there anything you'd like to get off your chest?" or they let the subject go. A decision of deception is rendered from both physiological responses interpreted by the machine and the demeanor of the subject interpreted by the polygrapher. Anyone else see the problem with that?

As for the NAS concluding that the CQT was better than chance for specific incidents it is because in specific incidents the CQT has a cognitive component to it. That is, questions in the test can be formulated for specific features of the incident which only the guilty party would have knowledge of, ergo cognition increases its ability to detect deception. In these instances, the CQT becomes more like its cousin, the guilty knowledge test (GKT) which is based solely on cognitive response...

Lastly, the studies the NAS relied on were either mock crime lab experiments done on college students or they were observational field studies. The mock experiments suffer from real world validity in that they cannot match the threat of actually being accused of a crime. Observational field studies suffer from selection bias in that most subjects who undergo polygraph testing in a criminal investigation have been identified through other means as potential suspects, increasing the likelihood that they're guilty. If you knew that 80-90% of the people you were going to test were actually guilty, how hard do you think it would be to identify the guilty just through good interrogation alone?

 

[.13.]

Then why do both the NAS report and the Office of technology report (here) indicate that for specific incident investigations the polygraph works better than pure chance.

The OP was provided in the context of a polygraph being used against 100 atheists asked the question "Do you believe there is a god?". The claim was that this resulted in over 80% of them failing the test.

Based on what we know of polygraphs, if this test were actually done appropriately with a competent examiner, we could safely conclude that more than half of those atheists did in fact believe in god.

Likely some of the 80% would be false positives, and some of the 20% would not have been correctly identified, but we could still conclude that more than half of those atheists do believe in god, even though we would be unable to tell specifically which of them did.

Digithead has a point in post #218 about deception rates.

I'm not disputing that you can correlate deviation from the baseline and deception with better than chance accuracy in a laboratory setting when you know what the truthfull answers are.

What I'm asking is how do you distinguish between a nervous reaction and deception in a real life situation when you don't know the truthfull answers?