Lucky,
Their conclusion about the scientific evidence for the polygraph was that one cannot generalize from any results obtained from subjects in lab or field studies because there is no good scientific theory of why the CQT polygraph works the way it does in the lab. Additionally, there are so many potential confounders (i.e. conditions that have a similar physiological response as deception) for the CQT polygraph that haven't been properly studied that it is unknown how those confounders affect whatever lab accuracy CQT polygraph may have shown.
Is lying correlated with nervousness and emotional response? Yes, but it is not a 1 to 1 correlation. CQT polygraphy in my opinion makes the fallacy of composition. It is pseudoscientific to assume that all liars are nervous and can be detected from physiological response because some liars are nervous and can be detected by physiological response.
Additionally, when I worked in environmental science, we had immunoassay soil test kits that could detect certain chemicals down to the ppm level (pentachlorophenol I think but it's been 15 years). One of the caveats was that the test did not work in soils with high levels of certain heavy metals (iron I think) as it rendered the test useless. Would it be scientific to use that test anyway in an environment with high concentrations of the confounding element and report the results despite the fact we knew they were useless? No, it would be pseudoscientific to use it that way and that's why CQT polygraph is considered pseudoscientific. It's how and when it's used, not that it might be able to detect lies above chance levels in the lab.
Now I've gotta get back to my other stuff. School started yesterday and I've got lectures to prepare. I've also neglected my dissertation for a week and I got the evil eye from my chair this afternoon...
But I have been given food for thought on my paper on the polygraph and how to better argue against its use in sex offender treatment...
That's why I love being on the JREF forum. When the dialogue and arguments are respectful, one can learn a lot from some very smart people, even if there's disagreement...
Regards...
digithead, I am enjoying this discussion, too. I appreciate your posts, which have given me a lot of information and also prompted me to do some further reading and thinking. My interest in the subject of polygraphy is not with its application, but its relevance to the issues of validity and accuracy in different kinds of population testing.
My main disagreement with you is over the lack of a precise understanding of the causes of the response patterns, and whether this is important. I would say that it is a side-issue, and the real objection to use of the polygraph is operational, having to do with the poor results we can expect in real-world applications. However, in itself low accuracy need not rule out useful applications – the reasons why I believe that polygraphy is a special case, where low accuracy can be expected to rule out any valid, widespread application, are a subtle mix of the scientific and the social.
It is very instructive to compare polygraphy with a medical screening test, say a serum test for a tumour marker. These typically have high detection rates but also fairly high false-positive rates, so can't be used as diagnostic tests – a positive result will be followed by a biopsy or equivalent. There is nothing at all unusual about medical screening tests with ROC results similar to those for the polygraph. Nor is it necessary to have an evidence-supported theory of how/why the test works. (In some cases the analyte's usefulness as a marker is a chance finding, and we may have no knowledge of the underlying mechanism.)
Here's a typical study of a potential
multi-marker urine screening test for various cancers. There isn't a 1:1 correlation between tumour stage and marker levels, and the pattern of levels in different cancers and at different stages is not understood. But the technique is likely to be useful.
Generally, in a screening application we are using a cheap method of limited accuracy to reduce a large population to a small group that requires further, more expensive investigation. An important corollary is that we are screening people
in (for further attention) rather than
out (rejecting them).
There will be both false positives and false negatives, which can have bad consequences for both the individual and the screening programme itself. The consequences of a false negative to an individual should be no worse than not taking the test, and the non-negligible rate of detection failures must not invalidate the programme. The consequences of a false positive will tend to afflict individuals rather than the screening programme itself, and they
are likely to be worse than not taking the test (unnecessary anxiety, at least).
The general principle must be that the sum of the beneficial effects to the subjects and to the organisation doing the testing (in relation to the purpose of the programme, e.g. public health) must outweigh the sum of the harmful effects to the subjects, and to the organisation, from incorrect results. Also, the test must work better than any cheaper alternative (or, conversely, be cheaper than a more accurate alternative).
I'd say the difference between low-accuracy medical screening and low-accuracy polygraphy is simply that these conditions are frequently fulfilled in medicine, because screening is cheaper than offering a diagnostic test to everyone and better than doing nothing, but they are unlikely to be fulfilled by the polygraph. First, it is not cheap. Second, the problem of false negatives may invalidate the programme, especially because confounding factors are likely to be severe (there may be no solution to the problem of criminals being trained to fool the test). Third, the consequence of a false positive to the individual is likely to be very severe.
The other point I would make is that invalid application, exaggerated claims, error, and even fraud, do not constitute
pseudoscience (which is a useful term if we don't broaden it to include any type of poor science).
I think it is incorrect to label any technology a 'pseudoscience' if it can be shown to have
some area of validity, even if the area is limited and the accuracy is low, and even if we don't understand how it works - and I would suggest you don't use the term in your paper. As an interested reader from a field with some similarities, it would jar, and perhaps make me think you have an axe to grind.
I am grateful to you for prompting me to develop and set down my ideas on the utility of screening programmes (and, I have to admit, to CFLarsen for starting the thread).
I'd be very interested to read your paper when it's finished.
