Poll: Accuracy of Test Interpretation

Wrath of the Swarm said:
That's the whole point of the question - it's counterintuitive.
Click to expand...
And that's the point I'd have liked to pursue from the start - that it's only counter-intuitive if you load the question in a specific way.

However, your ridiculous insistence in porting this "accuracy" concept from whatever field you do have familiarity with to a field you clearly understand little about has cause this discussion to stall for about six or seven pages.

What use to man or beast is a term which can only be applied to the rare and flukey situation where sensitivity and specificity happen to be equal?

Rolfe.
 
Rolfe said:
I'm not throwaway stabbing in the dark, I'm simply requiring you to do what I require any student to do before discussing the topic. Understand the definitions of the terms used, and be able to define mathematically with reference to these four figures what any other term they want to use actually means.
Click to expand...

Those four concepts are derivable from the provided information in this particular case. This case was chosen because it was a simple example - in reality, things are often more complicated, but it should be easier to answer the question than it would be otherwise.

Doctors generally deal with more sophisticated forms of error than we were here. That doesn't mean the simpler form of error is wrong or invalid in any way.

Rolfe doesn't understand these points in a mathematical sense. She learned the procedure doctors need to know and memorized it. Now she can plug values into the algorithm and generate results, but she doesn't understand what she's doing at all.
 
exarch said:
You're right. if the accuracy is the occurence of false positives, it can't be.
Ceptimus, this whol thing hinges on WotS's misunderstanding between "accuracy" and "specificity" (occurence of false positives).

WotS's test is in essence only 9% "accurate" at positively identifying someone as having the affliction, not 99%.
It is 99% specific though.
Click to expand...
Exarch, please read the bit that BillyJoe provided. The useful data is getting buried under Wrath's ad-homs and blind assertions here, and I don't have time to type it all again.

You have to be able to define how you derive any numerical description of the test you are using, in terms of only four variables you derive when you evaluate the test (TP, FP, TN and FN). "Accuracy", if you want to use the term, has to be defined in that way too.

Wrath's usage can't be defined like that, because it is only applicable to a particular subset of situations where TP / (TP + FN) happens to come out equal to TN / (TN + FP). Which virtually never happens in practice.

You know how Geni says, give me the maths for that? Well, this is a situation where you have to be able to "give the maths" for how you derive any attribute of the assay you quote. If you can't, don't quote it.

Rolfe.
 
Wrath of the Swarm said:
Rolfe doesn't understand these points in a mathematical sense. She learned the procedure doctors need to know and memorized it. Now she can plug values into the algorithm and generate results, but she doesn't understand what she's doing at all.
Click to expand...
Wrath, you don't have a bloody clue what you're talking about, do you?

Rolfe.
 
WOTS: I will give you one more try. What is you point here? I don't give a d*mn what you think about Rolfe, it is simply not interesting. Do you have some point about medical testing (other than extreme figures yield extreme results)?

Hans
 
Rolfe said:
And that's the point I'd have liked to pursue from the start - that it's only counter-intuitive if you load the question in a specific way.
Click to expand...
No, the general result is counter-intuitive. We don't need to "load" the question at all - tests performed by doctors are rarely even 99% accurate, and they frequently involve relatively uncommon conditions.

Even considering that lab tests are often ordered because of worrisome symptoms (with the identification of such symptoms serving as a crude pre-test which alters the population sample), such tests have significant margins of error. This needs to be acknowledged and understood.

What use to man or beast is a term which can only be applied to the rare and flukey situation where sensitivity and specificity happen to be equal?
Click to expand...
Mathematicians seem to find it useful indeed.

In a broader sense, we can even discuss accuracy when alpha is not equal to beta, but we can't use that value in discussions of cases where the subject falls into a specific category. It doesn't help to know that a test is 99.9% accurate in general if it's only 20% accurate at detecting a very rare condition and we're concerned that a person might have have that condition.
 
Originally posted by slimshady2357
This thread has been bizarre, to say the least.

I would say my reaction to the initial post was very similar to exarch's and ceptimus'. I just assumed he meant the error rates were the same. To tell the truth it seemed obvious to me.

It's interesting that you see people come into this thread and insult WOS for various things, most of which amount to insulting him for continuing on this ridiculous thread for so long. But not many people are saying much about Rolfe continuing the thread for just as long....
Click to expand...
The only problem I see is Wrath of the Swarm immediately attacking Rolfe for pointing out an innacuracy in the wording of the problem. And his insistence that the wording of his problem was not wrong, and that he had proof of this, but that Rolfe doesn't know what she's talking about, being one of the "ignorant doctors".

I would have let it slide if it had been anyone else, probably because anyone else would have conceded right away that Rolfe actually has a point, and not have started insulting either. The problem I have is with WotS's attitude. The holier than thou "I know it better than all of you", and his unwillingness to admit even a small mistake. He's very quick to point out even the tiniest problem in everyone else's posts though, but he gets extremely upset if you do the same to him. My last 8 or 9 posts in this thread are just an attempt to toss more oil on the fire. Maybe rubbing it in might humiliate him just enough to think twice in the future before immediately ad-homming everyone who dares to disagree, falsely assuming he himself doesn't make mistakes.

It's extremely annoying and it doesn't further the conversation one bit. Which has been shown extensively on this thread I think :), where everyone else is nitpicking HIS post and HIS statements.

I doubt he'll learn though.
 
MRC_Hans said:
WOTS: I will give you one more try. What is you point here? I don't give a d*mn what you think about Rolfe, it is simply not interesting. Do you have some point about medical testing (other than extreme figures yield extreme results)?
Click to expand...
The first point is that people aren't very good at interpreting the results of statistical tests - even people who do so regularly and who make life-or-death decisions on the basis of such tests.

The second point is that doctors generally do not have that great an understanding of what they do, or the strengths and weaknesses of the methods they were taught. As I pointed out quite some time ago in this thread when someone asked me this question previously, that's why doctors aren't very good at detecting when there's something wrong with treatments.

For treatments that can be carefully tested by people not prescribing them (like drugs), it's a bit easier. Surgical treatments are especially known for being used without strong experimental support for their safety and effectiveness, and many turn out to be useless or even harmful.

I remember reading an article in Discover magazine some years ago about a surgical treatment for strokes that involved rerouting blood through veins. It was considered an effective and appropriate treatment, and it was used for something like thirty years. The author even discussed practicing the complicated procedure of tying off and reconnecting vessels on rats in school. Then a comprehensive study was conducted to see if patients who had the procedure were better off. Not only did they not improve, the procedure turned out to increase the rate of further strokes significantly.

The procedure was abandoned, but the author's point was that for decades, everyone was convinced that it was helpful and beneficial. Surgeons swore by it. And the whole time it was worse than doing nothing at all.

I don't believe the article is still online. If you search through library archives, you can probably find a copy of it.

I offer my memories of this article as an example of what I'm talking about.
 
exarch said:
The only problem I see is Wrath of the Swarm immediately attacking Rolfe for pointing out an innacuracy in the wording of the problem. And his insistence that the wording of his problem was not wrong, and that he had proof of this, but that Rolfe doesn't know what she's talking about, being one of the "ignorant doctors".
Click to expand...
But that's my point - it's not an inaccuracy.

If I had merely made a mistake, I would have accepted it. But Rolfe's claims are wrong.

And for the record, I find Rolfe's "I'm an expert, so if what I say is nonsensical or contradicts basic logic, you must just be too ignorant to understand" attitude to be completely offensive. So is her attitude that any criticism toward any aspect of medicine is an attack on the field as a whole and its practitioners in general.
 
MRC_Hans said:
WOTS: I will give you one more try. What is you point here? I don't give a d*mn what you think about Rolfe, it is simply not interesting. Do you have some point about medical testing (other than extreme figures yield extreme results)?
Click to expand...
Hans, there is a point. I'd like to get to it. In fact it is what Wrath touches on in the post immediately below yours here.

Now, I might see a way through.

Wrath. Do you accept that the sensitivity value of the test in question is irrelevant to the problem you posed?

Do you accept that you intended it to be assumed that the specificity value was 99%? (Seems to follow, if you intended it to be assumed that specificity was equal to sensitivity and that both were equal to 99%.)

Do you therefore accept that I don't honestly care what the sensitivity is? Not for the purpose of your question. You can have it as 99% if you like. Or 100%, or 20%. I don't care because it doesn't matter.

So can we continue on the assumption that you want us to know that specificity is 99%?

To go on stalling just because you also want to assert that the sensitivity is 99% too (even though the figure is irrelevant to the calculation), seems rather sterile by now.

Rolfe.
 
Having trawled these many pages of semantic tittle-tattle, I still find the need to post the same comments as I would have done about 4 pages ago!

I do not claim expertise in this field, but I have never heard or read the term 'accuracy' used in the context of diagnostic testing except and only in the casual use of the word as a qualitative judgement.

Rolfe has beaten me to the point that 'accuracy', with 'precision', has an exact meaning in measurement of numerical variables.

Diagnostic testing, reported as 'positive' or 'negative' does not employ the term 'accuracy' among its jargon of specificity/sensitivity, NPV and PPV and that jargon can completely define the behaviour of a diagnostic test without resort to use of the term 'accuracy'.

WotS' link to a Wolfram site rather confirmed the use of 'accuracy' in the sense described above and not as it relates to binary diagnostic testing in which the values are counts and proportions not continuous variables.

I stand to be corrected, but what it would require is a reference explicitly and formally linking specificity and sensitivity and something called accuracy.
 
Wrath of the Swarm said:
The first point is that people aren't very good at interpreting the results of statistical tests - even people who do so regularly and who make life-or-death decisions on the basis of such tests.

The second point is that doctors generally do not have that great an understanding of what they do, or the strengths and weaknesses of the methods they were taught. ....
Click to expand...
Oh wow, no agenda here at all.

Since these are all asssertions, which Wrath would like to get to by his selective use of statistics and rigged scenarios following his own little train of thought, can we leave them as points still to be demonstrated, or not, as we discuss the wider implications of the test?

Wrath has frequently told us that he is annoyed that the "accuracy" question was holding up what he wanted to do with the thread, which was make doctors look stupid. I can see how he wants to do it, but I think he has to actually do it rather than just assert "this is what I will prove".

After all, it might not be entirely as he imagines.

Rolfe.
 
exarch said:
The only problem I see is Wrath of the Swarm immediately attacking Rolfe for pointing out an innacuracy in the wording of the problem. And his insistence that the wording of his problem was not wrong, and that he had proof of this, but that Rolfe doesn't know what she's talking about, being one of the "ignorant doctors".

I would have let it slide if it had been anyone else, probably because anyone else would have conceded right away that Rolfe actually has a point, and not have started insulting either. The problem I have is with WotS's attitude. The holier than thou "I know it better than all of you", and his unwillingness to admit even a small mistake. He's very quick to point out even the tiniest problem in everyone else's posts though, but he gets extremely upset if you do the same to him. My last 8 or 9 posts in this thread are just an attempt to toss more oil on the fire. Maybe rubbing it in might humiliate him just enough to think twice in the future before immediately ad-homming everyone who dares to disagree, falsely assuming he himself doesn't make mistakes.

It's extremely annoying and it doesn't further the conversation one bit. Which has been shown extensively on this thread I think :), where everyone else is nitpicking HIS post and HIS statements.

I doubt he'll learn though.
Click to expand...

While I agree with most of your post, and (Mandy forgive me!) I even support your oil throwing in a way :D, I do think you're missing something.

You see, everything Rolfe has said was said by page one or two. The fact that Rolfe is also willing to continue this nonsense for 6 pages does not reflect well on him/her either. Why continue? What is the point?

But then again, why am I still reading this thread? Why am I posting to it? Hmmmmmmmmmmmmmmmmm........

Die thread, DIE!

Adam
 
p.s. I did get WotS' original question right even though I am only an idiot clinician.

p.p.s. WotS, you've been asked before and I still haven't seen an answer, or I may have missed it in all the pages of bickering. Please explain what you mean by alpha and beta probabilities and tell us from what field this is standard jargon, because it isn't standard for me. I wondered whether you were talking about alpha and beta error rates in relation to hypothesis testing, but that doesn't seem to fit your usage.
 
Originally posted by Wrath of the Swarm
http://mathworld.wolfram.com/Accuracy.html

Nice try. Well, it wasn't actually a very good try at all, but you get points for effort. Well, since it was a throwaway attempt at a stab in the dark, no you don't.
Click to expand...
Accuracy

The degree to which a given quantity is correct and free from error. For example, a quantity specified as 100 ± 1 has an (absolute) accuracy of ± 1 (meaning its true value can fall in the range 99-101), while a quantity specified as 100 +/- 2% has a (relative) accuracy of +/- 2% (meaning its true value can fall in the range 98-102).

The concepts of accuracy and precision are both closely related and often confused. While the accuracy of a number x is given by the number of significant decimal (or other) digits to the right of the decimal point in x, the precision of x is the total number of significant decimal (or other) digits.
Click to expand...
Odd. I don't see anything regarding statistics, false positives or alpha/beta here :confused:

Accuracy also appears nowhere on this page :p
 
'Accuracy' has a meaning in terms of the mathematics that underlie medical testing. Whether the term is ever brought by in the field of medical testing or not is irrelevant. It has a simple, easy-to-understand meaning that can be applied in this example with a minimum of thought.

Rolfe: You can't weasel out that easily. You insisted that the question could not be answered without my explicitly providing the rate of Type I error (although you used different terminology) and proclaimed that my question was ambiguous because of my ignorance.

The data I gave you was more than sufficient to answer the question, and you know it. Your mealy-mouthed evasions wouldn't be nearly as mealy if you weren't aware of how deep a hole you've dug yourself into.

You're simply wrong, and no amount of complaints about "ad homs" (they're insults, toots, not ad hominem attacks, not that you can tell the difference) or attempts to troll for sympathy as a poor little misunderstood fighter for skepticism and justice will change that.
 
Looking again at the Wolfram site, I don't think it is about testing at all.

http://mathworld.wolfram.com/Accuracy.html

http://mathworld.wolfram.com/Precision.html

In the realm of laboratory testing accuracy mean the closeness of the mean measured value to the true value of the test parameter measured by some gold standard or known a priori. Precision is effectively the same as reproducibility.

Maybe I am labouring under a misapprehension here and Rolfe can correct me, but if I am right the Wolfram citation is some very narrow mathematical programming use of these terms not how they are used in statistics generally.
 
slimshady2357 said:
You see, everything Rolfe has said was said by page one or two. The fact that Rolfe is also willing to continue this nonsense for 6 pages does not reflect well on him/her either. Why continue? What is the point?
Click to expand...
First, Wrath is also continuing with the same assertions he was making on about page 2. Just avoiding defining what he means. While he continues, why should I not continue?

This is my academic subject. Wrath has continually asserted that I am stupid, and ignorant, and worse. I feel that to retire while he is still taking that attitude is to concede to him, and I will not do that. He is trying to pontificate on a subject he is not familiar with, and I will not have it appear by my retiring that I give the slightest acknowledgement to his ill-conceived and ill-thought-out arguments.

Also, there is a more interesting discussion waiting in the wings. One which may make this look like a tea-party given Wrath's temperament, but interesting nonetheless.

Now if we can only agree that Wrath meant to imply that both specificity and sensitivity were 99%, and as sensitivity doesn't matter an iota for the purpose of the problem he sets, he might as well have said specificity is 99% from the start, we migth just manage to go on.

Rolfe.
 
You must log in or register to reply here.

Back
Top Bottom