Poll: Accuracy of Test Interpretation

[yersinia29]

We've been over this before, Rolfe.

If the test can be said to have an accuracy independent of a sample population, then we can derive both the false positive and false negatives rates from that value.

No assumptions are necessary.

Wrong.

You assume that "accuracy" = both sensitivity AND specificity.

Most medical tests do NOT have the same sensitivity and specificities, therefore the term "accuracy" is not used very often.

 

[Rolfe]

If the test can be said to have an accuracy independent of a sample population,

But you cannot have your definition of accuracy unless you assume that the same percentage of unaffected people test positive as the percentage of affected people who test negative.

In no other case is there a definition possible.

It's still the same assumption.

Rolfe.

 

[drkitten]

If the test can be said to have an accuracy independent of a sample population, then we can derive both the false positive and false negatives rates from that value.

I believe that at this point either of you can claim a draw under the standard rules of Chess, viz "Repetition of Position."

 

[Rolfe]

Independent variables cannot be inverse, Rolfe. 'Inverse' refers to a specific relationship between the two variables, and 'independent' means the variables are not related at all.

I said independent OR inverse. There is a tendency for an inverse relationship in many cases.

The only thing left is a direct relationship, which is what they don't have. Which is why the idea of assuming that they are the same is so irrelevant.

Rolfe.

 

[Wrath of the Swarm]

Ah, but she's repeating an incorrect position.

Since one aspect of the test that was ESTABLISHED is that it has a general accuracy, we know that the two rates are equal.

We don't need to assume this. It follows from a given in the problem.

The point of the problem holds even if this is not a given. If the two rates are different, the seemingly counter-intuitive result still holds, barring truly extraordinary values.

 

[yersinia29]

Wrath,

In case you missed it, Rolfe already admitted he was wrong about that. I give him/her credit for that.

You on the other hand, despite direct contradiction of your claims, likes to play a shell game and try to weasel your way out of your idiocy.

I can respect somebody who steps up to the plate when they realize an error has been made. I have no tolerance for idiots like yourself who refuse under any conditions to do the same.

 

[Wrath of the Swarm]

You mean like the way I didn't admit that I'd made an error in thinking that the sources I linked to contained the same exact question as the one I asked?

Gee, I guess I *am* arrogant for not admitting that, admitting it was stupid, and apologizing. I'm a terrible, terrible person, yep I am!

 

[Wrath of the Swarm]

I said independent OR inverse. There is a tendency for an inverse relationship in many cases.

The only thing left is a direct relationship, which is what they don't have. Which is why the idea of assuming that they are the same is so irrelevant.

NO!

There are not only three possible relationships. Alpha tends to increase as beta goes down, and beta tends to increase as alpha goes down, but there're other factors involved. The relationship is not linear - therefore it is not an inverse relationship.

Nor are they independent. The value of one is indeed influenced by the value of the other - it's just not determined.

It is NEITHER independent NOR inverse.

 

[Rolfe]

Oh, and PS. I just noticed that I wsa right earlier when I thought I spotted a mistake in BillyJoe's definitions. There is one.

He correctly defines the positive and negative predictive values. If you do the sums he says, those are the values you obtain.

However, he has accidentally described the PPV as meaning the same as specificity, and the NPV as meaning the sensitivity, when in fact the PPV is the percentage of all positive results that are correct and the NPV is the percentage of all negative results that are correct. (Both highly dependent on the composition of the population being tested of course).

I saw something wrong, but I only checked that the defined terms were correctly calculated, not that the verbal descriptions were correct.

End of digression, I just thought I'd mention that one while we're at it. It doesn't affect the validity of his calculations at all.

Rolfe.

 

[Rolfe]

The relationship is not linear - therefore it is not an inverse relationship.

Nor are they independent. The value of one is indeed influenced by the value of the other - it's just not determined.

It is NEITHER independent NOR inverse.

Here, hang on, I think we are agreeing on this one, no need to shout.

The relationship is not inevitably linear, so yes, it's not correct to describe it as "inverse", baldly, just like that.

But it's not independent either. One does influence the other, in an inverse direction.

This is what I was trying to convey when I said independent (or inverse). Some tests more independent, others with more of an inverse relationship. So we seem to mean the same thing.

Consider an ELISA. You have to decide where your absorbance (OD) cutoff will go. Higher than the cutoff is positive (usually, unless you have a descending reaction which is unusual), lower is negative. You fiddle with this to try to optimise the test. Raise the cutoff and you get fewer false positives - but at the price of more false negatives. Lower the cutoff and the opposite applies.

I think this one is fairly linear actually.

Or a Western Blot, where you are recognising characteristic bands in the gel. Require only two matching bands to call positive, and you'll get too many false positives. Require all four, and you'll get too many false negatives. Settle on three for optimum performance.

This isn't linear, but there's still a qualitative inverse relationship.

And some tests have relatively little relationship.

But in no case is the relationship direct.

For this "accuracy" invention to be valid, you'd need to have tests that vary in exact direct relationship, and the thing is, they don't.

Rolfe.

 

[steve74]

You mean like the way I didn't admit that I'd made an error in thinking that the sources I linked to contained the same exact question as the one I asked?

Gee, I guess I *am* arrogant for not admitting that, admitting it was stupid, and apologizing. I'm a terrible, terrible person, yep I am!

Wrath, you only admitted that when I made your position untenable by pointing out the blatant falsehoods in your posts. You would have continued to peddle those falsehoods had I not called you on them. In fact, the first time I called you on them you lied to try and cover yourself. Here, for those who missed it (and let's face it there is a lot to get through here) is Wrath's lie:

I finally found the sources that duplicated the question (I even pointed them out, remember?).

Only when I called you on this did you back down and admit that you had no sources to back up your question, even though you had earlier claimed:

Point 1: The question, as I presented it, is the same question that was used in research with doctors.

Wrath, you have lied in this thread. That is plain for all to see. You have accused me of lying, which is false. You owe me an apology and you owe an apology to the people who have posted in this thread for trying to hoodwink them with your lies.

 

[Wrath of the Swarm]

The relationship is not inevitably linear, so yes, it's not correct to describe it as "inverse", baldly, just like that.

But it's not independent either. One does influence the other, in an inverse direction.

This is what I was trying to convey when I said independent (or inverse). Some tests more independent, others with more of an inverse relationship. So we seem to mean the same thing.

The preceding statements, considering in a mathematical light, are utter garbage. They mean nothing.

Two variables cannot be "more independent": either they are, or they aren't. Likewise, they are either linear, or they aren't.

There is a tremendous difference between an inverse relationship and an inverse proportion. Unfortunately, as you never learned either of these concepts (passing by rudimentary mathematics on your way to learning "medical statistics", perhaps?) you have no idea that utter and complete rubbish is spilling out of your mouth.

 

[Wrath of the Swarm]

In response to steve:

Yes, I was wrong. I admitted this as soon as I realized it was the case. I was sloppy and stupid, and didn't read the sources I'd found carefully enough. They contained much of the information originally presented to me years ago in college courses, but not the one piece of information I was referring to: the formulation of my question.

However, you have made further claims which are mistaken. Furthermore, even when I pointed out to you that they were mistaken, you continued to make them. When you pointed out my error to me, I admitted I'd made a mistake. You lied and repeated the falsehoods.

I am merely a fool. You are a fool and a liar.

 

[Wrath of the Swarm]

But in no case is the relationship direct.

For this "accuracy" invention to be valid, you'd need to have tests that vary in exact direct relationship, and the thing is, they don't.

No; you don't get it. The two variables do not need to vary in a directly proportional relationship - they merely need to be equal.

In the hypothetical example, alpha happens to equal beta. That's why it's possible to give the test an accuracy - its rate of error is independent of the nature of the population.

How many times have I repeated this? How many times have you ignored it?

You're in way over your head, Rolfe.

 

[Rolfe]

The preceding statements, considering in a mathematical light, are utter garbage. They mean nothing.

Two variables cannot be "more independent": either they are, or they aren't. Likewise, they are either linear, or they aren't.

There is a tremendous difference between an inverse relationship and an inverse proportion.

All right, if I've been loosely using terms that mean very specific things to you, I'll endeavour to rephrase. If you will hang off the insults long enough to inform me of the phraseology you find acceptable, I'll endeavour to use it.

I've described two examples where as a rough tendency, efforts of the test developer to improve sensitivity will have an adverse effect on specificity, and vice versa. In one case it's a matter of choosing an absorbance cut-off, in the other it's a matter of choosing the number of gel-band matches that gives optimum performance.

All I can say about the - er, relationship? - between sensitivity and specificity in these cases is that as one improves, the other tends to deteriorate. Without having very exact data about the individual assay in question, no more can be said (and in fact it's only a generality that I'm trying to establish). The one thing that we can say is that an alteration aimed to improve sensitivity will at best leave the specificity unchanged. And vice versa. The one thing that can't happen is that a change in cut-off (or choice of matches) will simultaneously cause an improvement in either value. Most probably there will be some sort of see-saw effect, of variable and unpredictable - er, relationship?.

Now it's taken me quite a lot of words to try to explain the situation without outraging your sense of propriety by misusing a term you want to reserve to a specific definition. So what words would you use to get this point over without typing three paragaphs?

Rolfe.

 

[Wrath of the Swarm]

All right, if I've been loosely using terms that mean very specific things to you, I'll endeavour to rephrase.

These are not personal definitions. They have generally accepted meanings in mathematics - and if we put those meanings into your sentences, we find that they don't make any sense.

If you will hang off the insults long enough to inform me of the phraseology you find acceptable, I'll endeavour to use it.

You're an expert, remember? You've written a book? Surely you can use the mathematical terminology correctly, yes? Why haven't you done so, Rolfe? Are you playing with us? Is that it?

Oh, right. You don't have the slightest idea what you're talking about other than the concepts you've learned by rote.

 

[steve74]

In response to steve:

Yes, I was wrong. I admitted this as soon as I realized it was the case. I was sloppy and stupid, and didn't read the sources I'd found carefully enough. They contained much of the information originally presented to me years ago in college courses, but not the one piece of information I was referring to: the formulation of my question.

However, you have made further claims which are mistaken. Furthermore, even when I pointed out to you that they were mistaken, you continued to make them. When you pointed out my error to me, I admitted I'd made a mistake. You lied and repeated the falsehoods.

I am merely a fool. You are a fool and a liar.

Wrath, you really are thick aren't you? I mean mind-numbingly, jaw-droppingly stupid. I have already pointed out to you once in this thread that you do not appear to know the meaning of the word 'liar'. It appears you still don't get it.

All I have done in this thread is post facts and opinions that I believe to be correct. I still believe everything I have posted in this thread is factually correct. So, I ask you once again (you failed to answer last time), how can I be a liar?

You, on the other hand, have blatantly lied. You claimed you had found studies and posted links to them. You did not. Is it really plausible to believe that you had a memory of finding the correct studies and posting links to them? Do you often have false memories like this? If so, I suggest you seek help from a mental health professional.

Wrath, you are either a liar or suffering some sort of mental problem. Which is it?

 

[ceptimus]

Yes. 99% accurate means exactly what it says:

99 out of every 100 tests give the correct answer.

Period. That's all you need to know. You don't need any alpha or beta in this case - you don't need to know what proportion of the population has the disease to understand this fact.

The anti-Wrath crowd may continue to bluster and obfuscate, but this clear and simple fact will continue to stick out like a very sore thumb.

 

[geni]

Yes. 99% accurate means exactly what it says:

99 out of every 100 tests give the correct answer.

Period. That's all you need to know. You don't need any alpha or beta in this case - you don't need to know what proportion of the population has the disease to understand this fact.

The anti-Wrath crowd may continue to bluster and obfuscate, but this clear and simple fact will continue to stick out like a very sore thumb.

Yes but you can get that level of correct tests (ok a bit higher) by haveing the test never giving a false posertive but almost always giving a false negative.

 

[Rolfe]

No; you don't get it. The two variables do not need to vary in a directly proportional relationship - they merely need to be equal.

Yes, I do get it.

The point is that for this to pertain in any general sense, you'd have to be dealing with tests where an improvement in sensitivity automatically improved specificity along with it, to keep the two the same.

But we are dealing with systems where improvements in sensitivity tend to cause deteriorations in specificity, and vice versa. Which is why the term is meaningless.

You expected an assumption to be made. That's what we're discussing.

Now I admit that I made a mistake. Sensitivity does have a small effect on predictive value. I should have agreed with you when you maintained that the assumption had to be that both values were 99%.

I made a mistake when I presented you with the easy get-out of saying "I just meant specificity". This was wrong. You were correct to keep asserting that the assumption you wanted made was that both values were 99%.

But it's still an assumption which must be made before any attempt can be made to solve the problem. Which is what we were talking about in the first place.

Rolfe.