Poll: Accuracy of Test Interpretation

[Badly Shaved Monkey]

WotS

I think may still have missed it if you have provided a source for the conventional use of the formal term 'accuracy' in the sense in which you have used it, which, unless I am mistaken, is a single variable that can be cited if and only if false positive and false negative rates are the same and then it is numerically equivalent to both. It still looks like a term you have made up on the fly and invoked in order to set a question that actually only required specificity to be specified.

As I have already asked, can you provided a citation to a source in which 'accuracy' defined and used in this particular sense? This would simply let you show that your usage is conventional even if it may be unfamiliar to Rolfe, me and several others here.

 

[MRC_Hans]

I do have more important things to do, you're right. But I'm also a teacher (though I no longer hold a university teaching post, I still teach this subject at post-graduate level).

If I'm failing to get my point across, if there is one of the audience (or it looks like more) who don't understand me, I like to try to do better, one more attempt to make the situation clear.

Rolfe.

For the record, I understood it around the bottom of page 1.

If you test for something rare on a large population, then even a very small percentage of false positives will create more positives than the real positives.

This is basic in all testing. At some point the errors generated by the test surpasses the real faults, at which point it is usually pointless to continue testing.

What I STILL fail to understand is the point of this discussion. What is it's relevance to the real world?

Hans

 

[Brian the Snail]

Looking again at the Wolfram site, I don't think it is about testing at all.

http://mathworld.wolfram.com/Accuracy.html

http://mathworld.wolfram.com/Precision.html

In the realm of laboratory testing accuracy mean the closeness of the mean measured value to the true value of the test parameter measured by some gold standard or known a priori. Precision is effectively the same as reproducibility.

Accuracy as defined on the Wolfram site seems to have the same meaning as what is more commonly called "error" in physics. So if for example I make a measurement of a physical constant, and it has a value 3.0 with an error of 1%, then it means that the true value has a 95% probability of lying in the range 2.97 to 3.03 (assuming that I'm using a 2 sigma error bar). This seems to match your definition of accuracy for medical lab. testing.

WOS seems to be using "accuracy" in a different sense, corresponding in some way to probabilites of finding type I and type II errors in significance testing. However, this seems to me to have a very different meaning to the definition of accuracy contained in the Wolfram site.

 

[MRC_Hans]

Brian, yes that is precisely what accuracy means. What WOTS is talking about is really what I would call predictive value, which is the ability of a given measurement method to predict the distribution of an actual set of values. Of course, this could be termed the overall accuracy of a given instance.

When working with very extreme distributions as in the example he gives, this can give results that are counterintuitive.

Why he feels that this in any way justifies abusive behavior is beyond me.

Hans

 

[steve74]

Wrath, I'm still waiting for the apologies. You need to apologise to the people who've posted in this thread for telling a blatant lie. You also need to apologise to me for calling me a liar.

You got caught telling a blatant lie to hide your discomfort that you had no sources to back up your point. You thought you could bullsh1t the newbie and that I wouldn't check; you were wrong. Admit you got caught out and apologise.

For those who missed it, Wrath's lie:

I finally found the sources that duplicated the question (I even pointed them out, remember?).

 

[Brian the Snail]

Brian, yes that is precisely what accuracy means. What WOTS is talking about is really what I would call predictive value, which is the ability of a given measurement method to predict the distribution of an actual set of values. Of course, this could be termed the overall accuracy of a given instance.

Well, now I'm confused, since WOTS originally posted the link the Wolfram site to show that he's using the term accuracy in the correct sense mathematically. He posted this on page 6, in reply to drkitten who said "I don't think that the term "accuracy" means what you think it does in mathematics, either."

Originally posted by Wrath of the Swarm

http://mathworld.wolfram.com/Accuracy.html

Nice try. Well, it wasn't actually a very good try at all, but you get points for effort. Well, since it was a throwaway attempt at a stab in the dark, no you don't.

But the link seems to define "accuracy" differently to how WOTS defines it.

Perhaps he could clarify why there appears to be a difference.

 

[Prester John]

At the risk of being labelled a groupie, i find that Rolfe is talking to me in the langauge i understand. I work in a lab, always have done and have performed the tests (not that there are many yes/no answers in bacteriology but anyway) of the kind in question.

I think Rolfe nailed it earlier, when she suggested that this may be a pure statistics vs applied medical statistics argument.

Tests are very rarely done in isolation, for screening programmes, there should be a multilayered approach, with the initial test aiming to have a low false negative rate, and a confirmatory test, preferably using a different technology having a low false positive rate.

 

[exarch]

Originally posted by Wrath of the Swarm
The basic point holds even if we don't use an example with identical alpha and beta rates. People aren't able to answer the question correctly no matter what permutations are used.

Wrong WotS, If the alpha or beta error values are 0, the overall "accuracy" as you call it, could still be seen as being 99% (with 1% of the total test population testing either false positive in the former case or false negative in the latter instead of 0.5% of both, or is that 1% of both?).

 

[exarch]

Originally posted by MRC_Hans
For the record, I understood it around the bottom of page 1.

If you test for something rare on a large population, then even a very small percentage of false positives will create more positives than the real positives.

This is basic in all testing. At some point the errors generated by the test surpasses the real faults, at which point it is usually pointless to continue testing.

What I STILL fail to understand is the point of this discussion. What is it's relevance to the real world?

Relevance to the real world? Well, as Rolfe pointed out on page 1, there is no relevance, except that doctors aren't staticticians.

What's really going on is a lot of people getting even with Wrath of the Swarm for constantly nitpicking the slightest details in their posts. Further more, Instead of just admitting right away that his question might have been worded more clearly, he did his usual trick of trying to bluff his way out, and as such started digging himself a hole he couldn't get out of any more without losing face.

People have now caught him on a number of lies, one being his statement that he worded the problem "exactly" as it is being given to medical students (it wasn't, as shown by steve), the other being that the conventional meaning of the word "accuracy" in mathematics is "exactly" the same as the meaning he's claiming it has (also shown not to be true, by the wolfram website no less, a link he provided himself, but obviously didn't bother to read).

Right now, it's up to wrath to either concede and take a big bite of humble pie, or continue his false assertions and discredit himself for all future arguments. Let's just say we're getting pretty close to making a Larssen-list of Wrath's lies. Maybe we should call it a wrath-list? a WotS-list?

 

[Wrath of the Swarm]

I was wrong about the research - at least, I couldn't find sources that used the question in the same way, and I got overeager and thought I had.

The Wolfram definition of accuracy is perfectly consistent with the way I've used the word.

And I'm not bluffing - my argument has been correct. If you were more interested in learning the truth instead of attacking someone you don't like, you'd have realized that by now.

 

[steve74]

I was wrong about the research - at least, I couldn't find sources that used the question in the same way, and I got overeager and thought I had.

The Wolfram definition of accuracy is perfectly consistent with the way I've used the word.

And I'm not bluffing - my argument has been correct. If you were more interested in learning the truth instead of attacking someone you don't like, you'd have realized that by now.

Oh come on now, Wrath, you don't expect me to take this seriously do you? Let me remind you what you wrote:

I finally found the sources that duplicated the question (I even pointed them out, remember?).

Are you seriously trying to say that you genuinely thought that you had found the correct sources and pointed them out, when you hadn't? Do you often have memories of events that didn't happen?
Overeager? Don't make me laugh. You told a blatant lie and then had the audacity to accuse me of lying. Apologise for lying on this thread, Wrath, and apologise for calling me a liar.

 

[Wrath of the Swarm]

I made a mistake. It happens sometimes when I read sources too quickly.

Lies are intentional attempts to deceive. This was just a random act of stupidity on my part. I shouldn't have said I found the particular question until I was sure I had.

 

[Wrath of the Swarm]

With regard to whether the information given in the question was sufficient to answer the question, I have only this to say. Yes, it was fairly easy to make an assumption about what you were getting at and give the answer, as you defined it. But, it is also true that an assumption had to be made and that while this assumption was the most likely one, it certainly wasn't the only way to interpret the question.

Liar.

Your question was underspecified. This has been pointed out to you many times by many posters, all of them far more able than I to make this clear.

Again, liar.

 

[Paul C. Anagnostopoulos]

Wrath said:
I will save my anger for all those who speak against the truth, whether it be Rolfe, Hopkins, or myself.

What you might consider is that the truth is sometimes not so obvious, and even with some give and take may not reveal itself easily.

~~ Paul

 

[Wrath of the Swarm]

What has become obvious is that "bandwagoning", where people take assertions about an argument and run with them regardless of whether they're true or not, is a major problem with this board.

It doesn't matter whether assumptions are necessary for the original question (they are not) or whether this question describes a grossly improbable situation (it does not) -

- people looking for errors so they can attack will seize on the suggestion of error and begin treating it like fact.

I have to say, it doesn't bode well for these boards being able to have rational responses when dealing with potential woos. When a valid argument that appears similar to previous invalid ones comes along, it's going to be as virulently attacked as the nonsense.

 

[geni]

Ok show mathmaticly how you reach a unique solution based on the information avaible in the orginal question.

 

[steve74]

With regard to whether the information given in the question was sufficient to answer the question, I have only this to say. Yes, it was fairly easy to make an assumption about what you were getting at and give the answer, as you defined it. But, it is also true that an assumption had to be made and that while this assumption was the most likely one, it certainly wasn't the only way to interpret the question.

Liar.

Your question was underspecified. This has been pointed out to you many times by many posters, all of them far more able than I to make this clear.

[/b] Again, liar. [/B]

Wrath, time and again you have demonstrated your ignorance. But I have to say I'm still amazed that you don't even know the meaning of the word 'liar'. I made those quotes believing in their veracity. I still believe in their veracity and stand by them 100%.
So how can I be a liar?

Your conduct in this thread has consisted of blatant lies, a complete unwillingness to admit your untruths (until you were forced to do so), shoddy research (at best) and a boneheaded refusal to admit the wrongness of you argument.

You are intellectually bankrupt.

 

[Badly Shaved Monkey]

Accuracy as defined on the Wolfram site seems to have the same meaning as what is more commonly called "error" in physics. So if for example I make a measurement of a physical constant, and it has a value 3.0 with an error of 1%, then it means that the true value has a 95% probability of lying in the range 2.97 to 3.03 (assuming that I'm using a 2 sigma error bar). This seems to match your definition of accuracy for medical lab. testing.

No, I don't think the Wolfram site is using a definition of accuracy that is like thaht used in lab testing. I thought it was from it's first paragraph.

"The degree to which a given quantity is correct and free from error. For example, a quantity specified as 100 ± 1 has an (absolute) accuracy of ± 1 (meaning its true value can fall in the range 99-101), while a quantity specified as has a (relative) accuracy of (meaning its true value can fall in the range 98-102)."

I would recognise this as a definition of accuracy, but the explanation goes on to say;

"The concepts of accuracy and precision are both closely related and often confused. While the accuracy of a number x is given by the number of significant decimal (or other) digits to the right of the decimal point in x, the precision of x is the total number of significant decimal (or other) digits."

Well, this is very different. I don't understand the context in which this would be used at all unless it is a computer programming convention to handle floating point decimals according to rules that depend on the number of decimals used.

Consider that second paragraph.

I weigh a car and find it weighs 1.295 tonnes. The accuracy by Wolfram's definition "is given by the number of significant decimal (or other) digits to the right of the decimal point in x" i.e. 3 places of decimals. If I weigh it on another set of scales that report a weight of 1,294,864.4g. This has only 1 place to the right of the dp. Does that make it less accurate? In truth it neither makes it more nor less accurate because a single number cannot convey accuracy you need a measure of spread. But to use the Wolfram definition again, is 1.295 +/- 0.002 tonnes more accurate than 1,294,864.4 +/- 0.3g.

So I'm still left confused as to what the Wolfram definition, in its full form, is used for.

(Edited for clarity)

 

[Wrath of the Swarm]

Oh brother.

Chance that someone will have the disease: .001
Chance that someone won't have the disease: .999

Chance that the test will give an incorrect answer in any particular circumstances: .01
Chance of correct answer: .99

Chance of getting a false positive: (chance disease-free)(chance of error) = .999(.01) = .00999

Chance of getting a true positive: (chance disease)(chance of correctness) = .001(.99) = .00099

Percentage of correct positive diagnoses: (true positives) / (true positives + false positives) = (.00099) / (.00099 + .00999) = (.00099) / (.01098) ~ .0901639

Conclusion: If the test results come back positive, there is only about a 9% chance that the patient really has the disease.

 

[geni]

Chance of getting a false positive: (chance disease-free)(chance of error) = .999(.01) = .00999

Chance of getting a true positive: (chance disease)(chance of correctness) = .001(.99) = .00099

There is not enough information in the intial question to do the above without making assumptions.