Very hard question in online test: all answers are wrong?

[JJM 777]

An interesting dilemma, I guess this discussion will end with a difference as big as 79% between the opinions of some discussers. You wouldn´t expect that from a math discussion, would you?

Some people want to think straightforward: the test is 95% reliable, so the probability of the reported test result being correct is 95%.

Others think that the 95% probability of the test result being correct somehow _changes_ when we think about how many different people could possibly be included within the 5% of false positives. This viewpoint does not hold water, methinks: the probability of the test being correct is given as 95%, so when you have as many suspects as you ever wish, and you do the test 100 times, 10,000 times or 1000,000 times, you still get the correct indication with 95% of takes and incorrect indication with 5% of takes.

Thus the probability of the indication being correct is 95%.

 

[Perpetual Student]

An interesting dilemma, I guess this discussion will end with a difference as big as 79% between the opinions of some discussers. You wouldn´t expect that from a math discussion, would you?

Some people want to think straightforward: the test is 95% reliable, so the probability of the reported test result being correct is 95%.

Others think that the 95% probability of the test result being correct somehow _changes_ when we think about how many different people could possibly be included within the 5% of false positives. This viewpoint does not hold water, methinks: the probability of the test being correct is given as 95%, so when you have as many suspects as you ever wish, and you do the test 100 times, 10,000 times or 1000,000 times, you still get the correct indication with 95% of takes and incorrect indication with 5% of takes.

Thus the probability of the indication being correct is 95%.

Really? So, if we had a population of, say a million people, with only one guilty person and the single one you happened to choose at random "rang the bell," you would conclude with a 95% certainty that you had the guilty person and that there is only a 5% chance that you chose an innocent person who happened to ring the bell? -- out of a million people? It makes no difference how many people we start with? It will always be 95%? Think man!

 

[Roboramma]

Really? So, if we had a population of, say a million people, with only one guilty person and the single one you happened to choose at random "rang the bell," you would conclude with a 95% certainty that you had the guilty person and that there is only a 5% chance that you chose an innocent person who happened to ring the bell? -- out of a million people? It makes no difference how many people we start with? It will always be 95%? Think man!

Or, as I said, go a step further: what if there are no guilty people?

Imagine, for instance, a test that is 95% accurate at identifying martians. Test someone at random, and the test goes "ping". What are the chances that the person is a martian? By JJM's logic, 95%.

 

[Modified]

Why repeat the test? You already have evidence that the suspect is lying, so the next step to take him down to the police station and beat a confession out of him!

That's not the way it works. First you decide who is guilty based on gut feeling, then you tell them they failed the lie detector, then you beat the confession out of them.

 

[JJM 777]

On second thought, I made an analogy for myself, and realized what exactly is the logical error in this type of evidence.

The test indicates that it is 95% probable that the guilty one is someone like the suspect. (For example: a DNA test might indicate with z% probability that the guilty one is someone with blood like the suspect has.)

This means that with z% probability (in this case, 95%) we have recognized what the guilty one is like. Who the guilty one actually is is a different question and has different probabilities.

 

[BillyJoe]

JJM,

I didn't think it was possible for you make less sense than you previous post.
You proved me wrong.

Please forget about your solution for a moment
Think about how the correct solution is derived:

Here is the problem:

In a room of 100 people only one of whom is a criminal, a machine which has a 95% accuracy indicates that a randomly selected person is the criminal.
What's the probability that the machine has actually picked the criminal?

In other words, what is the probability that the positive result obtained by the machine is actually a true positive?

Here is the solution:

P = P_{true positive}/(P_{true postiive} + P_{false positive})
P_{true postiive} = 1/100 x 95/100
P_{false positive} = 99/100 x 5/100
P = .161016949 or 16.1016949%

Can it be any simpler?
And as soon as you understand that...bonus: you immediately realise why your solution is wrong.

 

[TubbaBlubba]

Let's ignore false negatives and use a slightly different method.

You're a doctor in the 80'ies, you just discovered a useful way to detect AIDS. Problem is, 5% of the times it gives a false positive. It will never, however, give a false negative. In the population you're testing, you know that 1 in 1000 people have AIDS.

If you get a positive result, what is the chance that the tested person has AIDS?

Well, it clearly can't be 95% - the test is 100% reliable when testing an infected person. So we must take into account all people that would come out as positives over 1000 trials, 1000 being the average amount of trials we would have to run to get a person with AIDS.

Chance for true positive given any test = 1/1000 or 0.001
Chance for false positive given any test = 5/100 or 0.05

Thus
P = 0.001/(0.001 + 0.05) =~ 0.02

So in reality, given a positive, the odds that it's a true positive is just below 2%.

I hope this helps.

 

[69dodge]

Chance for false positive given any test = 5/100 or 0.05

That should be (5/100)(999/1000).

(But this has only a very small effect on the final answer, changing it from 20/1020 to 20/1019, i.e., from about 1.9608% to about 1.9627%.)

 

[69dodge]

Others think that the 95% probability of the test result being correct somehow _changes_ when we think about how many different people could possibly be included within the 5% of false positives. This viewpoint does not hold water, methinks: the probability of the test being correct is given as 95%, so when you have as many suspects as you ever wish, and you do the test 100 times, 10,000 times or 1000,000 times, you still get the correct indication with 95% of takes and incorrect indication with 5% of takes.

Yes, you get a correct indication 95% of the time you administer the test. However, in general, you do not get a correct indication 95% of the time that the test says "lying", nor do you get a correct indication 95% of the time that the test says "telling the truth". The 95% figure is just for all tests combined.

Thus the probability of the indication being correct is 95%.

No, because in this problem, we already know that the test said "lying", so to get the right answer, we need to focus only on tests that say "lying", not on all tests regardless of result. In what fraction of tests that say "lying" is the subject lying? That's the answer being sought.

 

[BillyJoe]

Let's ignore false negatives and use a slightly different method.

What difference does it make?
False negatives don't even come into the calculation

Chance for true positive given any test = 1/1000 = 0.001

P_{true positive} = 1/1000 x 95/100 = 0.00095

Chance for false positive given any test = 5/100 = 0.05

P_{false positive} = 999/1000 x 5/100 = 0.04995

Thus
P = 0.001/(0.001 + 0.05) =~ 0.02

P = 0.00095/(0.00095 + .04995) = 0.0186640472

So in reality, given a positive, the odds that it's a true positive is just below 2%.

Actually 1.8664 %

I hope this helps.

Well, I hope it does.
In this case, of course, it didn't make much difference, but that won;t always be the case.

 

[69dodge]

P_{true positive} = 1/1000 x 95/100 = 0.00095

How'd you get that?

In the example, one in a thousand people have AIDS, and for them the test will always come back positive. So it should be simply 1/1000.

 

[Prometheus]

The problem as stated in the OP makes no mention of whether the machine's errors are equally likely to be false positives or false negatives. Wouldn't that make a big difference?

 

[69dodge]

If you get a positive result, what is the chance that the tested person has AIDS?

Well, it clearly can't be 95% - the test is 100% reliable when testing an infected person.

Not sure what you mean here.

If 19 out of every 39 people had AIDS, the answer would be 95%.

 

[69dodge]

The problem as stated in the OP makes no mention of whether the machine's errors are equally likely to be false positives or false negatives. Wouldn't that make a big difference?

Yes, it would make a difference.

It seems reasonable to interpret a statement like "the lie detector correctly classifies 95% of people" as meaning that the lie detector correctly classifies 95% of liers and also 95% of truth-tellers, because otherwise the statement would need to include an assumed distribution of liers vs. truth-tellers in order to give any figure at all. The more liers there are, the closer the overall figure would be to the figure for liers alone and the farther it would be from the figure for truth-tellers alone, and vice versa. Only if the two figures are identical does it not matter how many liers there are vs. truth-tellers.

 

[BillyJoe]

How'd you get that?

In the example, one in a thousand people have AIDS, and for them the test will always come back positive. So it should be simply 1/1000.

 

[psionl0]

Not sure what you mean here.

If 19 out of every 39 people had AIDS, the answer would be 95%.

95.49% to be precise but this is a situation where the number of people with aids is very nearly equal to the number of people without aids. When the proportion of people with aids decreases, so does the probability that a positive reading is a true positive.

 

[69dodge]

95.49% to be precise

How'd you get that? I think it's exactly 95%.

I solved

[latex]$$\frac{p}{p + \frac{1}{20}(1 - p)}} = \frac{19}{20}$$[/latex]​

for p, and got 19/39.

but this is a situation where the number of people with aids is very nearly equal to the number of people without aids. When the proportion of people with aids decreases, so does the probability that a positive reading is a true positive.

Yes, I agree.

 

[rjh01]

Here is a thought. They test the 99 innocent people. About 5 of them will test positive, as well as the 95% of the guilty party. So you end up with about 6 people with positive results. Now you work the rest out.

 

[psionl0]

How'd you get that? I think it's exactly 95%.

Sorry, a typo.
I originally entered
(19/35 *1) / (19/39*1 + 20/39*0.05) = 0.95487

when I should have entered
(19/39*1)/(19/39*1+20/39*0.05) = 0.95.

 

[psionl0]

Here is a thought. They test the 99 innocent people. About 5 of them will test positive, as well as the 95% of the guilty party. So you end up with about 6 people with positive results. Now you work the rest out.

That's the best layman explanation I have seen on this thread.

Nominated.