Very hard question in online test: all answers are wrong?

[stevea]

This is soooooo dead obvious I can't understand how it occupies a forum for (discussion of) critical [st|th]inkers. I'm beginning to believe some of you don't belong here.

I believe 0.01/(0.01*0.95+0.99*0.05) = .1695
is the correct answer.

Correct, well its 16.9491525% but w/ a little rounding ....

Can you explain how you got that?

It is GIVEN that we have 99 innocent and 1 guilt party in the sample space.

There is a real question - when the statement is made that the detector correctly classifies 95% of the time we can ask about the "false positive" vs "false negative" rates, but without qualification we must assume that the F+ F- rates are identical (the "ideal" case for a detector design).

Among the 99 innocent we expect 95% correct classification, and among the 1 guilty we expect 95% correct classification. So the classification is

99 innoc. tests as=> (99*0.95)innoc. + (99*0.05)guilty = 94.05%inn 4.95%guil
1 guilty test as => (1*0.05) innoc. + (1*0.95)guilty = 00.05%inn 0.95%guil

So we expect test results of a large population with 1% guilty as:
94.1%innocent-by-test, 5.9% guilty-by-test
So IF we select one person of the 100 at random, then *IF* the sample tests as 'guilty' the conditional probability that they are actually guilty is:

0.01/(0.01*0.95+0.99*0.05)

I'm pretty sure that's just a small error. Perpetual Student is doing what Ivor et. al. are doing, except incorrectly using 0.01, rather than 0.01*95%. It's ignoring the possibility of the detector being wrong about the liar, but only in the numerator.

No ! The numerator '0.01' is the expectation value that a random sample individual is guilty a priori (1 of the 100 is known to be guilty). The denominator is the number we expect to find guilty given the detector error rate. This correctly gives the chance that a random sample individual is found guilty given the detector fallibility.

Yes.

Arriving at the correct answer and then agreeing with your detractors is no better than being wrong in the first place. Join the bartenders - your life is like a box of chocolate.

 

[69dodge]

It is GIVEN that we have 99 innocent and 1 guilt party in the sample space.

There is a real question - when the statement is made that the detector correctly classifies 95% of the time we can ask about the "false positive" vs "false negative" rates, but without qualification we must assume that the F+ F- rates are identical (the "ideal" case for a detector design).

Among the 99 innocent we expect 95% correct classification, and among the 1 guilty we expect 95% correct classification. So the classification is

99 innoc. tests as=> (99*0.95)innoc. + (99*0.05)guilty = 94.05%inn 4.95%guil
1 guilty test as => (1*0.05) innoc. + (1*0.95)guilty = 00.05%inn 0.95%guil

So we expect test results of a large population with 1% guilty as:
94.1%innocent-by-test, 5.9% guilty-by-test

I agree, so far.

So IF we select one person of the 100 at random, then *IF* the sample tests as 'guilty' the conditional probability that they are actually guilty is:

0.01/(0.01*0.95+0.99*0.05)

[quote from Dilb, saying that's not quite right]

No ! The numerator '0.01' is the expectation value that a random sample individual is guilty a priori (1 of the 100 is known to be guilty). The denominator is the number we expect to find guilty given the detector error rate. This correctly gives the chance that a random sample individual is found guilty given the detector fallibility.

You're just asserting this. I don't see any explanation. And I disagree.

The two numbers we should be comparing are the total number of guilty results (in the denominator) and the number of guilty results that are correct (in the numerator). That yields the probability that the subject is guilty, given that the test says he's guilty.

Suppose we test someone whom we know is guilty, and the test says he's guilty. Your method yields 1 / (1*0.95 + 0*0.05) = 1.0526... as the probability that he's guilty, which is obviously wrong. Probabilities are between 0 and 1.

 

[LandR]

The test doesnt work for me in either Firefox or Internet Explorer.

It asks me to drag and drop a few things on a risk bar then goes to a page where it says:

The next task is loading. This may take a little while - please do not refresh the page...

Then does nothing.

 

[ThunderChunky]

No ! The numerator '0.01' is the expectation value that a random sample individual is guilty a priori (1 of the 100 is known to be guilty). The denominator is the number we expect to find guilty given the detector error rate. This correctly gives the chance that a random sample individual is found guilty given the detector fallibility.

No, the numerator is probability that you pick the criminal AND that he fails the lie detector test. The denominator is the probability that the person you pick fails the lie detector. This gives you the probability that someone is a criminal if they were picked randomly and failed the test. That was the question asked. This was explained by multiple people, myself included, on the previous page.

 

[zooterkin]

Not that one, the one whether it was more likely for a coin to come up 6 heads in a row or (I think, I may have read it wrong) for two dice to both come up with 6.

I calculated the coin odds to be about 1.6% (just 0.5^6), and figured that the dice must be 1 in 36, which is about 2.5-3%. So I'm not sure what went wrong.

Wait, maybe I just saw wrong and it was 5 heads, not 6? That might be it.

Yes, it was five, so 1 in 32, as opposed to 1 in 36. Not sure why you'd convert to percentages, that seems to be doing unnecessary work.

 

[Ivor the Engineer]

With reference to the probability tree below, the problem can be split into two stages.

The first stage is the probability of picking either the criminal or an innocent person from the population. As there is 1 criminal and 99 innocent people in the population, the probabilities are 1/100 = 0.01 and 99/100 = 0.99, respectively.

The second stage is the probability the detector pings or stays silent. In the question it states the detector classifies 95% of people correctly. Therefore, the probability the detector pings given the criminal has been selected is 0.95. Similarly, the probability the detector stays silent given an innocent person has been selected is also 0.95.

The probability the detector stays silent given the criminal has been selected is 1-0.95 = 0.05. Similarly, the probability the detector pings given an innocent person has been selected is 1-0.95 = 0.05.

What we're interested in is the proportion of pings from the machine which represent a true positive (i.e. a criminal being tested) to the total number of pings from the machine for all people in the population being tested. This is given by:

P(Criminal given Ping) = P(Criminal)*P(Ping given Criminal) / [P(Criminal)*P(Ping given Criminal) + P(Innocent)*P(Ping given Innocent)]

Which, as others have already pointed out, is Bayes' rule. Plugging in the numbers gives:

P(Criminal given Ping) = 0.01*0.95 / [0.01*0.95 + 0.99*0.05] = 0.16 (to 2sf).

 

[Perpetual Student]

So, am I a bartender whose life is like a box of chocolates? How to decide?

Well, suppose we look at the inverse question (what is the probability the machine pings but we have an innocent person?) and apply Stevea's method and apply Ivor's method. Obviously the results of the two questions should add to 1.

Using Ivor's method, we would get .99(.05)/.059 = .8389 which is the result we would expect, since .8389 + .1610 = 1
Using Stevea's method we would get .99/.059 = 15.25, which cannot even be a probability since it is greater than 1

Sorry, Stevea, you are now relegated to bartender status; enjoy your box of chocolates.

 

[TubbaBlubba]

That's the best layman explanation I have seen on this thread.

Nominated.

It's also something like the third or fourth time it's posted, I posted this on page 1:

I just thought, "OK, in 100 people there will be about 5 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 100/6, or 16-17%."

 

[Perpetual Student]

It's also something like the third or fourth time it's posted, I posted this on page 1:

Originally Posted by TubbaBlubba
I just thought, "OK, in 100 people there will be about 5 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 100/6, or 16-17%."

Sorry, that approach does not work. You were just lucky to get a close result.

Consider 1,000 people with one guilty person. Your approach would yield:

in 1,000 people there will be about 50 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 1000/51, or 19.6%."

Using the correct method demonstrated by Ivor, we would get:

.001(.95)/.001(.95) +.999(.05) = 1.8%

 

[TubbaBlubba]

Sorry, that approach does not work. You were just lucky to get a close result.

Consider 1,000 people with one guilty person. Your approach would yield:

in 1,000 people there will be about 50 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 1000/51, or 19.6%."

Using the correct method demonstrated by Ivor, we would get:

.001(.95)/.001(.95) +.999(.05) = 1.8%

What? Oh, see, I expressed myself very poorly. It's 100 as in 100%, not 100 people. So in your case it would be 1.96%, which is close enough to the true value (and inexact for various reasons). A better way to put it is 1/51, that is, 1 person out of 51, in which case you get it in decimal value.


ETA: Avalon's below edit refers to an earlier version of this post where I managed to misinterpret even my own writing, and I edited this while he was writing his.

 

[AvalonXQ]

Sorry, that approach does not work. You were just lucky to get a close result.

Consider 1,000 people with one guilty person. Your approach would yield:

in 1,000 people there will be about 50 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 1000/51, or 19.6%."

I disagree with your interpretation of TB's approach; I believe the 100 in the numerator is just a way of getting the percentage. I believe that appropriate extension would be:

In 1,000 people there will be about 50 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 100/51, or 1.96%.

... which is, again, pretty close -- still discounting the false negative chance.

EDIT: Apparently it was a genuine error; I still prefer to have given TB the benefit of the doubt.

 

[TubbaBlubba]

I disagree with your interpretation of TB's approach; I believe the 100 in the numerator is just a way of getting the percentage. I believe that appropriate extension would be:

In 1,000 people there will be about 50 false positives and 1 true positive. Therefore, the odds that any positive is a true positive is about 100/51, or 1.96%.

... which is, again, pretty close -- still discounting the false negative chance.

EDIT: Apparently it was a genuine error; I still prefer to have given TB the benefit of the doubt.

No, that is the way I thought of it as well, I just misinterpreted my own writing - which if anything shows how poorly written it was!

Gets a bit confusing with all the edits - I didn't see Avalon's post until I had edited my above post, and he wrote his original while I was writing my original post.

 

[Perpetual Student]

What? Oh, see, I expressed myself very poorly. It's 100 as in 100%, not 100 people. So in your case it would be 1.96%, which is close enough to the true value (and inexact for various reasons). A better way to put it is 1/51, that is, 1 person out of 51, in which case you get it in decimal value.


ETA: Avalon's below edit refers to an earlier version of this post where I managed to misinterpret even my own writing, and I edited this while he was writing his.

Please help me understand this better. Suppose we started with one million people. How would you use your approach in that case?

Never mind! I got it. Ah, yes! Excellent way to get a good approximation. Well done.

 

[P.J. Denyer]

The test is expected to identify 6 people out of the 100 as criminals.

Nope, it's expected to identify 5 out of the 100 incorrectly, that could mean 4 false positives and a false negative.

 

[boooeee]

Nope, it's expected to identify 5 out of the 100 incorrectly, that could mean 4 false positives and a false negative.

Nope. Beth's number isn't exactly correct either, but it's good within rounding.

The expected number of "pings" from the machine is 5.9.

5.9 = 0.95 x (1 criminal) + 0.05 * (99 innocents)

 

[P.J. Denyer]

I understood that, but I was pointing out that it wasn't necessarily 6, 4 was a possible, if less likely outcome. (Although I could have been misinterpreting Beth's comment and if I'd got further through the thread I wouldn't have bothered as it's gone way past my somewhat trivial observation and I'll get me coat...)

 

[69dodge]

I understood that, but I was pointing out that it wasn't necessarily 6, 4 was a possible, if less likely outcome.

In probability, "expected" has a specific technical meaning, which takes into account all the possibilities, each weighted according to how likely it is.