Can you show me where (in other than this forum) it says that you can't guess at both the priors and the likelihood ratios?
Why are we excluding material presented as part of the debate already? If I point to where that was stated in this forum, it would be the third time I've had to refer you to it, and likely the third time you've ignored it. Your
ad hoc filter is extremely revealing because I'm sure you remember that the restriction I'm talking about was presented by someone whom you have previously considered an authority and whose wisdom you have previously solicited. How disingenuous of you now to exclude his advice when it undermines your attempt.
You can put any numbers you want into any formula, including Bayes' theorem, and come up with a result. Whether that result means something, however, depends on whether one has provided something other than idle guesses. The same is as true for Bayes as it is for any other formula. You have been suckered in by the fringe belief that Bayesian methodology lets you assign an arbitrary probability to any old belief and then, somehow, obtain a fact from it. As I have explained countless times, it does nothing of the sort. You can certainly guess at your priors
and also guess at your likelihood ratio. But what you may not do under those circumstances is pretend that the number that falls out of the bottom of such a computation has any more significance than mindless arithmetic gristmilling. It has the significance you require only when you have given proper obeisance to the significance of the elements of your model.
Garbage in, garbage out.
To point you to where, outside this forum, you can learn about the constraints on statistical inference, I would literally refer you to practically any introductory statistics textbook. You have frankly admitted that you don't understand the Bayesian model or the Bayesian methdology. So it's exceptionally rude and hubristic of you to demand that your critics cow-tow to your remediation.
Nevertheless here it is. You may thank me later for indulging your ignorance and for saving you a trip to the library.
Reliable inference relies upon the degrees of freedom in the model. If you know where you are at some spot on Earth, and someone provides you additional data in the form of a desired destination, you can reasonably accommodate that destination data by charting a course in a particular direction. Your location and direction are degrees of freedom, one of which is corralled in this case, and that -- combined with additional data -- helps you determine the "floating" degree of freedom. Conversely, if you're traveling in a known direction from an unknown point, where you end up can help you determine where you once were. Here we corral the other degree of freedom (direction) and, with the help of additional data (our accidental destination), determine what the value according to the other degree of freedom might have been. If you don't know either the direction you're going or your starting point, you have an underconstrained problem. You may arrive at an accidental destination, but your inability to know neither what direction you were going nor where you came from makes it possible for your starting point to be essentially anywhere. You can't infer a reliable solution from an underconstrained system because you don't know to which causal variable to attribute variation in the effect.
In order to be useful, statistical inference -- including Bayes -- requires you to corral either one or the other of two degrees of freedom. In that respect Bayes differs little, mechanically speaking, from any other exercise in conditional or joint probability. Where the magic of Bayes lies is in your ability to corral one of those degrees of freedom based on what you can determine or deduce non-statistically about the system and thereby possibly reduce ambiguity under multiple degrees of freedom That's the quantification of belief we talked about. But the belief still has to have some basis in fact in order for the inference to produce reliable results.
I'll resurrect two previous examples.
In the first, we have two coins. One is fair. The other is imperceptibly gimmicked so that it lands heads not with p = 0.5 as we would expect from a fair coin, but at p = 0.2. We're handed one of the two coins and we flip it repeatedly in order to try to determine whether it is the fair one or the gimmicked one.
Because of what we know about the problem, we can say that the prior probability of being handed the fair coin is p = 0.5; there were only two coins in the set. And if we're told with what bias the gimmicked coin misbehaves, we can set up a likelihood ratio for one toss coming up heads, E. P(E|H), "the probability of heads given a fair coin," is 0.5. P(E|~H), "the probability of heads given the gimmicked coin" is 0.2. We have a likelihood ratio of 0.1. Now to save time we're not going to work the actual arithmetic. The point is that, via a fairly straightforward computation, we can arrive at a posterior probability that is much better than p = 0.5 to determining whether we have the fair coin. (In the original example the probability of heads in the gimmicked coin was p = 0.01, which means that one toss will determine with high confidence which coin we're holding.) For now we only need to know that if the toss did come up heads, the posterior probability of having a fair coin is now greater than 0.5.
Now normally in a statistical inference what we'd do is plug that posterior probability back in as the prior and get new data -- i.e., toss the coin again. The likelihood ratio remains unchanged. We specified those as part of the problem, so that's a well-corralled value. But we are one data point down the road, so our prior is starting to converge to an answer. The original prior was also well-corralled because we specified the number of coins as part of the problem. After some small number of trails, we'll be able to infer with high confidence whether we have the fair coin.
Now what would have happened if we, against all statistical wisdom, suggested that the original prior probability of picking the fair coin was much higher -- say, p = 0.8. Let's say we professed to have some sixth sense for gimmicked coins. What would the model do? Well, with the nailed-down likelihood ratio in place, successive trials would modify that ongoing probability regardless of where it started. It would just take longer to get there, because we'd have to allow the system to recover from our absurd initial prior.
But abstractly what did we do? We allowed the prior to have a greater degree of freedom than a strictly rational method would have allowed. Rather than look at that data that suggested a reasonable prior, we quantified some ineffable property we professed to have. The system still succeeded because the likelihood ratio was firm. Its a degree of freedom that was very tightly corralled.
Now we introduce some uncertainty. Let's say that instead of there being two coins to choose from, there's an unknown number of coins. One is fair and all the rest are gimmicked the same way as above. What is our prior? Thinking statistically, we'd want to choose 1/
n, where
n is the number of coins. But we don't know
n.
Are we buggered? No. Bayes lets us make a reasonable guess for
n based possibly on non-quantitative data. If our proctor draws the coin from a container, we can make a reasonable estimate for
n based on the size of the container. If it's the size of an tuna can, we may say there can't possibly be more than 50 coins in it, so we set our prior at p = 0.02. That's how "quantifying belief" is supposed to work in Bayes. Our prior now has a greater degree of freedom than it once had, but it's still within reason and the overall problem is still computable because we have very solid likelihoods. Even if we threw caution to the wind and supposed that tuna can could hold a million coins, starting with a prior of p = 1 / 1,000,000 just means we have to apply those rigid likelihoods over more trials to get our probability to converge. With Bayes, it's in your best interest to choose your priors well if only not to waste time.
Let's change the problem a different way. Two coins -- one fair, one gimmicked -- as before, only you don't know the degree of bias in the gimmicked coin.
Are we buggered? No. We'll try to put some boundaries on the degree of bias. You might, for example, say that the practical mechanics of engineering a gimmicked coin that can't be detected by simple inspection limits the amount of bias that can be achieved. We don't know this by any sort of computation, but it's just what we believe based on how we suspect coins are made and how they might be gimmicked to come up preferentially head or tails. We might therefore guess that a gimmicked coin comes up heads according to a normal distribution centered on 0.5 and varying at most 0.3 in each direction p = (0.2-0.8). This means our likelihood ratio isn't a single-valued entity anymore but a distribution we'd have to model appropriately. As such it's not as corralled as a single-valued ratio would be. It's a degree of freedom that's been allowed to vary. This is another example of quantifying an uncertain belief and allowing the system to shake out the uncertainty via ingesting data.
No worries, we've gone back to the known number of coins. That means our prior, p = 0.5, is still rock solid. We know by that completely corralled degree of freedom that any variance from prior to posterior is more likely to be the result of the gimmicked-versus-fair behavior than of inaccuracy of the prior.
Okay, now we change the problem again. There are an unknown (but possibly guessable) number of coins, one of which is fair. But the coins that are gimmicked are gimmicked along a spectrum of bias (again, possibly guessable). Some coins may be biased, as before, so that P(heads) is as little as 0.02, and other coins might be biased so that P(heads) is as much as 0.8, or any value in between, including, say, p = 0.50001. Our formulation just got vastly more complicated because you can't use the fair-gimmicked bifurcation to derive your likelihoods, even when the result is a distribution and not a single value. There are some number of coins, and each coin has a different, unknown, response to a toss. You know that a fair coin has an even likelihood of coming up heads, but you can know only roughly with what probability one of the gimmicked coins will come up heads.
Most importantly, the system can't attribute a change from prior to posterior to the prior being way off (because we don't know how many coins there were) or the likelihood ratio being way off (because we don't know to what degree an outcome is affected by gimmicking). Both can cause changes in either direction. Both degrees of freedom in the model have been allowed to drift beyond our ability to guess them into proximity, and thus beyond the ability to declare that the system will converge to a stable value that is the correct answer.
Second example was the Bayesian search.
Say we have a house with five rooms. Our pet hamster is missing. The naive prior probability that the hamster will be found in any particular room is p = 1/
n where
n = 5, or 0.2. But is that the best we can come up with? If we just saw the hamster in its cage in the drawing room not a minute ago, we can probably say that the probability of finding the hamster is highest where he was last seen. We're allowed to reflect that belief in our set of priors {P(
i)} for finding the hamster in room
Ri, based on the distance of that room from the hamster's last known location.
Let's say we know the hamster likes the kitchen. We can also encode that information in our priors even though its merely our belief that the hamster will be more likely to head for the kitchen. Yes, it's based on past observation and knowledge, but the exact quantity we choose may be mostly a guess.
Our data is a search of each room. Therefore we have E
i for each
Ri. In a Bayesian search, our likelihoods express the likelihood that if the hamster were in there, our search would have been successful. In the kitchen, with its various nooks and crannies, we might say that the chance of a false negative is very high; the hamster might be there but we just can't see it. Conversely in the drawing room, which is empty aside from the hamster cage, we can say that a false negative is very unlikely; there simply isn't anyplace for the hamster to hide.
We have not necessary underconstrained our problem, even though there is reasonable wiggle room along both degrees of freedom. In practice, Bayesian searches parameterized as I have just done tend to be extremely successful because they severely limit the degree to which the priors and likelihoods may vary. But of course we can cripple it by assigning absurdly high search confidence or absurdly low priors for the hamster being in the room it actually turns up in.
This is a technique that's actually used to find downed airplanes, etc. The search area is divided up into a grid. The last known position and trajectory of the airplane on the grid is used to assign a prior probability of the airplane wreckage being in each grid square. We assume, for example, it will be found downrange of its last known position, at a distance consistent with falling from its last known altitude. The degree to which this corrals the degree of freedom depends on the quality of the available data and the skill of the plotters.
Then for each grid square, an estimate is made of the probability of a search finding the downed plane if it's really there. H is "the airplane is in this square." If E is "searching that square found the airplane" then P(E|H) is the estimated success rate of searching that square. Experts familiar with the area furnish their best estimates for each square. If a square contains a deep lake, P(E|H) might be very low, say 0.3. That means a search has only a 30% chance of finding the airplane in that lake if that's where it landed. Conversely a patch of desert (where most of my SAR activity occurs) might have P(E|H) as high as p = 0.99.
You start by searching the square with the most probable prior. Then you apply the result of the search to the priors and get a set of posterior probabilities. That tells you, given that it wasn't found in the most probable place, what the next most probable square is to search in. Maybe the lake comes up again, so you search there harder. The likelihood vector in this case is your confidence in the search. If you absurdly assigned p = 0.99 to the lake and p = 0.01 to the desert (i.e., you allow too much freedom in the likelihood), the posteriors you get as outputs at each step would essentially be random numbers, not an optimal search order. You'd get a number, but the number would be meaningless.
Your model pulls its priors out of your kiester. They have almost unlimited freedom. Your estimates for P(E|H) and P(E|~H) are so fanciful as to be entirely useless. Your model is underconstrained. It can produce no useful result.
