- For another thing, in many situations, the specific event is only one of NUMEROUS possible results (millions?) -- and for unlikelihood to be of consequence in such a case, the specific result has to be meaningfully set apart from most other possible results.
- So, I think you're just saying that my explanation for how I am meaningfully set apart re OOFLam doesn't work. Is that right?
That is right. Your italicized paragraph is correct enough but importantly incomplete. In most situations there are many possible outcomes. For the likelihood of any one outcome to have the sort of consequence you require in your model, the specific outcome has to be identified ("set apart") and characterized
before the outcome occurs, and by criteria that rise above simply later being the outcome that occurs. Pre-specification, pre-identificaion, pre-ordination -- pick your word. The key in all cases is "pre-". You cannot choose the "intended" outcome with knowledge of what the outcome was or will be and declare the likelihood to have the kind of statistical import you're purporting in your model.
The analogy I chose was the poker game. The rules of poker have been around quite some time. For each particular variant, a list of hands or prototypical hands is provided along with an ordering ranking them. This forms the basis for assigning significance to sets of five cards drawn or dealt from the deck. The raw probability of being dealt any combination of five cards from a well-shuffled deck is the same for all combinations. You have as much chance of being dealt a royal flush of spades as any other combination of five cards. What makes the royal flush significant is that it is one of the pre-specified hands. We agreed prior to playing that this would be a winning hand. That agreement constitutes an assignment of significance.
What we may not do in poker is, after being dealt a random hand with no pre-specified significance, declare that to be a new winning hand. It attempts to assign significance based on knowledge of the outcome. The outcome is significant only because it was chosen, not because it was significant before the deal, and not -- for example -- because it follows the general criteria by which other outcomes are assigned significance (i.e., patterns of colors, values, and suits).
Your argument does the equivalent of inventing new winning poker hands after having been dealt the cards. You're trying to say the cards you were dealt are somehow now significant simply because there was a very small chance of being dealt that hand out of all the hands that were possible. A moment's thought demonstrates that would be true for all the other players at the table, for all hands played.
This analogy becomes a little more illustrative if we change the exercise from playing poker to determining whether the deck is stacked. You're handed a deck of cards and you are told the deck is either stacked or well shuffled. We can use Bayesian inference to find this out.
("Stacked," of course, means the cards are intentionally ordered in such a way as to produce hands of significance for some given game. And there are many games you can play with a standard deck of cards. To stack the deck for blackjack would mean arranging the cards so that a sequence of them adds cardwise to 21. To stack the deck for baccarat would mean arranging the cards so that certain strings of two or more cards have a modulus of 9. To stack the deck for poker would mean arranging the cards to create patterns of known winning poker hands. Also, stacking the deck is typically done to favor one player among N players, so it means every Nth card is intentionally arranged for. We'll assume two players of a simplified form of poker that simply involves evaluating each hand -- no hole cards or draws, and no replay of discards.)
What we have to work with are the hypotheses: the deck is stacked, or the deck is well-shuffled. And we have a set of easily-computed probabilities for drawing each kind of significant hand from a well-shuffled deck. Our priors could start with equal probability for stacked and shuffled. Our data, naturally, will be a series of hands played. What takes us from our priors to our posteriors is a model based on the set of probabilities that each hand will arise in a randomized deck. These are objective and immutable; they derive from the mathematics of the card deck. Where we encode subjective belief for this inference, if any, is in the priors. We may decide the guy who gave us the cards is shifty and assign a higher prior probability that the deck is stacked. Something like that would make this truly Bayesian.
If the deck is randomized, we would expect "significant" poker hands to be drawn by both players in a distribution roughly similar to what the likelihood ratio predicts for a randomized deck drawn to exhaustion. If one player draws "significant" poker hands at a markedly more favorable distribution than his opponent, the posterior probability increases that the deck is stacked.
Okay, that's a basic example. Here's where the Texas sharpshooter fallacy fits in. Let's re-run the experiment, only this time one of the players is allowed to arbitrarily declare his hand to be a "new" winning poker hand after he draws it. Now in the case where the deck was truly well-shuffled, this would bias our inference toward stacking, because the Texas-sharpshooter player would be drawing random hands that wrongly get counted as "wins" and skew his win rate. That would lead us to arrive at a posterior probability that the deck is stacked, but which factually is the wrong answer. That's why this is a fallacy. The experiment only works if we stick to the rules of poker and count as winning hands only those that were identified as such before the experiment began.
In order to get around this, you've cited hypothetical examples that you say justify identifying the outcome as the intended target after the fact. But in each of those cases I was able to show you how you interpolated into the example hidden sources of information that informed the intended outcome and would have been known or inferable prior to sampling the outcome. You cannot show any such circumstances for your actual model, wherein you simply identify those people who currently exist as the desired outcome, on no more basis than that they were chosen. Your hypothetical examples are not analogous to your model. Your model commits the Texas sharpshooter fallacy.