• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Probability of a probability being wrong?

Well a pair of dice is a bit of a restrictive example in this case, as there are quite obvious minimum values of 2 and maximum values of 12.
It's not that restrictive. Casinos are full of games of chance where the distribution is well defined and sampling is not necessary to estimate the mean or standard deviation. They are typically Bernoulli trials in which the probability of success (winning the bet) is well known so they are perfectly defined by the binomial distribution.

What I'm trying to point out is that when you said
The X% confidence interval is simply the number range in which you expect X% of your results will fall within each time you run the trial a series of trials.
The wording of that statement could have been misleading so I have fixed it up.
 
Last edited:
Relatedly, I found http://bayes.wustl.edu/etj/articles/confidence.pdf an entertainingly provocative read, and the p196 (b) example surprising even for me as a critic of confidence intervals, where an example is given for a calculated confidence interval that clearly cannot contain the population parameter at all.
 
Ultimately what I figured out (since nobody told me) is that a random variable is simply a trial in which all of the possible outcomes are numbers. Rolling a dice is the classic illustration of a random variable.


You are conflating the definitions of "random variable" and "random experiment." Our discussion of confidence intervals will be clearer if we properly separate the two terms. A random variable is usually defined as a function from a sample space into the real numbers, whereas rolling a die, or a pair of dice, is an example of a random experiment. For a pair of dice rolled once, the sample space of the experiment comprises the Cartesian product S = {1,2,...,6} × {1,2,...,6}. One can then define any number of random variables on S, such as

W = the sum of the values of the two dice = {2,4,...,12};
Y = 0 if wi ∊ W is odd, or 1 otherwise; or
Z = the expected value of wi for a pass line bet at craps.

The X% confidence interval is simply the number range in which X% of your results will fall within each time you run the trial.


That's an ambiguous description of a confidence interval. For clarity, we should do better: Given a sample X of size N from a distribution with a parameter θ, a 95% confidence interval for θ is a random interval [L(X), U(X)] that has a 95% probability of containing θ.

It is crucial to understand that X (big "X") is a random variable, and consequently that a 95% CI, as defined here, is a function of random variable. The importance of this fact is that the "95%" in "95% CI" is the probability, before the sample is observed, that a 95% CI will cover θ. In contrast, a specific realization, x (little "x"), of X is not a random variable, but a fixed quantity, and thus a computed, or realized, CI, [L(x), U(x)], is not random, but is fixed as well. Therefore, it is not the case that the probability that a realized 95% CI covers θ is 95%.

It is, however, easy to get fooled into thinking that the probability that a realized 95% CI covers θ is 95%, because ordinarily θ is unknown. A way to get the correct intuition about what a realized 95% CI is is to consider the case when θ is known. Imagine that we draw a sample from a normal population with mean µ=0 and variance σ²=1, and we calculate a 95% CI for µ of (–.1, .2). The probability that the interval (–.1, .2) contains 0 is obviously not 95%. The statement is ridiculous. Likewise, had we calculated the 95% CI (.1, .3) it would be ridiculous to say that there was a 95% probability that it contained 0.

As I showed above, when we know the value of θ, it is obvious that whether a realized 95% CI contains it or not is not a matter of probability. Only when we don't know the value of θ are we tempted to think probabilistically. What this shows is that when θ is unknown we are thinking probabilistically about θ itself. But in frequentist statistics θ is a fixed value (whether we know the value or not), not a random quantity; it has no probability distribution, and therefore it is meaningless to talk about it having a probability of being in some interval. The probability that θ is contained in some observed interval such as a realized 95% CI is a Bayesian posterior probability, which, given the data, x, we can only derive from a Bayesian prior probability—but we have not specified a prior probability. Thus, just as we can make no frequentist probability statement about a realized 95% CI containing θ, we can make no Bayesian probability statement about whether θ is in the interval. That is, there is no probabilistic interpretation that we can attach to a realized 95% CI at all.

If you know the type of distribution of the RV and its mean and standard deviation then it is relatively easy to calculate the confidence interval.


As I demonstrated above, a confidence interval is always a function of a sample, and is calculated exactly the same way whether you know the values of the parameters of the distribution or not. The values of the parameters do not enter into the calculation, even if they are known.

In this instance, the RV is the number of earthquakes observed in a 300 year period. The problem is that the trial was only run once and the number generated was zero. This doesn't give us a mean (zero) that we can have any confidence in and you can't get a SD at all if the sample size is only 1. Therefore, we can't really determine the confidence interval here.

As an alternative, I assumed that 1) the RV could be modeled by a binomial or normal distribution and 2) the result was within the 95% confidence interval. That is, I approached the problem backwards. You might question the validity of either assumption but I think the mathematics pans out.


Actually, the way the OP was phrased, the random variable of interest was the annual risk of an earthquake occurring, so the assumption of a binomial model, with sample size of 300, the number of years observed, was a perfectly reasonable model for the problem.

I'm probably teaching you how to suck eggs here but I'm hoping that if you know where I am coming from then there might be less tension arising from any future discussions from this topic.


Having looked up the highlighted phrase, I will take your post in that spirit and will hope that you do the same for mine.
 
Last edited:

Back
Top Bottom