Originally posted by Beth Clarkson
While it's true, I don't require the use of four controls and four tests for the binomiial analysis, using the means gives me with more confidence in the results. I feel I'm less likely to get a false positive.
Hmm ... I don't see it. On the null hypothesis, the probability of a success would seem to be 50%, either way. (And if not, we shouldn't be using a binomial test with
p = 1/2, to begin with.)
Means will always follow a normal distribution if the underlying data is even approximately bell-shaped.
Well, almost always. A Cauchy distribution is bell-shaped too. But, yeah, it's usually true, in the limit, if we average enough values. (The underlying distribution doesn't even have to be bell-shaped; it can be anything at all, provided it has finite variance. Cauchy distributions have infinite variance.) However, the important question is, how many values are enough?
Trying to answer that question brings us back to the question of the "right" way to average angles.
If we have a uniform distribution on the interval [-180, 180) of the real line, and we average together a bunch of values taken from it, the distribution of the mean will approach a normal distribution centered on 0. On the other hand, if the wax drops are uniformly distributed around a circle, the average of a bunch of drops---for any reasonable definition of "average"---should likewise be uniformly distributed around the circle; it should not favor 0 degrees, nor any other position. Right? What else could the distribution of the mean be? If no position is special, then no position is special; averaging isn't magically going to pick out one position from all the other equivalent ones.
But how can this be? What happened to the central limit theorem? Averaging is supposed to reduce the variance and bring us closer to a normal distribution; yet here we end up with the same distribution we started with, no matter how many values we average!
The same thing happens if we average a bunch of Cauchy random variables; we end up with an identically distributed Cauchy random variable. There, we can explain it away by appealing to the infinite variance; reducing infinity by a factor of ten, or a hundred, or a thousand, leaves us still with infinity. But, here? Does a uniform distribution on a circle have infinite variance??
In a sense, yes. Variance measures how spread out a distribution is. Infinite variance means it's really, really spread out. A distribution on the real line that's uniform on the interval [-180, 180) can be further spread out, e.g., to [-360, 360). But a uniform distribution on a circle is the most spread out of any distribution on the circle. How could it possibly be further spread out? It already covers the whole circle.
Ok, so how can we define the "average" of wax drops in such a way that it will have the properties we want it to? We can think of the positions of the drops as two-dimensional vectors in a plane, and average them that way, instead of as one-dimensional positions on a circle. Then, the mean of many drops has a distribution that approaches a 2D Gaussian about the center of the circle---a real bell, if you will, instead of a cross section of one. The more drops we average, the narrower the Gaussian becomes. So, if we do things this way, the (2D) variance does decrease as expected.
Although each drop lies on the circle, the average of many drops usually won't. If, for some reason, we want it to, we can project outward from the center, back onto the circle. This procedure yields a uniform distribution of angles, as we earlier decided it should. And now we can see why the variance doesn't decrease if we want the average to lie on the circle. It did decrease, but then we somewhat artificially increased it again when we moved the average out to the circumference of the circle, farther from the center than it really was.
So far, I've been supposing that the drops are distributed uniformly around the circle of wax. If the flame is not precisely centered, so that the drop distribution around the circle is not precisely uniform, then the mean of a bunch of drop position vectors will approach a 2D normal distribution that's somewhat off-center. In this case, projecting outward from the center, back onto the circle, will yield a distribution of angles that approaches normality as the 2D distribution narrows; however, the better centered the flame is, the more drops need to be averaged, to approximate normality to any specified degree of closeness. Four drops might not be nearly enough.
Accurately centering the flame increases the sensitivity of the experiment; an infinitesimally small psychic deflection of a perfectly centered flame is sufficient to move the drop from one side of the ring completely to the opposite side, 180 degrees away. But, by the same token, a centered flame makes it easier for chance, too, to move the drop by a large angle, and we need to take this fact into account in the statistical analysis.
We take it into account by not assuming a normal distribution for the angle of the drops (or even, for the angle of the average of four drops) unless we're sure the distribution really is nearly normal. If the true distribution is more uniform than a normal distribution is, the true probability of getting a value far from the mean is higher than for the normal distribution, so using the normal distribution instead of the true distribution will yield p-values that are too significant.
Trial to trial data seems to follow a uniform distribution (as would be expected). Deviations from the mean within a trial (looking just at control data to establish what would be typical) have a nice bell-shaped distribution. [...] The within trial variance is not constant from one trial to the next.
I do not understand. Can you give more details? What is uniform and what is bell-shaped?
The four points of any single trial are just four points; that's not enough to figure out a shape.
If different trials have different distributions, as the different variances appear to indicate, of what use is combining data from different trials? What could such a combined distribution tell us about any single trial's distribution?
---------
To summarize, I think it's best to treat this whole thing as a 2D problem. It seems rather shaky to compute means and variances by treating angle measurements as if they were measurements of position on an infinitely long straight line, ignoring the difference in topology between a circle and a line.
I guess if I was setting up the experiment, I would do much as Dave suggests - set up the apparatus, then apply known forces (puffs of air, holding glass with hand, tapping table or floor, etc), and working to eliminate their effect on the flame.
