The question of statistical significance

Diamond

Illuminator
D
Joined
Jun 2, 2003
Messages
4,729
I'm not a whiz at statistics, so my understanding of the meaning of standard deviation and "one-sigma confidence limits" is pretty poor.

Can any math whiz explain to me in simple language what Steve McIntyre is trying to express in these two posts:

MBH Confidence intervals and MBH Confidence Intervals #2 .

Please: Don't derail the thread with a discussion of the merits of global warming or the Mann Hockey Stick. We've had plenty of those.

I'd just like an explanation of what "confidence limits" are, and what the graphs, that he's produced, mean.
 
I'll do my best to make this as lucid as possible, but no guaruntees -- It's past my bedtime. ;) :p

First off, standard deviation and confidence intervals:
Any set of measurements is going to fall over a range. Say, measure the boiling point of water fifty times and you might get numbers anywhere from 94 degrees C to 105 degrees C with a cheep-o thermometer.

Then you run your data through a quick analysis and find that your average boiling point is 99 degrees C with a standard deviation (sigma) of 2 degrees C. This tells you that about 68% of your measurements fall between 97 and 101 degrees (99 +/- 2).

Another way to put this is that your confidence of any given meaurement falling into the +/- 2 range is 68%. This is your one-sigma confidence interval. (sigma = 2, 1*sigma = 1*2 = 2. The 68% relates to how these values are calculated in the first place and does not vary from case to case.)

You're two-sigma confidence interval tells you that your confidence of any given meaurement falling into the +/- 4 range is 95%. (sigma = 2, 2*sigma = 2*2 = 4. Again, 95% is standard).

You're three-sigma confidence interval tells you that your confidence of any given meaurement falling into the +/- 6 range is 99%. (sigma = 2, 3*sigma = 3*2 = 6. 99% is standard).

And you can go on and on from there.


I'll take a closer look at Steve McIntyre's posts after I wake up if someone hasn't already done so by then. At the moment, my eyes are too bleary to give it a fair read.
 
In general, it's impossible to measure things exactly, so the reported result of a measurement is not just a single number, but some sort of range. He is questioning how the size of some ranges were arrived at. If a range is reported as being narrower than is justified, the result will seem more precise than it really is and therefore conclusions might be drawn from it that are not actually warranted.

For example, suppose my thermometer says that yesterday's temperature was 60.3 degrees and that today's is 60.5 degrees. If my thermometer is good to a hundredth of a degree, it's safe to say that today is warmer than yesterday. But if it's only good to a full degree, we can't tell with very much confidence which day was warmer. So, it's important not to report that it's good to a hundredth of a degree if it's really only good to a degree.
 
Thanks for both responses.

So what is Steve McIntyre showing? That the claimed statistical significance for tree ring proxies is unlikely given that the one-sigma result appears to be more accurate than the 20th Century thermometer record? Did I miss something?
 
Diamond said:
Thanks for both responses.

So what is Steve McIntyre showing? That the claimed statistical significance for tree ring proxies is unlikely given that the one-sigma result appears to be more accurate than the 20th Century thermometer record? Did I miss something?
Click to expand...

I had an initial skim of the blog. He appears to be pointing out another case where the methodology reported by Mann does not correspond to the output you get. In this case the step changes in the standard deviation of the series, which ocurs as the components used switch at discrete points in time.
 
Drooper said:
I had an initial skim of the blog. He appears to be pointing out another case where the methodology reported by Mann does not correspond to the output you get. In this case the step changes in the standard deviation of the series, which ocurs as the components used switch at discrete points in time.
Click to expand...

Could this be the result of the different numbers of proxies that Mann is employing? Is it an indication of "cherry-picking"?
 
Diamond said:
Could this be the result of the different numbers of proxies that Mann is employing? Is it an indication of "cherry-picking"?
Click to expand...

No idea. This is at the heart of the problem. There is complete confusion about what Mann actually did and what data he used. That was the source of M&M's original complaints about Mann - MBH98 (the orignal Hockey Stick paper) can't be replicated and Mann won't help to square the circle. Read McItrick's site for the background on the data and code discrepencies that still have not been cleared up.

BTW, I just read the second instalment. There seems to be some additional curiosities in this. One thing that is mentioned is the way in which the confidence limits have been calculated. MaCintyre indicates that there may be some serial correlation in series (which for tree rings doesn't sem illogical). That means that the value in one period is a function of the value in previous periods. The implication is that the noise component will be a function of past noise.

So if Y(t)= f{X,Y(t-1)} + u(t) (Y depends on certain things this perioda s well its previous value)
N.B. - u(t) is the random error component at time t.

Just by looking at this you can see that Y(t) will have erro component u(t) plus some contribution from u(t-1) via the Y(t-1) term, pus a u(t-2) contribution and so on.

What this means is that you will need to use some slightly more complex calculations to derive the standard deviations and hence confidence limits - otherwise you will underestimate your confidence bands. MacIntyre seems to think this is the case here, by inferring that Mann used some relatively naive statistics, for a field that is very well covered in Econometrics literature.
 
Drooper

I don't understand this. Why should a confidence limit in one proxy set be dependent on another?

Unless the proxies have been somehow co-mingled...
 
Diamond said:
Drooper

I don't understand this. Why should a confidence limit in one proxy set be dependent on another?

Unless the proxies have been somehow co-mingled...
Click to expand...

Not from one proxy sdet to another. From one time period to the next.

Meaning, the error componenet in a variable in say, 1921 will be partly random noise from 1921, plus some spillover from the error in 1920, 1919 etc.

This leads to a breach in one of the assumptions used to estimate time series processes and any associated confidence band - namely indpeendent error terms (over time).
 
You must log in or register to reply here.

Back
Top Bottom