Merged AGW without HADCRUT3

[CoolSceptic]

That's a rather creative interpretation of what Halley wrote. He did say that paleoclimate reconstructions are imprecise, but also found that they would need to be underestimating the variance by a factor of 4 before in order for it to be at all plausible for the recent warming to be within the bounds of natural variability.

You're kidding me. Did you actually read the paper? Merely changing the version of the instrumental series halved the magnitude of the effect he measured. And let's be clear here: when I say "changing the version", I don't mean swapping CRU for GISS or some such. I mean swapping CRU v3 for CRU v2. HALVES the magnitude of effect he observed, reducing the variance factor to just over two. Kudos to Halley for being honest and including this result (shame there was no indication of this in the abstract) rather than burying it, though.

Halley attributes this to sampling, and whilst this is a reasonable speculation, I cannot see how he can dismiss other reasons, such as changes to the CRU algorithm itself, to which his parameter estimates are extremely sensitive.

This has a consequence though: it is clear that the error bounds on his estimates of the LTP behaviour are way too small, for such a large change in result to be caused by what should have been a trivial change in input data. It is clear that there is a source of error here which is not understood and not accounted for properly.

(As an aside: since Halley uniquely relies on the CRU instrumental series, technically it is not admissable on this thread, as we are supposed to be discussing what if CRU were not available. Of course, Cohn and Lins would also have to go. But then, much of Koutsoyiannis work would still stand!)

Halley goes on to point out that if the instrumental series is undersampled, then the palaeoclimate reconstructions are also likely to suffer from the same problem. He then points out that it has been argued that there is a systematic underestimate of these temperature reconstructions (citing von Storch's demo that variance matching of a noisy signal to the instrumental series suppresses historical variability) and outlines what must be done to build confidence in this. He accepts in the paper that the present reconstructions are inadequate for drawing clear conclusions. (I would similarly suggest that the huge sensitivity of the result to instrumental versions show that these records do not yield parameter estimates acccurately enough for such an analysis either).

Of course, you only learn this by reading the paper itself. Those who merely read the abstract might be under the sort of impression that you give here.

 

[CoolSceptic]

You still don’t get it do you? You are using the Hurst Phenomenon as a god of the gaps. Worse you are appealing to the existence of gaps as proof your god exists.

LTP, as far as I goes is purely a statistical observation and one that occurs in many types of systems for well understood reasons. The fact that it occurs in systems where those reasons are not yet understood is not proof that it occurs without any reason at all. Despite this you are trying to evoke it as “proof” climate isn’t changing from a known physical effect and essential saying LTP is “proof” that “stuff just happens”.

In real science, however statistical modeling techniques fall by the wayside in the face of know physical effects.

Bwahahahaha

You gotta be kidding.

By gaps I assume you refer to the creationist fossil record gaps argument. This is clearly unfalsifiable - there will always be gaps in the record, and indeed it has been pointed out that every new fossil found creates two new gaps out of the old one.

LTP on the other hand, is quite falsifiable in many ways. You can show the underlying structure is not LTP. This can be done by providing the equations of the system in question and deriving the statistical structure of the output (not presently feasible for climate), or you can show that it is statistically inconsistent; it is easy, for example, to show a Markovian serial correlation structure is quite different from Hurst at low frequencies.

However, everyone who has looked into this and tried to falsify it (including those on the pro-AGW side, such as Rybski and Halley cited by TK) consistently find LTP is present in the climate system. Whoops.

And as for your ludicrous suggestion that physical effect is more important than statistics, here's a little story for you. Ever heard of the Torah codes? James Randi has. These were the claims that skip codes in the original text of the bible included predictions of the future. This was the physical effect being claimed, and the original authors claimed to find a statistically significant link, with the result way outside of the confidence intervals (with a p-value something like 10^-5)

Randi went along (can't remember now, but perhaps as part of a million dollar challenge claim) and found that they had forgotten to account for an important effect in the statistics. The people in the predictions had more than one way of spelling their name, and by being selective, it was possible to bias the result. When the correct statistics were applied, the confidence interval included "no effect".

What we have here, is an example of how the correct application of statistics widens the confidence interval and suddenly the evidence of the purported physical effect disappears.

In your simplistic world where physical effect always trumps statistics, you end up accepting AGW, that the bible contains predictions of the future, water divining and all sorts of other ridiculus things. Good luck with that.

 

[a_unique_person]

In your simplistic world where physical effect always trumps statistics, you end up accepting AGW, that the bible contains predictions of the future, water divining and all sorts of other ridiculus things. Good luck with that.

What have you been smoking?

 

[a_unique_person]

You talk about the Hurst phenomena as if it has some physical presence rather than a statistical observation.
It doesn't.

I guess that's what he's being told on the blogs.

 

[mhaze]

You're kidding me. Did you actually read the paper? ....

Of course, you only learn this by reading the paper itself. Those who merely read the abstract might be under the sort of impression that you give here.

I think I'll go read the paper.

 

[DogB]

Me too.

 

[lomiller]

And as for your ludicrous suggestion that physical effect is more important than statistics, here's a little story for you. Ever heard of the Torah codes? James Randi has. These were the claims that skip codes in the original text of the bible included predictions of the future. This was the physical effect being claimed, and the original authors claimed to find a statistically significant link, with the result way outside of the confidence intervals (with a p-value something like 10^-5)

Randi went along (can't remember now, but perhaps as part of a million dollar challenge claim) and found that they had forgotten to account for an important effect in the statistics. The people in the predictions had more than one way of spelling their name, and by being selective, it was possible to bias the result. When the correct statistics were applied, the confidence interval included "no effect".

You realize you are on the bad side of that example right?

In this situation we already know the underlying physics we are not trying to prove some new physical principal. Furthermore the observations and the predictions made from those physical principles statistically match. If they didn’t more work would be required to find out if the mismatch was due to bad statistics, an unknown physical principle or a misapplication of the known physical principles, but that isn’t the case.

You on the other hand are doing just what the guy in your example does. He tries to take statistical observations to make an extraordinary claim about an underlying physical principle and misapplies the statistics. Your extraordinary claim is that all our notions of causality are incorrect and that measurable effects occur without underlying causes. (almost the definition of supernatural)

 

[mhaze]

....

You on the other hand are doing just what the guy in your example does. He tries to take statistical observations to make an extraordinary claim about an underlying physical principle and misapplies the statistics. Your extraordinary claim is that all our notions of causality are incorrect and that measurable effects occur without underlying causes. (almost the definition of supernatural)

The only way to reach that assertion is to ignore considerable published science in long term persistence in the hydrology literature.

That's what you've done.

So Lomiller... a Denier?

 

[CoolSceptic]

What have you been smoking?

Well, I'm just blinded with science there. I'm clearly out of my depth.

You talk about the Hurst phenomena as if it has some physical presence rather than a statistical observation.
It doesn't.

I guess that's what he's being told on the blogs.

Ah, I've got macdoc on ignore, but it seems pointless because I still read his/her drivel through quotes.

The Hurst phenomenon has basis in physics. Just because you don't know about it doesn't mean it doesn't exist. The first person to derive the behaviour analytically from first principles was Kolmogorov in 1941 in his work on the intermittency of turbulence. Of course, if you understand how the dynamical core of a GCM works, Kolmogorov's work will be second nature to you (LOL). Where do you find turbulence in the climate system? Why, in the tropics of course, and Kolmogorov's theory provides a useful underpinning to the LTP observed in vertical tropical convection.

However, the consequence of Kolmogorov's work is not as clear-cut as later works.

Klemes analytically demonstrated LTP in the 1970s in a model of the hydrological system based on an infinite circular cascade of reservoirs. OK, so the "infinite" requirement is a bit of a problem, but he demonstrated that Hurst-like behaviour was evident even with a surprisingly small number of reservoirs.

Perhaps the most interesting derivation of LTP comes from Koutsoyiannis, though. Koutsoyiannis demonstrated a few years back that both LTP and Markovian dependencies can be derived conditionally from maximum entropy. Those conditions tie in well with many aspects of the hydrological cycle (clouds, water vapour) which in turn have a large influence on global temperatures. This is obviously a powerful technique as it potentially provides a differentiating mechanism between Markovian dependence (as assumed by the climate scientists) and Hurst dependence.

So, as we see, the theoretical physics behind the Hurst phenomenon is well grounded.

Oh, and please let me know which blog you think I got that info from, and, even if it were copied from a blog, why that would make it wrong.

 

[BenBurch]

I cannot find anything reputable that supports the Hurst phenomenon. Care to provide a few citations to journals?

 

[BenBurch]

(Beyond empirical studies, that is. Show me a theoretical underpinning and then how it disproved AGW - from published sources, not blogs, not self-published climate iconoclasts, not E&E.)

 

[macdoc]

What next CS, Yuri Geller is actually doing something once in a while?? LTP as applies to the paranormal....

What a pile of horsepucky - trying to extrapolate from an obscure paper on the statistics of hydrology to the physics of climate change...one of the more amusing bits of errata to hang a denier hat in....anomalies may cluster, they ALL have a driver basis.

Then you just casually slip in your clouds and water vapour, vapours.....sounds like a Victorian ladies description of the new fangled electricity....with about as much grounding in science....

Gonna wave the scented hanky now??.....given how "ripe" your redolent contribution is....

Perhaps a climate scientist view might clear the air a bit.... noisome around here just now.

Hypothesis testing and long range memory
Filed under:

— gavin @ 10 August 2008

http://www.realclimate.org/index.php/archives/2008/08/hypothesis-testing-and-long-term-memory/

and
http://tamino.wordpress.com/2008/06/10/hurst/

 

[CoolSceptic]

You realize you are on the bad side of that example right?

Nope - I'm on the good side. But your understanding is probably not.

In this situation we already know the underlying physics we are not trying to prove some new physical principal.

LOL. That's just what the Torah code guys claimed.

Furthermore the observations and the predictions made from those physical principles statistically match.

LOL. That's just what the Torah code guys claimed.

Are you seeing a pattern here?

If they didn’t more work would be required to find out if the mismatch was due to bad statistics, an unknown physical principle or a misapplication of the known physical principles, but that isn’t the case.

You do understand that there are such things in statistics as false positives *and* false negatives, right? That's why we have "type I" and "type II" errors. You seem to have conveniently lost one of those in your little false dichotomy here.

You on the other hand are doing just what the guy in your example does. He tries to take statistical observations to make an extraordinary claim about an underlying physical principle and misapplies the statistics. Your extraordinary claim is that all our notions of causality are incorrect and that measurable effects occur without underlying causes. (almost the definition of supernatural)

Nope. I'm merely observing that the choice of Markovian statistical model used by climate scientists cannot be justified from first principles, and cannot be chosen in preference to the Hurst model. Climate scientists merely assume that the Markovian model is correct.

If you can provide evidence that the Markovian model is correct (or falsify the presence of LTP), then please do so. And, you might want to publish, because that would be a huge scientific discovery. Until then, it would be prudent to relax that constraint and accept that either model could be correct. In this circumstance, the larger of the two CI's from calculations should be selected.

 

[CoolSceptic]

I cannot find anything reputable that supports the Hurst phenomenon. Care to provide a few citations to journals?

The theoretical underpinnings are discussed in the posts above, but here are the exact references (courtesy of DK), plus a brief note from me what they contain in italics:

Kolmogorov, A.N., Wienersche Spiralen und einige andere interessante Kurven in Hilbertschen Raum, Dokl. Akad. Nauk URSS, 26, 115–118, 1940. Analytical derivation of LTP in fluid dynamics

Hurst, H.E., Long term storage capacities of reservoirs. Trans. Am. Soc. Civil Engrs, 116, 776–808, 1951 (Published in April 1950 as Proceedings Separate No. 11). First empirical observation and analysis of LTP in hydrology; some early statistical analysis

Mandelbrot, B.B., and J.R. Wallis, Noah, Joseph, and operational hydrology, Water Resour. Res., 4(5), 909-918, 1968. Development of the mathematics of LTP in hydrology

Mandelbrot, B.B., and J.W. van Ness, Fractional Brownian Motion,fractional noises and applications, SIAM Review, 10, 422-437, 1968. Development of the mathematics of fractals

Klemes, V. The Hurst phenomenon: A puzzle?, Water Resour. Res., 10(4) 675-688, 1974 Simple hydrological model demonstrating a basis for LTP

Cohn, T. A., and H.F. Lins, Nature’s style: Naturally trendy, Geophys. Res. Lett., 32(23), L23402, doi:10.1029/2005GL024476, 2005. Statistical consequences of LTP in hydroclimatic variables

Koutsoyiannis, D., Uncertainty, entropy, scaling and hydrological stochastics, 2, Time dependence of hydrological processes and time scaling, Hydrological Sciences Journal, 50(3), 405–426, 2005. Analytical derivation of the Hurst phenomenon from the principle of maximum entropy

The last reference has an error in one of the equations, (at least in the preprint, not sure about the final version) extra marks if you spot it

 

[BenBurch]

Not one of those supports any relevance to AGW.


Sent from my iPod touch using Tapatalk

 

[CoolSceptic]

http://www.realclimate.org/index.php/archives/2008/08/hypothesis-testing-and-long-term-memory/

and
http://tamino.wordpress.com/2008/06/10/hurst/

Blog responses to my discussion of peer-reviewed papers? Well, I'm not surprised. I note both of those articles confirm the presence of the Hurst phenomenon in climate (although it is a shame that RealClimate quote articles with pretty basic errors in them; Blender et al find Hurst exponents greater than one, which are by definition meaningless... ho hum)

 

[CoolSceptic]

Not one of those supports any relevance to AGW.


Sent from my iPod touch using Tapatalk

Hmm, just two minutes after I posted them up, you're a pretty fast reader, especially as some of those papers you'll need to find a library to read (they are not all on teh internets)

You asked for the theoretical underpinnings. I gave you the theoretical underpinnings. You ignored them and posted a non-sequitur that aligns with your preconceived opinions.

Your reply is utterly unscientific. But don't worry; I expected nothing less.

Gotta go now anyway, got some slightly hairy deadlines to meet. Probably be back in a couple of weeks. Have fun while I'm gone.

ETA: Ben, you asked for AGW to be disproved. Can't prove a negative, Ben, you should know that. I can't disprove AGW any more than I can disprove the existance of god. It's all in the CIs, it renders the evidence worthless. I can say, there is no scientific reason to believe in AGW just the same as there is no scientific reason to believe in god.

 

[lomiller]

LOL. That's just what the Torah code guys claimed.

Are you seeing a pattern here?
[/QUOTE]

Indeed I am, though like the guy in your blurb it just isn’t the pattern you think it is…

You do understand that there are such things in statistics as false positives *and* false negatives, right?

And…

If you think it’s a false positive give us some reasons why Stephan-Botlzman, conservation of energy and Kirchoff's law laws are all wrong?

Nope. I'm merely observing that the choice of Markovian statistical model used by climate scientists

Again they don’t use statistical models at all, the results are based on physical principles.

 

[Megalodon]

ETA: Ben, you asked for AGW to be disproved. Can't prove a negative, Ben, you should know that. I can't disprove AGW any more than I can disprove the existance of god. It's all in the CIs, it renders the evidence worthless. I can say, there is no scientific reason to believe in AGW just the same as there is no scientific reason to believe in god.

Actually, as with any scientific hypothesis, you can't prove AGW. Disproving it is trivial. That's why we say that hypothesis must be falsifiable. Your curious notion of how science works is wrong...

As it stands, there is a huge amount of evidence supporting the AGW hypothesis, so if you think there's no scientific reason to think AGW is correct, then you must have spent very little time informing yourself on the matter.

 

[mhaze]

Actually, as with any scientific hypothesis, you can't prove AGW. Disproving it is trivial. That's why we say that hypothesis must be falsifiable. Your curious notion of how science works is wrong...

As it stands, there is a huge amount of evidence supporting the AGW hypothesis, so if you think there's no scientific reason to think AGW is correct, then you must have spent very little time informing yourself on the matter.

Ummm.....

Now whoa....Whoaaaa....

What is exactly that there hypotheis?