New Zealand climate data scandal

[MikeMangum]

http://briefingroom.typepad.com/.a/6a00d8341c51bc53ef012875dc003a970c-pi

The New Zealand Government’s chief climate advisory unit NIWA is under fire for allegedly massaging raw climate data to show a global warming trend that wasn’t there.

The scandal breaks as fears grow worldwide that corruption of climate science is not confined to just Britain’s CRU climate research centre.

Homogenized, adjusted, massaged data:

Raw temps:

 

[Arkayik]

And?

 

[Floyt]

Found out. Yep, we scientists are all in cahoots to deceive, bedazzle, and otherwise put one over on the rest of the world. Let's hope the oil lobby does something about it and imposes law and order.

 

[lomiller]

umm you realize raw weather station data needs to be corrected for changes in the way it's been collected over the years right? For example it's collected in a cooler time of day then it was 60 years ago (which is the biggest single correction)

 

[Reality Check]

http://briefingroom.typepad.com/.a/6a00d8341c51bc53ef012875dc003a970c-pi
Homogenized, adjusted, massaged data:

http://briefingroom.typepad.com/.a/6a00d8341c51bc53ef012875dc003a970c-pi

Raw temps:

http://briefingroom.typepad.com/.a/6a00d8341c51bc53ef012875dc00a7970c-pi

The actual news article also states

NIWA's David Wratt has told Investigate magazine this afternoon his organization denies faking temperature data and he claims NIWA has a good explanation for adjusting the temperature data upward. Wratt says NIWA is drafting a media response for release later this afternoon which will explain why they altered the raw data.

So if he has a legitimate reason for altering the raw data then is is good homogenized, adjusted, massaged data.

What really scares me is how quickly this rumor spread over the AGW denier blogosphere, e.g. Anthony Watts' blog already has the story (with the usual conspiracy theory comments).
If I was paranoid I start thinking about an AWG denier conspiracy .

Somehow I doubt that a valid reason from NIWA will spread as fast.

ETA
The response is as expected - adjustments because of adjusted weather stations.

NIWA Media Release 26 November 2009

Warming over New Zealand through the past century is unequivocal.

NIWA's analysis of measured temperatures uses internationally accepted techniques, including making adjustments for changes such as movement of measurement sites. For example, in Wellington, early temperature measurements were made near sea level, but in 1928 the measurement site was moved from Thorndon (3 metres above sea level) to Kelburn (125 m above sea level). The Kelburn site is on average 0.8°C cooler than Thorndon, because of the extra height above sea level.

Such site differences are significant and must be accounted for when analysing long-term changes in temperature. The Climate Science Coalition has not done this.

NIWA climate scientists have previously explained to members of the Coalition why such corrections must be made. NIWA's Chief Climate Scientist, Dr David Wratt, says he's very disappointed that the Coalition continue to ignore such advice and therefore to present misleading analyses.

NIWA scientists are committed to providing robust information to help all New Zealanders make good decisions.

 

[Skeptic]

umm you realize raw weather station data needs to be corrected for changes in the way it's been collected over the years right? For example it's collected in a cooler time of day then it was 60 years ago (which is the biggest single correction)

That is of course correct. The problem is, more or less the same raw data was used 30 years ago to prove -- after sufficient "correction" -- that the planet was experiencing global cooling. Clearly the same data can be massaged in many different ways.

 

[lomiller]

That is of course correct. The problem is, more or less the same raw data was used 30 years ago to prove -- after sufficient "correction" -- that the planet was experiencing global cooling. Clearly the same data can be massaged in many different ways.

Yeah, not so much…


ftp://ftp.apop.allenpress.com/EP-inbox/bams-89-09/bams-89-09-pdf/bams-89-09-1325-1338-pete.pdf

In fact the bulk of climate science in the 70’s was already solidly predicting warming.

Some people could be confused if they get their science information from the popular media though...

 

[Reality Check]

That is of course correct. The problem is, more or less the same raw data was used 30 years ago to prove -- after sufficient "correction" -- that the planet was experiencing global cooling. Clearly the same data can be massaged in many different ways.

I was not aware that 30 years ago the NZ data was used for evidence of global cooling. Can you provide a citation?

I doubt that it is relevant though.
Science has this habit of improving it's techniques.
Thirty years ago there was a certain methodology for accounting for changes in weather stations during the time that they take readings (positions, different apparatuses, different enclosures, even different paint work if I remember correctly!).
Today there is a certain methodology for accounting for changes in weather stations during the time that they take readings.
IMO It is unlikely that they are the same and that the modern methodology is more compeletely worked out and tested better.

 

[MikeMangum]

umm you realize raw weather station data needs to be corrected for changes in the way it's been collected over the years right? For example it's collected in a cooler time of day then it was 60 years ago (which is the biggest single correction)

I understand the need to homogenize a large data set of weather stations that may have had site changes and/or moves, sensor changes, different TOD readings, etc. What makes me uncomfortable is when those corrections, taken in total, produce such a dramatic change in the outcome compared to the unadjusted measurements. When a clear trend is created by the adjustments where there is no discernable trend in the raw data, it greatly reduces confidence in the results.

Normally, wouldn't local changes in the sites, stations moves, etc. generally tend to cancel each other out for the most part?Sure, a tree at the site grew up over the years to shade the station, but another site might have had trees cut back around it, etc. With the exception of sensor changes (changing all stations over to a completely different model of sensor) and potentially increasing urbanization of the site (I can't imagine many would undergo decreasing urbanization), none of those other changes requiring adjustment would be something that would be expected to, on average, require adjustment in a particular direction.

That the adjustments themselves essentially constitute the entire trend, they need to be extremely careful in ensuring that those adjustments are valid, and I certainly don't feel it is out of order to scrutinize the results in such a case.

The same is true of the data that NOAA and GISS rely on, btw, which makes me even less confident in the surface station results:

http://cdiac.ornl.gov/epubs/ndp/ushcn/ts.ushcn_anom25_diffs_urb-raw_pg.gif

That's why I trust the satellite record significantly more. That may be wrong, but it seems to me that there are significantly fewer potential data quality problems with the satellite readings. Unfortunately, those readings don't go back very far.

ETA: personally, I think that if the adjustments to correct for what are deemed deficiencies in the actual observations differs that much from the observations, I would assume the results are too low quality to have confidence in getting a valid result of any sort unless it could be compared to some other data set that is of very high quality. Does anyone know if the MSU readings do any sort of reporting by geographical region, other than the basic tropic | subtropic | polar?

ETA2: I'm curious if they have all of the station metadata that these adjustments would have to be based on.

 

[Reality Check]

I understand the need to homogenize a large data set of weather stations that may have had site changes and/or moves, sensor changes, different TOD readings, etc. What makes me uncomfortable is when those corrections, taken in total, produce such a dramatic change in the outcome compared to the unadjusted measurements. When a clear trend is created by the adjustments where there is no discernable trend in the raw data, it greatly reduces confidence in the results.

Actually it is the other way around:
When you do scientific analysis of data then you have to take in account all of the factors that influence the data. In this case it includes changes in the station environment, e.g. position. If you do not then any trend is probably going to be hidden by the factors that you ignore.

In this case the data is even more sensitive to unaccounted for factors because of the small number of stations. The raw data comes from "between 2 (from 1853) and 7 (from 1908) long-term station records".

Compare the Climate Science Coalition graph to the NIWA one. You will see that it is missing a vital component - the trend. This means that you have no basis for stating that there is no discernable trend in the raw data.
I see a flat or slightly rising trend by eyeballing it. I am probably wrong.

ETA: There are no deficiencies in the actual observations. There are site changes and/or moves, sensor changes, different TOD readings, etc. of the station. These have to be taken account of or any attempt to analyze the data is useless. Any trends will be hidden by these unknown factors.

ETA2: NIWA may not have 100% of the station metadata that these adjustments would have to be based on. I expect that they have a large % of the station metadata. The more metadata that they have the more reliable are the measurements. IMO They really only need to know about the position of the 2 or 7 stations since that is the major factor that changes the ambient temperature.

My ETA
My degree is in theroretical physics so I am going by a 20 year memory of undergraduate experimental physics - don't take my words as gospel!

 

[Megalodon]

So again we have ignorant people who refuse to be taught, preferring instead to clench to their preconceived notions, and to slander scientists worldwide.

But who knows, maybe this guy Wratt called someone a poopyhead in an email. That would be the final final nail in the coffin of this AGW theory whose decades-old forecasts are happening all around, before our very eyes...

 

[macdoc]

Chris DeFreitas
scientific advisor
Source: NZ Climate Science Coalition website

Bob Carter
founding scientist
Source: NZ Climate Science Coalition website

pardon my

http://www.realclimate.org/index.php/archives/2009/07/friday-round-up-3/#comment-132078

Chris de Freitas (16:08:44) :

The paper by McLean et al (JGR, 2009) does not analyse trends in mean global temperature (MGT); rather, it examines the extent to which ENSO accounts for variation in MGT.
[...]

Since so much of the criticism in the blogosphere to date is about the failure of the McLean et al paper to detect trends, which was not the aim of the paper

What this means is that Mr. de Freitas is admitting that his paper says nothing of significance about the existence/non-existence of AGW, contra the statements in its press release. Kudos to Joel Shore (also in the WUWT comments) for screwing this admission out of Mr. De Frietas.

And if you are wondering how the paper made it through peer review in the first place, there is an interesting trick AGW skeptics will sometimes employ (Roy Spencer does this sometimes) wherein they make their wild and crazy assertions in the press release to the paper, but leave them out of the paper itself. That is, the paper becomes an occasion for a polemic conducted via press release, blogosphere, and MSM. That seems to be what has occurred here.

http://bigcitylib.blogspot.com/2009/07/chris-de-freitas-back-pedals.html

 

[Rolfe]

I was trying to find online copies of the animal disease prevalence data I was using in a student lecture last week, but I can't find the bloody files!

Anyway, there have been striking changes in the prevalence of certain diseases, clearly starting in 1997. The most blatant of these is liver fluke, a parasite that uses the mud snail as its intermediate host. Snails thrive on wet ground, and especially in warm weather. The wider the snail habitat, the more fluke we see.

I've just been updating the lecture notes, which used to say that fluke is especially prevalent on the west coast of Scotland, because of the higher rainfall. In fact we've got fluke coming out of our ears now here in the east as well. Despite best efforts in understanding and controlling the parasite, the prevalence has shot up.

We have weather data as well, and so far as I know these haven't needed to be corrected for major variables, and these show increasing temperatures and increasing rainfall coinciding nicely with the increase in fluke prevalence.

Oh, maybe it's just a blip, and we haven't been collecting data for long enough, and it will all go back to "normal" in a few more years?

I wish.

Rolfe.

 

[theprestige]

I am sure that, being totally legitimate science, the raw data, collection methods, adjustment techniques, computer code (if any), and any other relevant information will be forthcoming at any moment.

Indeed, I'm at a loss to explain why it is necessary to wait for a "media response", when of course the obvious answer is "well, all of our research has been publically available for years, and has already withstood the scrutiny of several independent researchers, who have consistently corroborated our analysis and interpretations."

That is how climate science is done in New Zealand, right?

 

[macdoc]

Rolfe wrote

I've just been updating the lecture notes, which used to say that fluke is especially prevalent on the west coast of Scotland, because of the higher rainfall. In fact we've got fluke coming out of our ears now here in the east as well. Despite best efforts in understanding and controlling the parasite, the prevalence has shot up.

The higher rainfall/or higher intensity of rain events is not a well circulated aspect of AGW yet has the potential to be the earliest arriving major impactor as we've seen in Great Britain last week with all time records for 24 hour rain and Bombay in 2005 and Taiwan this year.

N Australia is also getting increased rain while the south is getting less as the climate bands shift.

Hydrology engineers going to be much in demand.

 

[lomiller]

Normally, wouldn't local changes in the sites, stations moves, etc. generally tend to cancel each other out for the most part?

Why would you think that? Improving the siteing of a weather station almost always means getting it away from things that create spots that are warmer then the surrounding environment. This means when you improve a site location or process it reads cooler then before for a day of the same temperature.

If you want something more specific the largest single adjustment is the ToD corrections that need to be applied. Data is now collected in a cooler part of the day, which creates a large false cooling trend if not corrected for. This *always* results in a cooling trend and accounts for most of the difference between the raw and final data. (IIRC ToD by itself if left uncorrected would induce a false 0.8 deg C cooling over the last 50 years.)

Sure, a tree at the site grew up over the years to shade the station, but another site might have had trees cut back around it, etc.

Cutting a tree down would create a step discontinuity that would be spotted during the analysis and not used. (the trend over that period would be bridged with a nearby site that had no such discontinuity. The same thing would happen if, say, a parking lot were built next to the site, a sensor error, etc) A tree growing nearby is a tougher problem, that can’t easily be identified.

 

[Skwinty]

I have been reading the AGW debate for quite a long time now and have never had anything to add as I have no qualifications to do so.
I believe that one would have to be insanely stupid to not acknowledge mankinds input to changing global climate. One would have to be totally impervious to reality to think that the outpouring of billions of tons of pollutants into the atmosphere from industry and the burning of vast quantities of fossil fuels had no effect on the climate and global warming.

There is however a question I want to ask about the recording of temperatures. Why are these temperatures recorded during the cooler times of the day. Why not record the temperature continuously and take the highest during the day. Would this not then eliminate the need to correct the temperature data?

 

[Beanbag]

Here's the point: it's CRAP data, collected in a slipshod manner that isn't consistent over many decades. It's like measuring with a rubber ruler. You can't trust it. Seriously. Scientists are applying whatever correction values they think are needed to "adjust" the figures to what they think is the "correct" response.

They've waved these figures about as an absolute truth, set in stone, proof that AGW exists. Now, they're chagrinned to have it made public that the numbers are set in Silly Putty.

I don't know which set of numbers to believe. They're all crap. The value for the increase in average temperature is less than the "noise" in the measurements.

Look up Cargo Cult Science in wikipedia. Here, I've even included alink:

http://en.wikipedia.org/wiki/Cargo_cult_science

Looks to me like the climate scientists were building a runway in hopes that the gods would land.

Beanbag

 

[Ivor the Engineer]

I have been reading the AGW debate for quite a long time now and have never had anything to add as I have no qualifications to do so.
I believe that one would have to be insanely stupid to not acknowledge mankinds input to changing global climate. One would have to be totally impervious to reality to think that the outpouring of billions of tons of pollutants into the atmosphere from industry and the burning of vast quantities of fossil fuels had no effect on the climate and global warming.

There is however a question I want to ask about the recording of temperatures. Why are these temperatures recorded during the cooler times of the day. Why not record the temperature continuously and take the highest during the day. Would this not then eliminate the need to correct the temperature data?

The samples have to be taken at the same time of day so that they can be compared with each other. E.g., comparing a sample taken at 12pm on one day with one at 12am on another would not be valid.

Sampling continuously and picking the highest temperature would result in a station sample from different times of day from sample to sample and also make the readings sensitive to glitches.

 

[Ivor the Engineer]

Here's the point: it's CRAP data, collected in a slipshod manner that isn't consistent over many decades. It's like measuring with a rubber ruler. You can't trust it. Seriously. Scientists are applying whatever correction values they think are needed to "adjust" the figures to what they think is the "correct" response.

They've waved these figures about as an absolute truth, set in stone, proof that AGW exists. Now, they're chagrinned to have it made public that the numbers are set in Silly Putty.

I don't know which set of numbers to believe. They're all crap. The value for the increase in average temperature is less than the "noise" in the measurements.

Look up Cargo Cult Science in wikipedia. Here, I've even included alink:

http://en.wikipedia.org/wiki/Cargo_cult_science

Looks to me like the climate scientists were building a runway in hopes that the gods would land.

Beanbag

The wonderful irony is that I work for a company which sells data gathered from logging tools to the oil and gas industry and we use corrections (some more justifiable than others) all the time too.

But as our motivation is good honest profit, smoothing out the bits which don't look right is perfectly acceptable.