CoolSceptic
Muse
- Joined
- Mar 6, 2008
- Messages
- 689
Ah. This following post reminds me exactly why I usually choose not to engage in this ridiculous thread.
So Pixel42 seems to create a bizarre straw man, generates data that actually agrees with my original claim and then demonstrates astonishing incompetence with statistics.
First and most important point: the straw man. I did not claim 15 year trends were less noisy than 30 year trends. Your analysis isn't very good (too few points, and a strange decision to take the ratio of additive noise samples) but a decent analysis of sample near surface temperature data sets does indeed show that sample 15 year trends have greater variance than sample 30 year trends. I don't dispute this and this was never a part of my original claim.
I was addressing specifically the claim from lomiller and capeldodger that climate sceptics cherry picking 1998 had a big influence on the trend. I showed, with an example data set, that actually it is negligible influence, and most certainly does not result in a trend above the long term. Which leads to my second point - that your data corroborates my claim:
We then come to my third point, a very odd aspect of your analysis:
In your rather limited spread of data points, your results span 0.08 C per decade. But imagine if they had been 0 C/decade to 0.08 C/decade instead. You could claim the largest trend was *infinitely* larger than the smallest one! Oh dear. If it had been -0.04 C/decade to +0.04 C/decade, you would have had a trend -1 times larger! As you can see, using a ratio here makes no sense whatsoever. You have additive noise and the correct thing to do is to determine the spread. If you want to compare the spread with the ratio, you should use the sample standard deviation of the trends - but of course just using 5 points is not a good way to do this. Here is some MATLAB code to read and calculate all of the 15-year and 30-year trends in HadCRUT4:
On running it gives this result:
As expected, the 30 year trends which have twice as many samples show a reduction of standard deviation of a ratio of the square root of two. This is a valid ratio to determine - unlike the strange arithmetic you were doing above.
So in summary:-
- We agree 15 year trends in the temperature data sets show a higher standard deviation than 30 year trends, my analysis shows the spread is higher by a factor of approximately root 2;
- Your data corroborates my claim that exclusion of 1998 in the 15 year trends does not have a big impact, nor does its exclusion result in the trend jumping above the long- or short-term projections;
- I hope we can agree taking the ratio of largest to smallest trend has no useful mathematical meaning for trends with additive noise, and my more thorough analysis above shows the actual difference in noise between 15 year and 30 year trends of temperature data sets.
So Pixel42 seems to create a bizarre straw man, generates data that actually agrees with my original claim and then demonstrates astonishing incompetence with statistics.
First and most important point: the straw man. I did not claim 15 year trends were less noisy than 30 year trends. Your analysis isn't very good (too few points, and a strange decision to take the ratio of additive noise samples) but a decent analysis of sample near surface temperature data sets does indeed show that sample 15 year trends have greater variance than sample 30 year trends. I don't dispute this and this was never a part of my original claim.
I was addressing specifically the claim from lomiller and capeldodger that climate sceptics cherry picking 1998 had a big influence on the trend. I showed, with an example data set, that actually it is negligible influence, and most certainly does not result in a trend above the long term. Which leads to my second point - that your data corroborates my claim:
Note that in the samples you have chosen, there are three that include 1998 and two that do not. The two that do not have trends of 0.07 C per decade. Even conceding to capeldodger (see above post) that the long term trend is 0.16 C per decade, your figures of +0.07 C per decade confirm that 1998 is not a particularly important driver of sample 15 year trends, nor does its exclusion cause trends to rise above the long term trend. In fact, the highest 15 year trend you calculate *includes* 1998 near the start of the series.1996-2010: +.12 C per decade
1997-2011: +.05 C per decade
1998-2012: +.04 C per decade
1999-2013: +.07 C per decade
2000-2014: +.07 C per decade
We then come to my third point, a very odd aspect of your analysis:
This made me laugh a lot. Who on earth takes additive noise such as that on trends then declares a *ratio*??? Answer: someone who does not know what they are doing!The largest trend is three times the smallest
In your rather limited spread of data points, your results span 0.08 C per decade. But imagine if they had been 0 C/decade to 0.08 C/decade instead. You could claim the largest trend was *infinitely* larger than the smallest one! Oh dear. If it had been -0.04 C/decade to +0.04 C/decade, you would have had a trend -1 times larger! As you can see, using a ratio here makes no sense whatsoever. You have additive noise and the correct thing to do is to determine the spread. If you want to compare the spread with the ratio, you should use the sample standard deviation of the trends - but of course just using 5 points is not a good way to do this. Here is some MATLAB code to read and calculate all of the 15-year and 30-year trends in HadCRUT4:
Code:
hadcrutext = urlread('http://www.metoffice.gov.uk/hadobs/hadcrut4/data/current/time_series/HadCRUT.4.3.0.0.annual_ns_avg.txt');
data_table = textscan(hadcrutext, '%f%f%f%f%f%f%f%f%f%f%f%f');
num30yr_trends = size(data_table{1},1)-30;
num15yr_trends = size(data_table{1},1)-15;
ThirtyYearTrends = zeros(num30yr_trends, 1);
FifteenYearTrends = zeros(num15yr_trends, 1);
% Compute and store 30 year trend data
for loop=1:num30yr_trends,
olsfit = polyfit(data_table{1}(loop:(loop+29)), data_table{2}(loop:(loop+29)), 1);
ThirtyYearTrends(loop) = olsfit(1);
end
% Compute and store 15 year trend data
for loop=1:num15yr_trends,
olsfit = polyfit(data_table{1}(loop:(loop+14)), data_table{2}(loop:(loop+14)), 1);
FifteenYearTrends(loop) = olsfit(1);
end
% Display results
fprintf('Standard Deviation of 15 year trends: %.6f\n', std(FifteenYearTrends));
fprintf('Standard Deviation of 30 year trends: %.6f\n', std(ThirtyYearTrends));
On running it gives this result:
Code:
Standard Deviation of 15 year trends: 0.011321
Standard Deviation of 30 year trends: 0.007653
So in summary:-
- We agree 15 year trends in the temperature data sets show a higher standard deviation than 30 year trends, my analysis shows the spread is higher by a factor of approximately root 2;
- Your data corroborates my claim that exclusion of 1998 in the 15 year trends does not have a big impact, nor does its exclusion result in the trend jumping above the long- or short-term projections;
- I hope we can agree taking the ratio of largest to smallest trend has no useful mathematical meaning for trends with additive noise, and my more thorough analysis above shows the actual difference in noise between 15 year and 30 year trends of temperature data sets.
Last edited:
