Apologies, I see now that the paper is
Fildes, R. & Kourentzes, N. Validation and forecasting accuracy in models of climate change. International Journal of Forecasting. Volume 27, Issue 4, October–December 2011, Pages 968–995
http://www.sciencedirect.com/science/article/pii/S0169207011000604
That paper was first mentioned (but without citation) by Westwall. I can now confirm that Westwall misrepresented the paper. Since Westwall plagiarized his comments from one or more denier blogs, I can also confirm that the denier blogs have been misrepresenting that paper.
After a few years hanging around on these climate-related blogs, you know that the *AGW* deniers (being specific, just so the denialati out there can't opt for the sympathy vote by accusing us of conflating you with Holocaust deniers) never, ever read whatever is on the end of a science-based link that you send them to. I find this behaviour to be perplexing, if not understandable with regards to human nature. I mean, honestly, who really wants to read stuff that runs contra to your carefully constructed world-view, built up over oh so many years?
In the case of this paper, we see that AGW deniers haven't even read the papers they misrepresent in support of their own denial. Indeed, neither Westwall nor the denier site(s) he plagiarized could give anything approaching a proper citation of the paper.
We have seen many examples of partisans misrepresenting a research paper. If this example is more interesting than others, it may be because several partisans of wildly divergent views have commented upon it in this thread without having read it.
I will use this brown color to comment upon the comments that have been made about this paper within this thread and in various blogs. When commenting upon the paper itself, I will use black on white.
Ad hominem arguments.
The first author, Robert Fildes, is a Distinguished Professor of Management Science at Lancaster University. He was a co-founder of both the Journal of Forecasting and the International Journal of Forecasting. He has co-authored papers with another co-founder, J S Armstrong, who is a prominent denier of climate change.
For some here, those associations already provide an excuse to reject the paper, and perhaps even to reject all papers that have ever been published in those journals.
The second author, Nikolaos Kourentzes, is a post-doc in Management Science at Lancaster University, where he presumably works closely with Fildes. He does research on neural network prediction of time series, which turns out to be highly relevant to the paper.
Using these biographies to argue against the paper or its conclusions is an example of ad hominem argument.
Summary of paper.
The paper begins with two long sections that explain the differences between forecasting and climate models. The authors state their purpose as "to review the various criteria used to appraise the validity of climate models, and in particular the role of forecasting accuracy..." of decadal forecasts.
The bulk of the paper compares the
decadal climate prediction system (DePreSys), which is based on a physical climate model, to several purely statistical algorithms that attempt to predict future climate over periods of a few years by mere extrapolation of past trends in various ways.
Physical models are not extrapolated from existing data using statistical techniques the way statistical models are. What interesting statistical criticisms do you expect to find when statistics haven't been employed?
As explained at its web site, DePreSys attempts to harness a physical model (HadCM3) to make decadal climate predictions. If the physical model makes better decadal predictions that most of the purely statistical predictors, then that would serve as partial validation of the physical model.
Again, the underlying physics system behaves as a random walk for these time scales so they are of no value in validating model results. Other than possibly ENSO or similar phenomenon you would not expect to make usable predictions on these time scales nor does anyone attempt to. Showing you can't is therefor a trivial result that little to the discussion.
On the other hand, showing that you can is a non-trivial result that I personally find interesting.
The main results of the paper are:
- For forecasting 1-4 years ahead, predictions based upon physical models (DePreSys) outperform all of the purely statistical algorithms. In particular, those predictions outperform random walks by quite a bit, which directly contradicts statements made by both Westwall and lomiller within this thread.
- For forecasting 10 years ahead, predictions based upon physical models (DePreSys) outperform a majority of the purely statistical algorithms. (When comparing DePreSys to a large number of purely statistical algorithms, one would expect a few of the statistical algorithms to do better just by chance.) In particular, predictions based upon physical models outperform the random walk model by a large margin. Once again, this result directly contradicts statements made by both Westwall and lomiller.
- The best of the purely statistical predictors predicts the same annual increase in average temperature as the IPCC AR4 report's scenario A2 (which essentially assumes continuation of current policies). Once again, this runs directly counter to the impression Westwall and the denier blogs have been trying to create.
For predicting 1-4 or 10 years ahead, the best of the purely statistical predictors was a neural network that was black-box trained on two inputs: (1) annual emissions of carbon dioxide and (2) "lagged values of the temperature anomaly". That model contained 33 numerical parameters.
(For those who don't know about artificial neural networks: They use brute-force machine learning to provide a theory-free match to their training inputs. They routinely extrapolate features that aren't even seen by their programmers.
With 33 numerical parameters, it should not be surprising that the neural network's ten-year predictions were more accurate than DePreSys---especially when neural networks are the second author's research specialty.)
Localised forecasts.
So where did Westwall's misrepresentation come from? Was Westwall just lying about the paper?
No. He was plagiarizing the denier sites' misrepresentations. The numbers cited by Westwall and the denier sites all come from section 3.3 of the paper.
Section 3.3 of the paper considers the problem of forecasting
local temperatures. For example:
The GCM models performed substantially worse than the random walk....This implies that the current GCM models are ill-suited to localised decadal predictions, even though they are used as inputs for policy making.
Well, duh. The legitimate point that the authors are making here is that
global climate models (
GCMs) should not be misused to make
localised predictions.
Not only does that problem fall well outside the intended purpose of
global climate models (
GCMs), it seems to me that section 3.3 was so badly written that it nearly invited misinterpretation by climate deniers.
The peer reviewers fell down on the job here. To give an indisputable example: Tables 6 and 7 show errors calculated over differing time periods. For an explanation of that important point, the authors refer the reader to footnote 23, but footnote 23 has nothing to do with that. I suspect the authors meant to refer readers to footnote 25, which is at least marginally relevant, but footnote 25 doesn't provide the promised explanation either.
Those who raised ad hominem arguments against this paper can point to section 3.3 as justification. I wouldn't even try to argue against anyone who thinks the authors wrote section 3.3 because they were trying to give deniers some way to claim that the paper contains a substantive criticism of global climate models.
Conclusions.
Following their scientifically shaky excursion in section 3.3, the authors take quite a random walk before straying back onto solid ground. They emphasize the shortcomings of DePreSys, without emphasizing the fact that it outperformed the majority of their purely statistical models, and without emphasizing the extreme adjustability of their neural networks. For example:
In carrying out the analysis reported here, we have achieved improvements in forecasting accuracy of some 18% for up to 10-year-ahead forecasts.
That sentence would be more accurate if the highlighted "for up to" were replaced by "but only for". The authors should also have acknowledged that they were also able to achieve
reductions in accuracy approaching a factor of two. The 18% improvement they mentioned was a picked cherry.
However, there is no support in the evidence we present for those who reject the whole notion of global warming: the forecasts still remain inexorably upward, with forecasts which are comparable to those produced by the models used by the IPCC.
That may be the most important sentence of their conclusions, but it is not a sentence Westwall or the denier sites want to quote.
I'm not even sure the authors want it to be quoted. That sentence is buried deep within the middle of a long paragraph.
It's hard not to notice that the two pages of discussion and conclusions cite prominent skeptics of climate change more often than would be expected by chance. That pattern was seen within the two introductory sections as well.
In summary, I cannot counter expectations that were raised/lowered by the ad hominem arguments. Even so, it's abundantly clear that Westwall and the denier sites have been misrepresenting this paper.
?
?
