Correct Theoretically

From the trail of 1054756929.txt – the one that starts with the salutation “Hi Big Boy”.

Hi Keith,
Okay, today. Promise! Now something to ask from you. Actually somewhat important too. I got a paper to review (submitted to the Journal of Agricultural, Biological, and Environmental Sciences), written by a Korean guy and someone from Berkeley, that claims that the method of reconstruction that we use in dendroclimatology (reverse regression) is wrong, biased, lousy, horrible, etc. They use your Tornetrask recon as the main whipping boy. I have a file that you gave me in 1993 that comes from your 1992 paper. Below is part of that file. Is this the right one? Also, is it possible to resurrect the column headings? I would like to play with it in an effort to refute their claims. If published as is, this paper could really do some damage. It is also an ugly paper to review because it is rather mathematical, with a lot of Box-Jenkins stuff in it. It won’t be easy to dismiss out of hand as the math appears to be correct theoretically, but it suffers from the classic problem of pointing out theoretical deficiencies, without showing that their improved inverse regression method is actually better in a practical sense. So they do lots of monte carlo stuff that shows the superiority of their method and the deficiencies of our way of doing things, but NEVER actually show how their method would change the Tornetrask reconstruction from what you produced. Your assistance here is greatly appreciated. Otherwise, I will let Tornetrask sink into the melting permafrost of northern Sweden (just kidding of course).
Cheers,
Ed

There’s lots to talk about here.

The email is titled “Review- confidential REALLY URGENT”. The cosy back-room etiquette of peer review is often a convention more honoured in the breach than in the observance, but it is generally considered “not the done thing” to go passing round details of advance copies of other people’s papers prior to publication. Especially to competitors in the field. Ed knows this, which is why he put “confidential” on it, but it’s for the Cause, isn’t it?

Why is Ed Cook having to ask for data via this private back channel, rather than going to the public record of evidence published with the paper? The complaint is that they don’t re-do the Tornetrask reconstruction (a tree ring reconstruction of temperature) with their method, but apparently not even Ed can obtain the data needed to do so, without asking Briffa privately as a warmy-buddy. Clearly, the original study is not openly replicable, either.

There are many editorials out there dismissively stating that various warmists with vested interests have gone through all the Climategate emails and found none of them cast any doubt on the validity of the science. Which others then repeat, saving themselves the bother of looking. But I think that up there with “hide the decline” should be this phrase: “It won’t be easy to dismiss out of hand as the math appears to be correct theoretically”.

Speaking as a mathematician, I am outraged that any scientist should write “It is also an ugly paper to review because it is rather mathematical”. I know that some of you who suffered through school maths (which is to real maths as finger-painting in primary is to the Sistine Chapel ceiling) will not agree with me, but mathematics is a subject of beauty and marvel. That the climatologists seem uncomfortable with heavyweight maths and stats is very revealing. It has long been thought that they’re not very good at it.

Incidentally, that “Box-Jenkins stuff” refers to the classic and standard textbook on analysing time series, which is actually what Briffa & Co. are doing. It is the “classic, seminal, and authoritative book that has been the model for most books on the topic written since 1970.” It is the foundation stone of their entire subject! To dismiss it as “stuff” would be like someone working on General Relativity dismissing a criticism of their methods as full of “Einstein stuff”!

Because I think Box-Jenkins is pretty cool, I’m going to have a go at talking about some of what they were dismissing. To give you a flavour of it, anyway. People who don’t like maths should look away now. But the point I’m going to discuss is pretty important to the debate, anyway.

First, take a look at this chart:

I have plotted some data in dark blue, and helpfully drawn a trend line through it.

The data clearly goes up. Could it be going up by chance?

Before I explain, I’d first like you to take a look at another chart:

This data is a straight-line trend plus white noise. Looks very different, doesn’t it?

Now a lot of the maths behind looking at trends is based on the data behaving as in this second chart. We have a straight-line “signal”, we have random “noise” added to it, and we wish to distinguish the two, and hopefully get rid of the noise.

The question is, is the overall rise in the data bigger than the noise? Because if it is, we know it must be signal, and the trend is real. Now there’s a lot of maths been written to do this more precisely, but in this case our intuition is up to the job – it looks like the noise is about ±10, and the rise is about +40, so clearly the trend must be real.

But how did we estimate that ±10? We imagined a straight line through the data, assumed it was signal, figured the remainder must be noise, and looked at how wide it was. This is circular reasoning! We assumed the straight line was signal, to estimate the noise, to prove the straight line was signal.

And this is the statistician’s dirty little secret. If all you have is the data, this circularity is unavoidable.

We can of course be rather more sophisticated about it, but eventually it always comes down to the same thing. You have to start with a statistical model of the data, and then the maths tells you if it fits and what its properties are. Hopefully, you have some hint from the physics of whatever you’re measuring what the model ought to be. But if you don’t, you have to make an educated guess - and if you guess wrong, all your conclusions and significance tests will be invalid. If your guess is based on the data itself, the circularity reduces your confidence.

And there are a lot more possible models than people think. The basic ones are noise, linear trend plus noise, and some sort of curve plus noise. When you see people draw those smoothed lines and trend lines through data, that’s what they’re doing. They are suggesting one of these models. But there are far weirder possibilities.

Start with a sequence of normal random numbers. Now calculate the running total, the sum from the start of the sequence to the current point. This gives a new sequence which is also “random”, but with a very different behaviour. Each value is related to all the other values, especially the values close to it. It can get bigger and bigger, without limit. And it usually does. It is called red noise, or the random walk, or the drunkard’s walk, or several other names. (Einstein researched its connection to Brownian motion in 1905.) It’s a perfectly valid sort of randomness, and is commonly seen in practice when measuring something that accumulates.

We can also have accumulation with feedback – a quantity that accumulates random gains and losses, but that has a tendency to drift back towards the centre. One simple way to model that is to take the previous value, multiply by some number just below 1, so that it will drift towards zero, and then add our random gains and losses. (I used 0.998 to generate the data above.) This is technically called an AR(1) process (for “auto-regressive”, degree 1). There are also AR(2) processes that depend not only on the last value but also the current rate of change (think of it as the changes having a certain ‘momentum’, so once moving, it keeps on moving), AR(n) for higher integers n, MA(m) processes for moving averages (which is the standard result of “smoothing” data), ARMA(n,m) processes that mix the two, ARIMA(n,d,m) processes that combine the two with integration (red noise is ARIMA(0,1,0), for example), and many, many more.

Now if we have one of these accumulation-with-feedback sequences, we will get a mixture of properties. Over the short term, it looks like red noise accumulation. Over the long term, the feedback damps it down to a straight line plus noise. If I plot a few more values of that first sequence, you can see it doing so. (Remember, this is the same dataset as the first plot above, just zoomed out a bit to show more of it.)

Looking at short segments, we can see strong trends up and down, but watch it for long enough, and we see that there is no trend: there is no signal, it is all noise.

You can see it in the first chart. Over shorter intervals, the trends are steeper. The line is continuous: it doesn’t jump about instantly to any value, it always follows on from the previous value. But it isn’t smooth: the line is rough and irregular. When you see data doing that, you should suspect a cumulative process.

What does it do to your signal-detecting statistical tests, that you learnt in college? It messes them up. You have to make some quite complicated adjustments to fix them, and Box and Jenkins tells you how. But if you are not sure of your maths, one way to find out is to get a computer to generate thousands of artificial examples of random data according to the model, with and without added trend, and then see if your test can tell the difference. That’s called a Monte Carlo method. It’s mathematics done by experiment.

“Get to the point, Pa.” OK, here’s the point. There are a lot of charts of climate and weather data around, and a huge number of them have those straight trend lines drawn on them. Yes, the data goes up. But is it signal plus noise, or just noise?

When asked whether the temperature is going up, or has stopped, some people are saying that it is still going up. There is a ‘secret’ hidden temperature underlying the measured temperature, driven by global warming, and it is still rising. It is just being obscured by the short-term weather. Signal plus noise. But does this model make physical sense?

There is only one physical temperature. There is only one number for the total heat content of the Earth’s atmosphere and oceans. You could get something like the above model if the oceans, for example, were signal, while the air varied around them as noise. But the oceans are changing temperature on short timescales too. In the UK it is nice going swimming in the sea in August, but it’s a bit different in December. (A point, incidentally, that should be remembered by anyone talking about the heat capacity of the oceans delaying global warming for many decades. It’s not complete nonsense, but is a bit more complicated than they pretend.) So, how does the atmosphere know what temperature it is supposed to be? Where is the information stored? What is this ‘underlying’ temperature the temperature of?

Could it actually be cumulative? Could the temperature be better modelled by the effect of a leaky reservoir of heat with random gains and losses due to weather? Could the weather, in turn, be affected by the amount of accumulated heat? I think the answer is obvious.

The appropriate statistical model is very complicated – having many different cumulative effects operating on different timescales, many different feedbacks, and quite frankly, we have very little idea of what it is. It might even have the signal suspected. What we do know is that it isn’t just a straight line plus noise.

So all those straight lines and smoothed curves people put on charts are misleading – at least, if you take them as anything more than a visual guide to whether it happened to go generally up or down. They’re designed to suggest to you a trend-plus-noise model. And under such a model, the rise does looks significant. But statistically, it’s not nearly so obvious.

That’s what Box and Jenkins’ book set out in detail. Trying to figure out how to handle these sorts of strange behaviours. Is it a real trend, or just noise? And that’s what this paper Ed Cook was trying to block was apparently all about – whether they had screwed up on the maths and reconstructed the past from spurious rather than real trends.

Was the paper being reviewed by Cook right? Did it really show that all the tree ring stuff is rubbish? I don’t know. But if papers finding fault with the consensus can never get published, the faults in it will remain undetected. We cannot trust it if it has not been properly inspected.

This is what could really do some damage, to Science.