Quality Control, CRU Style

by John L. Daly

In the Beginning...

On Friday 10th August 2001, this website revealed a very visible and obvious error in the hemispheric and global temperatures being issued by the Climate Research Unit (CRU) of the University of East Anglia, Norwich, England <see the original `stop press' story here>. They  presented graphs which contained a contradiction as illustrated in Fig.1.

Fig.1 - Hemispheric & global trends compared. (Colour indicators added)

Their hemispheric temperatures were showing a cooling during 2001, while their global temperature was showing a warming, something which is of course physically impossible. They were clearly incompatible - and visibly so - as shown in the original CRU graph (Fig.2 below) which was obtained as a postscript file (hemglob.ps) from the data section of their website on Thursday last (9th August).  Also downloaded were the numerical data files from which the graphs were generated.

Fig.2 - CRU's triple graph showing the contradiction between hemispheres and globe 

Within hours of releasing the story on this website, attempts to gain access to the CRU website were met with `server down or not responding' notices, lasting for about 10 hours. During this `down' period, other British sites were being accessed without difficulty. When the CRU site became accessible again, nothing had changed - the same contradictory graphs were on display and the same data files. In addition, it was found that the CRU online magazine Climate Monitor was also showing contradictory graphs.

On Monday morning British time (13th August), CRU acted to correct some of the data, the first indication of which was to replace the triple graph with a new version, one which showed the hemispheres now to be warming in line with the global graph which remained unchanged. See Fig.3 below.  No explanation was offered as to why or how the change had been made.

Fig.3 - The new version of the triple graph which appeared Monday morning.

However, the contradictory graphs on the Climate Monitor section of the CRU site remained unchanged throughout Monday and Tuesday.


Later ...

Then, on Wednesday morning (15th August), Australian time, the following CC email arrived from Phil Jones, the head of CRU and the person responsible for the hemispheric and global temperature data.

Taking the `data manipulation' remark first, nothing in the story presented on this site could be construed as an accusation of deliberate data manipulation.  It is true that CRU's conclusions are not acceptable to most sceptics for reasons well explained and documented elsewhere, but this is a scepticism of CRU's procedures and methods, not of their ethics. To question someone's judgment and/or competence is not the same as questioning their honesty, even though Jones might appear to equate the two.

The central point of criticism of the contradictory graphs was not that someone made an error - that can happen anywhere to anyone, part of human nature. Rather, it was the failure of anyone among CRU's 40-strong staff to even see such an obvious and visible error on what is their primary public product - hemispheric and global temperature.  `Quality Control' is not merely about elimination of errors, desirable though that goal may be, but rather the active review of everything produced at each stage of processing to detect such errors before they are passed into the public domain.  It is this that CRU failed to do, raising serious doubts as to the effectiveness of their quality control system generally.

The Jones email nevertheless attempted to put as positive a spin as possible on what was really an indefensible failure on their part.


The Explanations ...

"There appears to have been some teething problems in the last few weeks" the email says disingenuously, adding "This relates mainly to the Climate Monitor pages". 

This is not consistent with what the CRU website was actually showing. The error was clearly present in the data section managed, so the CRU site says, by Phil Jones himself.  Fig.2 was downloaded directly from Jones' data section, not from Climate Monitor. It appears that the problem did not relate to Climate Monitor at all, which appeared to be merely repeating a mistake originating from Jones' data section.

"Data on the ftp site is correct but some of the time series on the Climate Monitor site and associated graphics have problems during 2001", said the email.

When Jones sent the email on Wednesday 15th August, it was by then true that the data on the ftp site was correct, but it was not correct at the time the story was first released on this website on Friday 10th August. Fig.4 shows an extract from the part of the data taken from the ftp site during last weekend, the part that shows the data for 2001.  It is clear that the faulty figure was then present in the 2001 annual figures for the hemispheres. Climate Monitor was merely reflecting that prior error, not causing it.

Fig.4 - Extract of data downloaded on Saturday 11th August.

"So the NH and SH series numbers are far too low for 2001, because they are only seeing the Jan and Feb data."

Look at Fig.4 again at the NH and SH data for 2001. It shows six months, not two, and the half-year average in the right-hand column is exactly the number you would get if you added January through June and divided by 12 instead of 6. If those numbers were to be divided by 6, the average would then match up to the new revised numbers published on the CRU site. The global data also showed all six months, but if those numbers are added and then divided by 6, the final average is correct. In the email above, Jones gave the revised averages as 0.5 for the NH, 0.32 for the SH, and 0.41 for the global.  The global figure is the same, but the revised NH and SH averages are exactly double the earlier figures, indicating that indeed the January-June total had been incorrectly divided by 12 instead of by 6.

In other words, the error was quite simple. Someone or something put an incorrect formula into the hemispheric final average spreadsheet cells for 2001, putting a `divide-by-12' function into it (as would be appropriate for a full year's data) instead of a `divide-by-6' function which would be necessary when only 6 months of data are available.

Once the data for the hemispheres was averaged incorrectly, which no-one seemed to notice, the resulting graphs generated from that data was passed to the CRU data section on their website.

What happened at Climate Monitor was even more intriguing. Their graphs showed the hemispheres to be even cooler than the faulty graphs and data from the CRU data section. There, the hemispheric files n17.dat and s17.dat did not show the same error as the files from the CRU data section but rather had quite separate errors, giving only 0.03 for the NH and 0.02 for the SH, much lower than even the faulty `divide-by-12' figures.  The puzzle is easily solved if one divides only the January figures in both hemispheres by 12.

In other words, CRU made two errors, not one, both based on incorrect averaging, the very core of their institution's work - making zonal, hemispheric and global averages of historic temperature. 

The faulty graphs were then visible to everyone, both within CRU and without, for about two weeks or more. Yet no-one in this well-staffed public institution spotted the blunders until it was revealed on this website.

"The correct values are in the files tavenh/tavensh/tavegl.dat on the ftp site and on the temperature data page".

Too late - the horse has already bolted.  Notice Jones uses `are' instead of `were'. By the time his email was issued, the offending files had indeed been changed. Those files are now listed on the data section of the CRU website as follows -

       tavegl.dat     13-Aug-2001    14:42     27k 
       tavenh.dat    13-Aug-2001    14:42     27k 
       tavesh.dat    13-Aug-2001    14:42     27k 

The fact that all three files have identical time stamps indicates that they were copied from
another location at exactly the same time.

Here are those same filenames as downloaded last Saturday (11th August) -   tavenh.dat    tavesh.dat.    These are the hemispheric files and examination of both will show that the hemispheric averages for 2001 (as shown in Fig.4) was obtained by a simple `divide-by-12' mistake, while the global average was done correctly with a `divide-by-6'.

Phil Jones who is both in charge of CRU overall, and of the hemispheric/global data sets, says in his own bio page, "I am principally known for the time series of hemispheric and global surface temperatures, which I update on a monthly basis."  In his email, he blamed a computer crash, problems with computer automation, and Climate Monitor.  From the above account, we can see that the data on the ftp site was faulty last weekend and earlier, the part for which Jones is responsible, and that its current corrected status has been due to changes made to the data and the charts on Monday 13th August.  The separate faults on the Climate Monitor remained uncorrected for another two days.

"There is no hint of us manipulating the data,..."

Who said there was? That was something Jones himself inserted into the issue, a red herring to explain away a silly and mundane mistake, but one which revealed an appalling lack of quality control in the very institution which has helped convince governments that the world was undergoing rapid warming. Data from satellites and well-maintained rural weather stations show such a claim to be untrue.

"The problem will be sorted once our web manager has some time to sort it." - The second error sat on the Climate Monitor part of the CRU site until Wednesday 15th August, 5 days after the error was first revealed on this website.  Is their quality control such that a glaring mistake can just sit there on a publicly funded website while their webmaster `finds time' to fix it?  Does this mean that CRU's quality control culture is so lacking that not only does a visible mistake escape the notice of the 40-strong staff for over two weeks, but that correcting it takes such a low priority?

Meanwhile ...

We have been repeatedly told by global warming advocates that the `surface record' compiled by CRU should be believed in preference to the satellite record because of rigorous and scientific quality control exercised by CRU.  We now see that this assumption is quite unjustified as the two errors in data handling not only involved the final product of thousands of individual calculations, but were so visible as to make one wonder how such a large staff could not see it.

Clearly, quality control at CRU is seriously lacking.   If such obvious errors can go uncorrected for two weeks or more, what of the errors that might occur earlier in the data processing chain?

Even though we now have the `correct' hemispheric averages for 2001, they are correct only to the extent that the monthly figures are correct.  If the monthly figures and the zonal and station data which underpin them are subject to the same kind of quality control, the monthly averages themselves might be quite meaningless.

This is suggested by the satellite record. Here is a chart (Fig.5) of northern hemisphere temperature according to CRU (red) and according to the satellites (blue). 

Fig.5 - Satellite and CRU surface record compared for the Northern Hemisphere

The CRU graph is trending upward and now leads the satellite record by nearly 0.3°C even though both are referenced to a common starting point in 1979 when satellite monitoring began.  In assessing the worth of the CRU view of NH temperatures in 2001, it should be remembered that only January through June is being measured and that vast areas of the NH experienced record low temperatures during the recent northern winter.

The satellite recording system has been subject to a number of searching and rigorous reviews by independent scientists, the result of which was the finding of only minor technical errors caused by satellite changeovers and orbital drift, none of which altered the record significantly.

Perhaps it is time that the surface record was given a similar rigorous scrutiny by outside and independent scientists, especially given the quality control problem that clearly exists at CRU.

John L. Daly
Wednesday 15th August, 1400 GMT

Thanks to Miceal O'Ronain and Jerry Brennan for their valuable assistance 

Return to `Still Waiting For Greenhouse' main page