Tuesday, 8 May 2012

The M&M Project: Replication Analysis of the Mann Hockey Stick

The M&M Project: Replication Analysis of the Mann et al. Hockey Stick 

Reproduced from HERE 



This page provides a guide to our work on replicating the "hockey stick" graph made famous by the 2001 IPCC Report. For those new to the subject here are some useful overview papers:
  • The crash course: Read my APEC Paper from 2005, then Steve's Ohio State University talk from Spring 2008.

  • The rest of this listing is in reverse chronological order.

  • Steve has put together his May 2008 presentation at Ohio State, which brings the story since the Wegman Report/NRC Report up to date.

  • A major investigation into the hockey stick, the Wegman Panel Report, was headed by Edward Wegman of George Mason University, also past Chairman of the National Academy of Sciences Committee on Theoretical and Applied Statistics. The Wegman panel not only fully endorsed our findings, but also presented a wide-ranging critique of the insularity of the paleoclimate community, their isolation from mainstream statistics, and their hostility towards external review and replication work. Wegman makes a good recommendations about the need for higher standards of disclosure and review scientific research is used in public policy.

  • Following the Congressional hearings into the hockey stick in July 2006, Rep. Stupak put a series of questions to Dr. Wegman. The questions and his answers deal with a lot of the purported refutations of our work. His response is here. (Note: large file, 10 MB)

  • On August 23, 2006, we published an Op-ed in the Financial Post about the Wegman Report.

  • The National Research Council Report on the hockey stick was released in June 2006. They accepted our argument that Mann's method is biased towards producing hockey stick-shaped PCs, that uncertainties have been underestimated and that the bristlecone data, on which the famous hockey stick shape depends, should not have been used. They also express very little confidence in the IPCC's claim about the 1990s being the warmest decade in the millennium. But you have to read the report closely to pick all these things up--they bury it in a lot of genteel and deferential prose.

  • On June 26, 2006 we published an Op-ed in the Financial Post about the NAS report.

  • On March 2, 2006, Steve and I made an invited presentation to a panel of the US National Academy of Sciences investigating millennial climate reconstructions. Our presentation was one of 11 solicited from experts around the world.

  • In April 2005 RM presented a paper What is the Hockey Stick Debate About? to the Australian APEC Study Group. This paper provides a nontechnical summary of the main issues as well as a discussion of why they matter in the larger debates over climate change.

  • Our research was profiled in the cover story of the Feb. 1, 2005 edition of Natuurwetenschap & Techniek (NWT) , a prominent European science magazine. High-Res (3MB) version; Low-Res (0.9MB) version. It was also the subject of a commentary and a lead editorial. The articles were well-researched and clear.

  • In January 2005 we prepared a pdf backgrounder to accompany two new journal articles. This provided a (somewhat) nontechnical overview of our work and some FAQs on the subject up to that point.

  • The exchanges in GRL are discussed below


  • Hockey Sticks, Principal Components and Spurious Significance Geophysical Research Letters, Vol 32(3), Feb 12 2005, copyright 2005 American Geophysical Union (doi: 2004GL012750). Further reproduction or electronic distribution is not permitted. This is a preprint of the GRL paper that shows Mann's program mines for hockey sticks and overstates the statistical significance of the final result. There have been 4 technical comments submitted to GRL in response. We submitted replies to all 4, and they were sent out for refereeing. Two of the comments have been rejected by GRL. The two that were published were accompanied by our replies. These exchanges are discussed below.
  • The M&M Critique of the MBH98 Northern Hemisphere Climate index: Update and Implications Energy and Environment 16(1)69-100. AVAILABLE ON-LINE AT ENERGY AND ENVIRONMENT by kind permission of the publisher. This paper shows how Mann's results can be reconciled to our results based on handling of the PC algorithm and a Gaspe cedar ring series. We also discuss the bristlecone pines in detail and show why they should not have been included in the original data set.
  • "Corrigendum" by Mann, Bradley and Hughes. Nature 430, July 1, 2004 p. 105. This arose from our Materials Complaint to Nature in the winter of 2004. The story is detailed on the page about our dealings with Nature (see below--link to Archive).
  • "Verification of multi-proxy paleoclimate studies: A case study". Accepted abstract for presentation at American Geophysical Union Meetings in San Francisco, December 2004. Steve travelled to the AGU in December 2004 and presented our research--this was the abstract.
  • "Corrections to the Mann et al (1998) Proxy Data Base and Northern Hemisphere Average Temperature Series" Energy and Environment 14(6) 751-772.
    This is the paper that started the whole ball rolling!


  • Geophysical Research Letters recently published comments on our paper by Huybers and von Storch-Zorita. The Comments and our Replies are available on-line at Steve's blog. Steve has also posted extended comments on both papers, which link from the same page.
  • Here is a brief (4 page) summary, but the underlying Comments and Replies are not long and are worth reading themselves.


  • Steve McIntyre (assisted by John A.) now presents an ongoing blog at CLIMATE AUDIT. Current topics of discussion include Mann's newly-released fortran code, the Barton letters, and the other paleoclimate studies regularly cited as support for the hockey stick 'even if the hockey stick is wrong'.


  • The argument that our work "doesn't matter" hinges on 2 points: (a) if MBH use 5 (instead of 2) PCs, they can salvage a hockey stick, and (b) there are many independent studies that also yield a hockey stick. Point (a) is treated in our Environment and Energy paper. The issue is robustness. If a low-order PC, representing less than 8% of the explained variance in a single regional proxy network, is going to be allowed to overturn the conclusion that would be indicated by the entire rest of the data set, why even include the rest of the data? In MBH98 it is just there for show, to create the illusion of a hemispheric data base, while the final results are simply the imprint of a sample of bristlecones (dubious as temperature proxies) from western USA. Point (b) is also discussed in our E&E article, but Steve is also demolishing it at greater length at his Weblog--which by the way is well worth visiting regularly.

  • Prof. Mann has recently boasted that our 2003 paper in E&E has been completely discredited. In that paper we made 10 claims. Steve has written up a scorecard that goes over the public record and reveals Mann's claim to be at odds with the facts.

  • In our E&E article we showed that the MBH98 reconstruction has high early 15th century values, as shown in the Figure below, after applying two changes: (1) using the archived version of the Gasp� tree ring series rather than the version with ad hoc editing by Mann et al.; (2) using exactly the same number of series as MBH98, but with standard centered PC calculations rather than the data mining method of MBH98. However, neither reconstruction has any statistical significance.

  • The R scripts used in the GRL and E&E papers can be accessed at Steve's old web page; also GRL will archive the scripts used for that paper.

  • We have also prepared a short explanatory note on PC analysis as it relates to the MBH98 issues.


  • All supporting materials and past updates are now linked through the M&M Archive page. This includes our dealings with Nature and the early background material

Go to RM's publications page
Go to Ross McKitrick's homepage

Steve McIntyre's Latest

Yamal FOI Sheds New Light on Flawed Data

Read Steve's latest post HERE.