Friday, December 04, 2009
ClimateGate: It's the code, stupid
A week after I first wrote about it, I continue to think that the real scandal to come out of the CRU document leak was not in the emails but in the revelation that the code in the CRU's climate model seems to have been written to generate specific output. For example.
It seems to me that before we enact policy on the basis of climate models, their code should be made public so that it can be studied and validated. Indeed, if we were at all interested in engineering credibility in to these models, we would rebuild them using modern techniques for developing software, included documented and validated change control so that third parties could determine why a change was made at each step and that the change in fact met the specification. If it takes a few years and a few billion dollars to rebuild these models professionally, would that not be money and time well spent given the stakes?
RESPONSE: Regular commenter Brian Schmidt offers this link from Tim Lambert in response. I lack the persistence to see if it holds, but Lambert is a smart guy so I pass it along for your consideration.
30 Comments:
By 
, at
Fri Dec 04, 12:27:00 PM:
You are correct. But then how would the Goracle sell snakeoil?? And be awarded a "Nobel Peace Prize?"
By , at
Fri Dec 04, 12:27:00 PM:
It seems to me that before we enact policy on the basis of climate models, their code should be made public so that it can be studied and validated.
Given the rank dishonesty of these "scientists" as they suppress contraindicating evidence, engage in ad hominem attacks on those writers and those magazines that publish them, and their willful, deliberate destruction of the raw data (to save space? Really? What scientist would destroy the very foundation of the research being conducted?) how could we possibly trust these people (I do not refer to them as scientists anymore) to tell us the truth about the "code" they used? We cannot even trust them to tell the truth about the techniques used to adjust the data so as to get a coherent picture--which techniques may very well otherwise be usable to regenerate the raw data, and which techniques, in honest hands, often are legitimate manipulations of the data. But these people even have refused to reveal their manipulations.
No. We have to start over. Collect raw data all over. Do the analyses all over. Nothing coming from UEA on climate can ever be trusted. Nothing coming from the people involved in this PC climate view can ever be trusted. Those people should never be able to get a job in science again. The relevant governments should be taking steps to recover the climate study-related grant money awarded to UEA and to the people who purported to work on the climate matter. Related loans till outstanding should be called in their entirety.
E Hines
By 
Ray, at
Fri Dec 04, 12:30:00 PM:
Oh, no, you can't. The science just isn't there yet. I've looked at the code, and it's classic of what I've jokingly called "research code," that is to say, code written for the purposes of testing out an idea, to prove that something works, etc. I distinguish this from "production code," which is code that somebody out there might actually rely on.
You only make the switch from the former to the latter when you have enough confidence in the techniques, assumptions, etc., built into your model to actually want to replicate and scale the code for large scale applications. That they haven't done so speaks volumes about their confidence in their models.
If the climate scientists themselves don't believe their models are ready for the big time, who are we to question them?
By Robert, at
Fri Dec 04, 12:48:00 PM:
I agree, however, I'd settle for researchers simply making their code available upon request. It doesn't seem too much to ask seeing as how they get government funding.
By the way, thanks for linking to my article :)
By Brian Schmidt, at
Fri Dec 04, 03:18:00 PM:
Typical skeptic attack, once again refuted, this time by Deltoid:
"But what is the code directly following the fragment Raymond quotes? Look:
;
;filter_cru,5.,/nan,tsin=yyy+yearlyadj,tslow=tslow
;oplot,timey,tslow,thick=5,color=20
;
IDL uses a semi-colon to indicate a comment, so the only code to use yearlyadj has been commented out. Raymond must have known this since he is an Emacs user and Emacs colour codes the comments. This doesn't seem to be a smoking gun so much as a gun that hasn't been fired."
By Brian Schmidt, at
Fri Dec 04, 03:19:00 PM:
Sorry, forgot to include the Deltoid link:
http://scienceblogs.com/deltoid/2009/12/quote_mining_code.php#more
By , at
Fri Dec 04, 04:39:00 PM:
None of this will matter one iota, because "climate change" isn't about science. It is about money, power, and politics.
The response has been set since the hockey stick graph was first discredited in 2004, when researchers showed that the CRU model generated a hockey stick even when completely random numbers were fed as input to the system. The wagons were circled, access to the data and the models were revoked, lame rebuttals were offered, and the critics were smeared. And so it begins again. Witness the lame response by Brian Schmidt and the equally lame analysis at the Deltoid link.
Climate change has nothing to do with science. This is an opportunity for a variety of parties, each with different but aligned interests, to manipulate a compliant and stupid mass media into driving a narrative that advances their respective interests. Democratic politicians define an environmental constituency who give them votes and support. In return they get Federal grants. Al Gore gets fatter and richer by investing in companies that will profit from Federal investments in clean energy. Goldman Sachs is salivating over the fortune they will make trading carbon credits.
And where the hell is the Republican party???? At best, playing Democrat Lite and proving once again that they are the Stupid Party.
The first Republican Senator who stands up and states publicly that the AGW scare is a fabricated crock, calls for a Congressional and criminal investigation of these researchers, and promises to lead a filibuster of Cap & Trade gets $1000 of my money. It's a safe bet I will keep my money.
By Brian Schmidt, at
Fri Dec 04, 05:21:00 PM:
I appreciate the update to the main post, TH.
Thanks.
By JPMcT, at
Fri Dec 04, 10:43:00 PM:
Re the "rebuttal" of the code: Apparently the same code appears elswhere in an unbracketed form...at least according to rebuttals of the rebuttal.
All this code posting aside, tree ring selection aside, dog ate my original data aside, intimidations aside, resignations of administrators of the data aside... can't EVERYBODY agree on one point:
The data is suspect.
Doesn't THAT point ALONE make it the obligation of the prudent legislator NOT to attempt a restructuring of an entire country's economy, and, in turn, that of the entire WORLD?
Well, DOESN'T IT???
On the other hand...if you are hell-bent on believing this cache of data that has now been splattered with excrement...even after all these counter arguments...wouldn't you want legislation that actually DOES SOMETHING about it?
Cap & Trade does nothing more than redistribute wealth. It will have essentially no predictable influence on any of the parameters that are so CRITICAL as to require radical action.
Soooo....I ask again...what is the purpose?
Clearly, Green is the new Red.
By Assistant Village Idiot, at
Sat Dec 05, 12:01:00 AM:
If I am understanding deltoid's criticism correctly - and I may well not be - then Ray's comment above answers it nicely. Yes, it is a gun that is not yet ready to fire. The programmer knows it is not ready to fire and includes comments to indicate same. But it gets fired anyway, because it's what's currently ready.
I was also puzzled at deltoid's snark doubting that a subset of environmentalists are communists. I talk with these folks a fair bit, a some are quite open, even blase, about their marxism. I think there is a liberal automatic response that whenever a conservative says "communist" it is obviously a silly exaggeration, smacking of McCarthyism, as if the conservative had said "stalinist." But communist is not a ridiculous synonym when referring to an avowed marxist. There seems to be a "jeez, these guys don't know how to get their lingo down so that they don't sound ridiculous to us. Therefore they must be stupid and wrong."
By , at
Sat Dec 05, 12:50:00 AM:
I challenge anyone on the face of the planet to produce a climate model that can accurately predict the PREVIOUS 150 years!
Yes, that is accurate. Produce a climate model that can model what we already know has happened.
How hard can that really be?
Other than the minor detail no one has been able to do it.
By Don Cox, at
Sat Dec 05, 06:15:00 AM:
"I challenge anyone on the face of the planet to produce a climate model that can accurately predict the PREVIOUS 150 years!"
What data would you build into this model? You can't use the data we have for 1859-2009, because that is what you are trying to predict.
Do you have accurate enough data for, say, 1759-1859? I doubt it.
By Assistant Village Idiot, at
Sat Dec 05, 08:43:00 AM:
Don, no problem. I have lots of back issues of the Farmer's Almanack. We're very resourceful up here making due with what we've got.
By , at
Sat Dec 05, 10:41:00 AM:
"Do you have accurate enough data for, say, 1759-1859? I doubt it."
And yet we're gonna use data that you claim isn't accurate enough to the adequately describe the past as the basis for future predictions that we're to bet the farm on? Way to go "full retard' on us, Don Cox!
By Brian Schmidt, at
Sat Dec 05, 11:58:00 AM:
Anon: look up hindcasting some time. The models work pretty well.
Village Idiot: I doubt you know more enviros than I do - I work in the field - and I don't know any who'd describe themselves as marxist. Everyone I know sees a large role for the private sector, which a marxist wouldn't.
I expect there are a few out there who are different, but they'd be out on the margins and not very important.
By Brian Schmidt, at
Sat Dec 05, 12:01:00 PM:
And it's not clear what exactly is done, if anything, with the similar parts of the code. Considering this particular piece was supposed to be the smoking gun and it's all nonsense, I'd like to see the skeptics give it their best shot, rather than doing more armwaving.
By , at
Sat Dec 05, 01:33:00 PM:
Brian Schmidt:
Village Idiot: I doubt you know more enviros than I do - I work in the field - and I don't know any who'd describe themselves as marxist. Everyone I know sees a large role for the private sector, which a marxist wouldn't.
Example: Van Jones, self described "communist," who described green jobs as a different path to achieve his goals. He talked about green jobs in the private sector, but never explicitly renounced communism, making it sound like the proverbial two steps backward, three steps forward.
No, not all ecofreaks- I am dating myself by using that term- are Marxists. Most are not Marxists. But there is that tinge. Recall Barry Weisberg? I was an eco-activist/ecofreak well before the first Earth Day. I worked alongside one eco-activist from that time,who told me that he wouldn't object to being drafted, so he could learn ordnance.
There was a split between "liberal" and "radical" ecofreaks at the time, so this person I quoted was NOT an outlier, at least in the college town we lived in. He never went the ordnance route, and works as a professional with a postgraduate degree. Most of us from that era regained our sanity.
I do not define "regaining sanity" as returning to autos without catalytic converters nor to using leaded gasoline, for example. I refer to leaving moonbat leftism.
By , at
Sat Dec 05, 01:36:00 PM:
As I've ranted about here, even basic data collection on climate science is suspect. In the last few days I've learned that even the determination of current GAT [global average temp] has issues, because of this. I thought climate scientists could at least get that much right. But take a look here:
http://www.coyoteblog.com/coyote_blog/2009/12/example-of-climate-work-that-needs-to-be-checked-and-replicated.html
Basically, this is saying that outsiders can't tell how it was that USHCN insiders combined data from two sites -- Grand Canyon and Tucson. Apparently, Grand Canyon has been showing no increase, but Tucson has. The latter has been affected by urbanization -- it's an "urban hot spot." USHCN won't say, but Mythbuster Steve McIntyre has done his own reverse engineering to show that USHCN has fudged Grand Canyon data up and Tucson data down in a crude averaging. This is bad science on so many levels.
My BS meter has been at 11 for awhile because I'm seeing a lot of parallels to financial frauds and debacles, which I know well. Financial frauds like Bernie Madoff's often involve the perpetrator being able to manipulate the reported data,, Sound familiar?
Brian is a whack a mole -- he can always come up with a piece of BS data -- because the data collection process isn't transparent, etc, etc.
Brian has just given himself up. He knows more "enviros" than we do, works in the field, and sees a commercial opportunity. So too do Al Gore, Carol Browner, and Van Jones.
By Assistant Village Idiot, at
Sat Dec 05, 08:41:00 PM:
Brian, my lack of precision caught me up. The people I know who work in environmental firms are decidedly non-marxist. The activists, who gravitate more to the academy, the nonprofits, and publishing up here in NH, have a stronger anti-corporate orientation.
I've been away from discussions here, so I don't know anon's reason (if any) for accusing you of foul motives. I don't share that assumption.
By , at
Sat Dec 05, 10:13:00 PM:
The EPA is poised to implement a lot of rules on the assumption that CO2 is a pollutant, is my understanding. I don't fully understand this yet nor the implications, but it looks like this train will leave the station -- and soon -- regardless of whether Congress adopts the Energy bill and whatever happens in Copenhagen. It can have big implications.
Meanwhile, the Second Circuit (NY mostly) and the Fifth Circuit (Texas and Louisiana mostly) have both reversed lower court dismissals of claims that global warming can give rise to tort claims for nuisance and trespass. The Ninth Circuit (California mostly) may soon may soon follow suit. Thus, if you think you've been injured by AGW find a lawyer! We may see a flood of lawsuits soon.
The Fifth Circuit test case is particularly comical. The defendants are a parade of the usual suspects: coal companies, oil companies, chemical companies and utilities. Plaintiff lawyers even say that AGW caused Katrina. Damage claims include all sorts of Katrina damage including "loss of consortium" -- so if you were in New Orleans during Katrina and didn't get laid for a week -- that's an element of class action damages.
There's even a claim against industry trade groups and the corporate defendants for "civil conspiracy" to "reposition global warming as theory rather than fact."
Complaint at page 20:
"All of this activity has been part of a concerted and tortious effort to intentionally decrease public awareness and divert public policy activity away from the real dangers associated with Global Warming and the known need to restrict the emission of greenhouse gasses."
If you believe in free speech you should be especially troubled by this.
Now ask Brian what he does for a living.
Not to just pick on Brian, go here: http://www.kpcb.com/team/index.php?Al%20Gore Al Gore is working hard to single-handedly take the risk out of venture capital investment, at least for Kleiner Perkins.
By Brian Schmidt, at
Sun Dec 06, 02:25:00 AM:
A quote from Van Jones, the alleged communist:
"We are entering an era during which our very survival will demand invention and innovation on a scale never before seen in the history of human civilization. Only the business community has the requisite skills, experience, and capital to meet that need. On that score, neither government nor the nonprofit and voluntary sectors can compete, not even remotely."
http://en.wikipedia.org/wiki/Van_Jones#The_Green_Collar_Economy
Van Jones matured.
(Village Idiot: I appreciate your dissociation from one of the anons.)
By , at
Sun Dec 06, 08:42:00 AM:
Brian, you ducked the question and pointed to Van Jones to change the subject. But I'll play.
Recall that Van Jones was supposed to be Obama's Green Energy Jobs Czar. You say that Van Jones has matured from communist to businessman?
Fresh out of Yale Law, Van Jones started out as a declared Marxist activist in San Francisco. He then ran some kind of Katrina-related hustle in New Orleans. I'm still waiting for the expose on how much money got ripped off and pissed away in the reconstruction of New Orleans. In one of the last episodes of The Sopranos, even Tony was talking about getting in on the action way down in New Orleans.
This led to Van Jones seeing the "Green in Green." so he wrote a book "The Green Collar Economy."
If we had real reporters in MSM we'd know how much Van Jones was involved in adding a last minute addition to the House Energy bill -- it would promote inner city jobs in the green economy by requiring Fannie and Freddie to buy second mortgages to finance green home improvements. There's actually express language about Fannie and Freddie having to relax their normal underwriting criteria to facilitate this. Do we ever learn?
Brian, this kind of business doesn't make us richer or better off. We used to call it graft but "graft" is too small a word for the scale of this thievery. I doubt that "my very survival" depends on Van Jones' managerial skills, nor on Al Gore's facility at capital allocation, nor on Carol Browner's magical environmental consulting skills.
Further, we can't class action litigate ourselves to prosperity.
By , at
Sun Dec 06, 12:11:00 PM:
The UN IPCC just said, "full speed ahead." Gordon Brown just called skeptics like me "flat Earthers." The PM doth protest too much.
In contrast, the BBC has a decent and fair summary here:
http://news.bbc.co.uk/2/hi/science/nature/8376286.stm
They've got open comments here:
http://newsforums.bbc.co.uk/nol/thread.jspa?forumID=7310&edition=2&ttl=20091206165551
This is more than we're seeing from US MSM, who appear to have snuffed Climategate. I expect this story to die out largely, barring some other major development.
Understand that anything that comes out of Copenhagen won't affect the course of AGW, even if it is happening. The events there will be comical.
By , at
Sun Dec 06, 05:45:00 PM:
Here is what I previously said:
Example: Van Jones, self described "communist," who described green jobs as a different path to achieve his goals. He talked about green jobs in the private sector, but never explicitly renounced communism, making it sound like the proverbial two steps backward, three steps forward.
Brian opines that the following Wikipedia quote refutes my statement, and shows that Van Jones has “reformed.”
"We are entering an era during which our very survival will demand invention and innovation on a scale never before seen in the history of human civilization. Only the business community has the requisite skills, experience, and capital to meet that need. On that score, neither government nor the nonprofit and voluntary sectors can compete, not even remotely."
This Van Jones quote from Wikipedia does NOT contradict my statement that Van Jones talked about green jobs in the private sector.
From the East Bay Express:
Jones' fixation on solidarity dates from this experience. He took an objective look at the movement's effectiveness and decided that the changes he was seeking were actually getting farther away. Not only did the left need to be more unified, he decided, it might also benefit from a fundamental shift in tactics. "I realized that there are a lot of people who are capitalists -- shudder, shudder -- who are really committed to fairly significant change in the economy, and were having bigger impacts than me and a lot of my friends with our protest signs," he said.
First, he discarded the hostility and antagonism with which he had previously greeted the world, which he said was part of the ego-driven romance of being seen as a revolutionary."Before, we would fight anybody, any time," he said. "No concession was good enough; we never said 'Thank you.' Now, I put the issues and constituencies first. I'll work with anybody, I'll fight anybody if it will push our issues forward. ... I'm willing to forgo the cheap satisfaction of the radical pose for the deep satisfaction of radical ends."
His new philosophy emphasizes effectiveness, which he believes is inextricably tied to unity. He still considers himself a revolutionary, just a more effective one, who has realized that the progressive left's insistence on remaining a counterculture destroys its potential as a political movement.
This supports my previous statement that Van Jones "never explicitly renounced communism, making it sound like the proverbial two steps backward, three steps forward." You notice change in tactics: this implies the goals have not been changed. His talking here about working with capitalists sounds an awful lot like the Popular Front tactics. We know who used those. He still considers himself a revolutionary, with radical ends: just a change in tactics.
Brian, if you have to revert to Wikipedia to make your point, perhaps you are not as well informed as you think you are. The Wikipedia article DOES cite the East Bay Express article, but not the quotes from it that I used. (I knew of the East Bay Express article months before I read the Wikipedia article.) This brings forth the issue that there is often bias in Wikipedia articles, making them less than authoritative for controversial issues, as they often present information selectively.
Anonymous @ Sun Dec 06, 08:42:00 AM: When Brian discussed Van Jones, he was NOT ducking anything, but responding to my posting @ Sat Dec 05, 01:33:00 PM.
By , at
Sun Dec 06, 07:17:00 PM:
Sorry for the confusion. I'm Anonymous @ Sun Dec 06, 08:42:00 AM
When I said Brian ducked my question, it was my asking Brian what he does for a living. I think I know and if I'm right it'd give context to his positions. If you read anon at Sat Dec 05 10:13 am you can guess what I;m driving at.
Van Jones should be a sideshow at this point. Not every professional AGW advocate is just a hustler, but I have to find one yet.
Diogenes For A Day
By , at
Sun Dec 06, 07:51:00 PM:
I'm reposting this above as well, because it's significant:
For all the heat over AGW lately, I haven't seen much light. The following is an eye opener. Apparently physicists are starting to get into climate science. String theory is going nowhere, so they need a new gig. You have to be really smart to be a physicist and they worship at the altar of the scientific method. From what I've learned lately, I wouldn't let a climate scientist valet park my Ford Escort.
The following link includes a video of a June 2009 70 minute presentation by Jasper Kirby, a physicist at CERN who's mostly been working on atom smashing. If we were back in World War II, Kirby is the kind of guy we wouldn't want to fall into enemy hands. If I was Vannevar Bush, I know where I'd send Michael Mann.
http://seekingalpha.com/article/175641-climategate-revolt-of-the-physicists
Kirby is working on the idea that the controlling mechanism of climate change is cloud formation driven by cosmic rays which are modulated by variations of the solar wind, which in turn is controlled by sunspot activity. Who would have thought that the Sun might be a big driver of climate change?
I've only watched part of the 70 minutes, but want to come back. I can tell already that Kirby talks like a scientist. He prefaces that he's only got a theory, that it seems to fit the facts, he criticizes his theory as he goes along, and says that he has lots of questions he can't answer without more data. But he is saying that his theory hangs together, fits with a variety of different data, and is explanatory. This is 180 degrees from forecasting TEOFTWAWKI from the rings of a single Siberian tree. Are you listening Brian?
I skipped to the end of the presentation. One of his points is that you can't make conclusions about AGW without better understanding the effect of the Sun, because it's potentially an overwhelming factor. No shit.
There's more -- here's the Holy Shit headline. Kirby and others have expected a decline in sunspots for a few years now -- so far validated by experience. Thus they expect that the Earth is entering a cooling period. Reread that last sentence slowly.
The Truth is Out There
By Brian, at
Sun Dec 06, 07:55:00 PM:
Anon (btw, you could try clicking on the Name/URL button and acquiring an identity): I work for a local environmental non-profit in the SF Bay Area. The info is available on my website that you can get to by clicking my name, so no huge detective work is required to figure this out.
Commenting here is done in my free time.
I hope that's helpful, although it may not satisfy any conspiratorial needs.
By , at
Sun Dec 06, 11:52:00 PM:
Brian, you're a plaintiff-side environmental lawyer. Say it loud, say it proud. Some of you do God's work, at least some of the time. You aren't necessarily hopelessly conflicted like Gore, Browner or Jones.
But tell us true. You and your colleagues stand to do well from increased regulation and litigation over greenhouse gases, won't you? That makes what you say suspect ... not wrong ... but suspect.
Personally, I honestly believe that AGW science is bent. See my post above about physicists and AGW. That's where my digging has got me. The more I dug the more convinced I got. You couldn't invent more ways to pervert the scientific method.
Worse than the science, is MSM coverage. Like many things in the last decade they've done an awful job on this.
Here's a subtle but I think important fallout of MSM coverage of AGW: There's over 20 million Rush listeners -- truck drivers etc etc -- who now know a lot about Siberian tree rings. But Katie Couric doesn't want to know from Siberian tree rings. The ombudsman at the New York Times just gave it a pass on not knowing from Siberian tree rings. But Katie and the NYT will say Sarah Palin is a no-nothing. Don't you understand what you're breeding here politically? ps. my Dad was a truck driver.
These Rush listeners now know more about Siberian tree rings than most college faculty members. That's a profound disconnect.
Worse than the science is the politics. Nothing on the table in Copenhagen will fix the AGW problem, if there is a problem. What the fuck are these world leaders doing in Copenhagen? It's a farce.
Watch what happens when Obama returns from Copenhagen expecting a triumph for cutting a "deal" that kills US jobs over bent science. He may be heralded in some quarters -- and MSM -- but the rest of America will be seething. Many of these folks slid over to Obama in 2008 over disgust with Bush -- my Dad likely would have -- but you can't tell where they'll go in 2010 and 2012. If you really piss off these kinds of people be ready for peasants at your doorstep with firebrands and pitchforks.
By Brian Schmidt, at
Mon Dec 07, 03:01:00 PM:
I'm not sure this thread should be about me, but I'm not a plaintiff side litigator. I was in the past, but I haven't been practicing in courtroom for six years. Like I said, I work for an enviro nonprofit.
In one way we'd benefit from more regs, but not from cap-and-trade or from a carbon tax. Both of those are market solutions that reduce the need for our type of work.
And no one's getting any accurate info from Rush.
By , at
Mon Dec 07, 04:27:00 PM:
"And no one's getting any accurate info from Rush."
No one's getting any information at all from MSM.
A random truck dirver is likely to know more about what's going on than a random resident of NYC's Upper West Side or a random college professor.
