Events move so fast that – the cliché goes – a week is a long time in politics. Science, on the other hand, usually moves at a comparatively glacial pace – both in execution and dissemination. However, a recent exception is the hype and controversy related to the publication of a study related to bacteria that can grow in high amounts of Arsenic (As), an element hitherto unknown to support life, where blogs have contributed to an alternative, accelerated feedback mechanism.
This write-up is an attempt to briefly outline this story along with a few opinions of my own. Many excellent blogs and comments therein have helped to shape some of my thoughts on this issue. These have been acknowledged in the post and in the ‘Further Reading’ section at the end. I hope this is also useful for those who haven’t followed the events closely (I have tried to keep the science as simple as possible).
On December 2nd, a group of astrobiology researchers led by Felisa Wolfe-Simon from NASA published a paper in the reputed journal Science, claiming to have found a ‘new form of life’ that utilized arsenic in place of phosphorus (P). It is known that life as we know it utilizes only carbon, nitrogen, oxygen, phosphorus, sulfur – and some trace metals – in its building blocks. Therefore, this research had the potential to be paradigm-shifting.
The controversy started with the fanfare accompanying the publication. Science, while making the report available to journals few days earlier, placed an embargo on its general release. Meanwhile, NASA’s public relations department got into act, calling for a press-conference to announce a major discovery that was supposed to “impact the search for evidence of extraterrestrial life”. Heady claims indeed! Unfortunately, this unnecessary secrecy – combined with the publicity – resulted in high expectation and even few wild speculations on the internet, including claims that NASA was going to announce discovery of extra-terrestrial microbial life!
In this backdrop, when Science eventually lifted the embargo (one and half hour earlier to quell further rumors) the research paper itself was bit of a let-down to scientists such as myself. What the researchers demonstrated was that they could gather bacteria from Mono Lake in California (where exists extreme living conditions) and one particular species, dubbed GFA-1, was able to grow in high concentrations of As in the media. However, it wasn’t as if the bacteria grew only in As, in fact it grew much better in P. Additionally, biochemical and biophysical experiments indicated only indirectly that the bacteria was incorporating As in proteins, lipids and most importantly, DNA.
This was a bit of – pardon the street expression – a meh. From my personal scientific point of view, the most disappointing aspect of the study were (a) the lack of proof that As actually formed part of the DNA backbone and (b) the lack of speculation by the authors about possible mechanisms for various enzyme to utilize arsenate instead of phosphate. These factors made the paper merely interesting, instead of great or ground-breaking.
Initially, a few scientists and science journalists, mainly through blogs, called out NASA on their claims of these research being associated with search for extra-terrestrial life or even of this bacteria being a ‘new life form’. Even while the mainstream media was running with the extra-terrestrial angle, blogs by Ed Yong and Anirban painted a more sober picture.
Thereafter, events unfolded rapidly. In addition to doubts about the impact of the results, a number of scientists started voicing concerns – again through blogs – about the lack of scientific rigor and proper controls in some of the experiments performed. Most prominent of these were Prof Rosie Redfield – a microbiologist at University of British Columbia – and Prof. Alex Bradley, a biologist at Harvard. Additionally, noted science author Carl Zimmer got in touch with a wide range of experts to solicit their opinions and wrote two articles – one for the general audience in Slate and a follow-up at his own blog, The Loom (the latter with the detailed critiques from all the experts). Most of the expert opinions were negative, with one scientist even stating “This paper should not have been published“!
I won’t go into the technical details, but do read the blogs mentioned if you are interested in the critiques. Briefly, the main flaw of the paper seems to be the lack of actual proof of As in the DNA backbone, which can be proved/disproved by some simple experiments. Additionally, an important step in the original experiments involved placing purified bacterial DNA in water. Now if this happened with an As replaced backbone-DNA, by all known chemical mechanisms, the DNA should have disintegrated. This did not happen. Thus many experts are speculating that the bacteria could be surviving in As medium by using the tiny amounts of phosphate that contaminates As buffers. This is quite contrary to the author’s claims of an As utilizing bacteria.
Unfortunately, NASA’s response to these critiques was a bit bizarre. When asked by CBC to comment on the criticisms leveled against the paper,
[NASA spokesman] Dwayne Browne …… noted that the article was peer-reviewed and published in one of the most prestigious scientific journals. He added that Wolfe-Simon will not be responding to individual criticisms, as the agency doesn’t feel it is appropriate to debate the science using the media and bloggers. Instead, it believes that should be done in scientific publications. (link) (emphasis mine)
Reactions to this comment were predictably swift and scornful. Though there were a few voices in support, on the whole, NASA was (IMO quite correctly) castigated for dismissing blogs as a medium of scientific discussion. Particularly galling was the hypocrisy of NASA in releasing research through the press and then not wanting to be held responsible in the same public sphere!
As many people have pointed out already, NASA or the authors do not have to respond to every blog and every comment against the work. However, to disregard skepticism just because they are not ‘peer-reviewed’ is bad form and exposed a stunning old-fashioned view from an institution that is supposed to be at the cutting edge.
As things stand now, Dr. Wolfe-Simon has since posted an update on her personal blog stating they are “preparing a list of “frequently asked questions” to help promote general understanding of our work”. No word yet though on how they plan to address the critics.
My own thoughts:
Impact of the work: From my own reading and the reasoned arguments by various scientists, it is very difficult to agree with the original interpretation that As is somehow being utilized by these organisms and incorporated into their bio-molecules. If some of the experimental results do hold up to scrutiny and reproducibility, this could still be an interesting find (e.g it might find some application in As scavenging with As in ground water being such a big issue in many countries). However, the work is nothing on the scale of discovering new forms of life, much less anything to do with extraterrestrial life.
Who was wrong here: Some people have accused the authors of this paper of dishonesty or outright fraud. However, I doubt there is any evidence of that, especially given the fact that they seem to have published most of their scientific data – even data that do not support their conclusions fully. At best the authors are guilty of over-interpretation and not performing proper controls. This could and does happen to any scientist, and both pre- and post-publication peer review corrects such mistakes.
It is probably fair enough to say that the pre-publication peer-review by Science should have done a better job at catching some of these glaring errors. It is also possible that reviewers did raise concerns but were overruled through editorial decision and over-eagerness to publish a high impact paper. But this is pure speculation.
One party that has come out looking really bad and severely tarnished its image over the incident is NASA. They have been rightly criticized in many quarters for jumping the gun and unnecessarily hyping an incomplete work, especially with the non-existent extra-terrestrial angle. As Athena Andreadis points out, NASA has done a great disservice to the entire science community and possibly to the lead researcher on this paper.
By disbursing hype, NASA administrators handed ready-made ammunition to the already strong and growing anti-intellectual, anti-scientific groups in US society: to creationists and proponents of (un)intelligent design; to climate change denialists and young-earth biblical fundamentalists; to politicians who have been slashing everything “non-essential” (except, of course, war spending and capital gains income). It jeopardized the still-struggling discipline of astrobiology. And it jeopardized the future of a young scientist who is at least enthusiastic about her research even if her critical thinking needs a booster shot – or a more rigorous mentor.
Impact of blogs: If there is a silver lining in the whole incident, it is lively role taken up by blogs of post-publication peer review (as well as the ensuing debate of the pros and cons of the role of blogs in scientific communication).
As I already mentioned, when the work was first announced, it was mostly scientific blogs such as Ed Yong’s that offered the most balanced perspective. Most of the mainstream media was focusing (including some prominent science-related media – such as NPR’s Science Friday or Neil deGrasse Tyson’s radio show) with the alien life angle. Thereafter, all of the criticisms of the paper came through blogs. In the pre-Web2.0 days, the paper would probably have been discussed within faculty and students at various institutions – at water-coolers or more formally, at journal clubs. However, these would happen in an isolated manner, with no counterpoints being offered on such a wide scale. A wider forum for discussions would have been scientific conferences if any of the authors presented their work there. Even then, discussions would be limited to a few questions asked from the audience. Effective criticism would only happen through letters written to Science. The peer-review by various blogs essentially sped up this process of scrutiny. And this is good for science.
Personally, I have long advocated the interactive capabilities of the internet as a pseudo-conference platform. Such forums are much more inclusive, and as demonstrated here, much speedier (not to mention cheap, with no travel or accommodation cost!). The main complaint is that it could draw in a lot of non-experts, or people with agenda. However, one can argue that in conventional peer-review it is easier for personal agendas to seep through, given that reviewers are mostly anonymous. Also, while trolling could happen (especially in case of politically charged topic), I’d like to think (perhaps naively) that scientists are good at ignoring illogical arguments. The comments section of the blog by Prof Redfield is a fine example, with a rich scientific discussion in progress. These include criticisms of Redfield’s critiques too (followed by counterpoints to the critiques of the critiques – as I said, a lively discussion). Of course, safeguards have to be built in too – but a full discussion of how an interactive web for science communication would work is beyond the scope of this post.
Worth mentioning here as well that this is not the first instance where blogs took the lead in criticizing a high impact publication – earlier in the year, a paper claiming novel high-throughput chemistry had the same experience. Those findings did not have as much mainstream audience – hence was not covered as much.
Of course, such large scale discussions are not going to happen with every paper published. This was a big impact paper and hence got the attention. However, I think this incident has got the ball rolling and got many in the community thinking about how to best leverage web-discussions efficiently for science. Prof Redfield herself has some good suggestions on her blog.
One good first step would be encouraging more scientists to blog. Currently, their number is small, but the quality of blogs written by true-blue scientists is exceedingly good. This includes a full spectrum from academic scientists such as Redfield (who also encourages her students to blog), Stephen Curry etc to those in the industry – Derek Lowe being a prominent example. Also heartening is the fact that many new science graduate students are taking to blogging (my own anecdotal observation though).
Finally, for those who continue to argue and side with NASA that scientific discussions must occur within the confines of peer-review, David Dobbs has the best answer on Wired:
[W]e should remember that the printed science journal was originally created to expand the discussion of science, allowing it to move beyond face-to-face contact at salons and saloons and meetings and into a medium more widely shared. It’s silly to now cite the printed journal’s traditions as a way to limit discussion.
Scientific debate and the wider public: An argument was made during the early part of this story that scientists should present an united front when talking to the general, lay audience, in order to not provide further ammunition to the anti-science community such a climate-change skeptics, creationists, anti-vaccination nuts and their like. However, I strongly disagree. What makes science different from religion or politics is the inherent transparency and the messy democratic process of peer-reviews and open criticisms. People who do not believe scientific evidence for evolution etc are often motivated by other political/religious factors – I doubt they will be convinced ever even if presented with clear evidence. Scientists need not forsake openness in debate for this extreme group. I completely agree with Dobbs (again!) that this sort of public messiness can only be good for science.
Because if the general public sees this sort of row in routine findings — if they understand that science routinely sparks arguments over data and method — they’re less likely to see such messy debate as sinister when they see it in something like climate science.
Hopefully this incident will be a good exercise in lessons learned for future scientists and institutions hoping to make a big impact on the public or scientific stage.
One also hopes that this spurs a paradigm-shift in terms of the role of blogs and other non-traditional media in scientific communication and dissemination of scientific ideas.
Many of the opinions I wrote here have been expressed before. Some of the blogs and comments below have also helped to reinforce or better shape some of my own thoughts. I highly encourage reading these if you are interested in understanding the science in further depth.
- For absolute up to date coverage of this story, do follow Guardian’s excellent story tracker here.
- Ed Yong’s blog is a great resource – here is his early report and here is a post-mortem after a week.
- A good review initially by blogger Anirban (Bhalomanush).
- Carl Zimmer’s Slate article does a very good job of covering the controversy in simple terms, but also read detailed critiques by various scientists here.
- Prof. Rosie Redfield and Alex Bradley’s review of the original paper.
- Really good coverage – with a critique of the paper, and a scratching rebuke of the manner in which NASA has handled the affair by Athena Andreadis at her site.
- David Dobbs, Deepak Singh respond to NASA’s refusal to consider questions from blogs.
- David Dobbs analysis of how scientific communications should evolve and how scientific debate should be open. These were essentially the points I was trying to make, but Dobbs expresses it so much better.
- Finally, the paper in question itself has been made available for free, if you are interested in reading.