As the oceans’ chemistry is altered by rising levels of atmospheric carbon dioxide, the response of sea-dwellers such as fish, shellfish and corals is a huge unknown that has implications for fisheries and conservationists alike. But the researchers attempting to find an answer are often failing to properly design and report their experiments, according to an analysis of two decades of literature.
Oceans absorb much of the CO2
emitted by human activities such as coal burning. This leads to a variety of chemical changes, such as making waters more acidic, which are referred to as ocean
acidification. The United Nations has warned that ocean acidification could cost the global economy $1 trillion per year by the end of the century, owing to losses in industries such as fisheries and tourism.
The past decade has seen accelerated attempts to predict what these changes in pH will mean for the oceans’ denizens – in particular, through experiments that place organisms in water tanks that mimic future ocean-chemistry scenarios. Yet according to a survey published last month by marine scientist Christopher Cornwall, who studies ocean acidification at the University of Western Australia in Crawley, and ecologist Catriona Hurd of the University of Tasmania in Hobart, Australia, most reports of such laboratory experiments either used inappropriate methods or did not report their methods properly.
Chemical errors
Christopher says that the “overwhelming evidence” from such studies of the negative effects of ocean acidification still stands. For example, more-acidic waters slow the growth and worsen the health of many species that build structures such as shells from calcium carbonate. But the pair’s discovery that many of the experiments are problematic makes it difficult to assess accurately the magnitude of effects of ocean acidification, and to combine results from individual experiments to build overall predictions for how the ecosystem as a whole will behave, he says.
The survey was based on a search of the Scopus database of research papers. Christopher and Catriona analysed 465 studies published between 1993 and 2014 that manipulated seawater chemistry and found that experiments often failed to implement widely accepted measures to ensure quality. For instance, to ensure robustness, manipulation studies should use multiple arrays of independent ocean-mimicking tanks. And in experiments that compare sea animals under acidified conditions with controls, these tanks should be randomised to remove bias.
But the pair found that in several papers, researchers used one main seawater tank to supply multiple, supposedly independent smaller tanks. The researchers also found mistakes in basic chemistry: Some authors simply added acid to a tank and ignored other chemical changes that result from the absorption of CO2, such as increased levels of carbonates. Although the frequency of these chemistry errors has dropped since the 2010 publication of an international “best practice” guide for ocean-acidification experiments, the researchers found no evidence for improvements in the design of tank arrays.
To keep in mind
Bayden Russell, an ocean-acidification researcher at the University of Hong Kong who reviewed drafts of the latest paper, has also noticed that some researchers fail to take into account the complexities of ocean acidification when designing their experiments. Overall, Christopher and Catriona found that in only 27 cases could they be certain that an appropriate experimental design had been used, and in 278 cases, the design was clearly inappropriate. The remaining studies had insufficient detail on experimental setup – a problem in itself. The pair present a series of recommendations for well-designed experiments and suggest a checklist of details that should be included in papers to allow replication of experiments, including which chemicals were used to manipulate the seawater chemistry and the configuration of tank arrays.
Ove Hoegh-Guldberg, director of the Global Change Institute at the University of Queensland in St. Lucia, Australia, suggests that researchers also need to take account of natural variations in temperature and CO2 in experimental setups and ensure that experiments that manipulate acidity also simulate the accompanying rise in temperature from global warming, which many do not.
Russell thinks that most research groups are now trying to use appropriate designs, but says that there are still problems, which he attributes to a variety of factors. “When superimposed on the increasing pressure to publish rapidly, and in top journals, some researchers or research groups are still attempting to publish what I would consider substandard research.” Jonathan Havenhand, who works on marine invertebrates at the University of Gothenburg in Sweden and who co-authored the 2010 guide, welcomes the latest paper: “Everybody should know the stuff that’s in Christopher and Catriona’s paper. It’s good that they wrote it. It is disappointing that they had to But whether people are going to be happy to be citing it, I do not know.”