Dr Peter Ridd Explains The Reproducibility Problem In Science

The reproducibility problem is well-recognised in the science community, although the potential implications of the problem, especially to government policy formulation, have been largely ignored.

In badly afflicted fields, such as the biomedical sciences and psychology, roughly half of peer-reviewed science reports and papers are likely to be faulty.

Due to the scale and ubiquitousness of the problem, it is inconceivable that irreproducible ‘science’ (i.e., pseudo-science) has not been used as evidence underpinning government policy.

The reproducibility problem is, to a large extent, a consequence of the failure of the peer-review system. Peer review is not the rigorous process that is generally assumed by the public. It is often nothing more than a cursory examination of the work, for a few hours, by a couple of ‘peers’. Peer review is thus a very limited quality assurance process and far from the ‘gold-standard’ that is often claimed by science organisations.

The reproducibility problem is only one of the issues caused by too much dependence on the peer-review process. The other major problem is that peer review of science publications, and science funding applications, is a system almost guaranteed to produce groupthink. Groupthink is the enemy of science.

Science evidence used by industry is far less likely to be afflicted by reproducibility problems than evidence used by governments. Industrial and commercial entities will generally use quality assurance systems that go much further than peer review.

The main concern of this inquiry should be the extent to which faulty research evidence has been used to underpin government policy and regulations, especially in fields where a degree of ideology may have formed under the façade of ‘science’. This includes policy relating to environmental issues, agriculture, education, social policy, criminology, and some aspects of public health.

One possibility is that the government establish quality assurance systems to audit research evidence used to underpin decisions. An option would be the establishment of an Office of Science Review (OSR). The OSR would need to be established within the National Audit Office to ensure independence from the organisations it would audit. Another option would be to require the UKRI to set aside funds to undertake reproducibility studies.

Download Dr Peter Ridd’s evidence on behalf of the Global Warming Policy Foundation for the Reproducibility and Research integrity inquiry for the UK Parliament here.