Forty per cent of animal studies on brain haemorrhages contain images that are incorrect, researchers from Radboudumc center discovered. Some of these publications may have been completely or partially fabricated.

It all began with a striking paradox in their research field, Radboudumc researchers René Aquarius (Department of Neurosurgery) and Kim Wever (methodologist at the Department of Anaesthesiology) recount in the canteen of the teaching hospital. Why was it that despite there being hundreds of studies with laboratory animals on treatments for a certain type of hemorrhaging, this had never delivered a drug for humans?

Hoping to find new leads, they examined 608 animal studies. Specifically, they looked at research on what is known as subarachnoid lesions, bleeding between the brain and one of the meninges. In humans, these prove fatal in one third of cases.

The analysis conducted by Wever, Aquarius and three of their colleagues led to an unexpected result. 243 of the publications they looked at (40%) were found to contain images with something wrong with them, as they describe in the latest edition of biology journal PLoS Biology. The erroneous images even appeared in top journals like Stroke.

Duplicate images

‘There were images that appeared twice in the same publication’, explains Wever, ‘or that appeared in several different articles.’ The images she and her colleagues looked at were microscopy images of animal tissue and visualisations of proteins, known as Western blots.

Initially, they had their interns manually put together the figures from a few dozen articles, then compare them at a glance. Aquarius: ‘This cursory examination already produced so many duplicate images that we decided to examine all the articles systematically using Imagetwin (a software programme, SP).’

Some of the problematic cases they discovered involved malicious intent, Aquarius and Wever believe. The scale of the problems is too great for any other explanation, they also write in their article. Aquarius: ‘Some authors had dozens of publications to their name with erroneous images.’ Wever adds: ‘Images that appeared multiple times were sometimes rotated, mirrored or otherwise edited.’ In other words, there is no way that happened by accident. Meanwhile, these issues being brought to light only led to a warning label (an expression of concern) or withdrawal of the article in one tenth of cases.

China

The findings of the Radboudumc researchers are not unique. The number of scientific articles that turn out to be partially or entirely fabricated is growing. This is partly because companies in countries such as Iran and China use this as a commercial model, the so-called paper mills. They write articles to order or sell authorships, for example via Telegram.

It is unclear whether the 243 problematic studies from the Nijmegen analysis are also products of paper mills. However, it is striking that five sixths of them came from China. Aquarius and Wever did not come across any problematic Dutch publications. One possible explanation put forward by the Nijmegen researchers in their PLoS article is that in China, researchers are under pressure to increase their research output to move up in international university rankings. And in a survey among hospital researchers in south-west China last year half indicated that they sometimes falsified research data.

‘The first question you should ask about a publication is whether the study actually took place’

The harm of such unreliable studies, Aquarius says, is that it leads researchers who build on them astray. ‘You therefore run the risk that treatments for diseases may not be discovered until later than they might have been otherwise.’ Wever: ‘Not to mention the unnecessary use of laboratory animals as a result of this.’

Tip of the iceberg

‘This article reveals a serious problem within biomedical research’, microbiologist Elisabeth Bik says when asked. Bik is known worldwide for her expertise in identifying problems with images like those in the PLoS study. ‘That 40% figure is probably just the tip of the iceberg. There can also be errors in graphs and tables, and these are even more difficult to detect. It is disappointing that the burden of raising such issues rests mostly on the shoulders of readers. Scientific publishers should try harder to prevent this.’

Aquarius does want to guard against the perception that science is unreliable. ‘It is mostly important for scientists to be more aware of this issue. The first question you should ask yourself when reading an article is whether the study actually took place. Especially when the results are much prettier and neater than you are used to from your own research or from similar literature.’ In this respect, it can already help a lot to install a plugin from PubPeer, a website that publishes critical analyses of publications.

But academia should be more critical across the board, he stresses. ‘Think of scientific journals and their editors, but also research funders, they need to start making more money available for this.’ Wever and Aquarius are proceeding with similar ‘forensic’ analyses, thanks in part to a grant from ZonMW.