Exploring Ophthalmology’s Orchard

A recent study on the impact that the introduction clinicaltrials.gov has made on the reporting of clinical trial results got me thinking (1). Clinicaltrials.gov was created thanks to a US law passed in 1997 that required all researchers (from the year 2000 onwards) to pre-specify the methods they were going to employ in the trial – and the outcomes they were going to measure.

Why did the US government mandate this? At the time, some companies were being accused of commissioning many small trials, but only reporting the ones with positive results – or cherry-picking data by switching out a trial’s primary endpoint evaluation post hoc to a secondary one that gave better results. But how much of this was hyperbole by the pharma industry’s critics?

It’s reasonable to assume that the impact of clinicaltrials.gov on the number of trials displaying positive results is a good marker of how fairly the pharmaceutical industry had been treated. According to the aforementioned study, the launch of clinicaltrials.gov fifteen years ago did have had a striking impact on the proportion of favorable trial results reported – but not in the way you might think. The study authors examined 55 large clinical trials of cardiovascular disease interventions published between 1970 and 2012. An impressive 57 percent of those studies performed before the year 2000 reported positive effects; after 2000, it dropped to just 8 percent. Pretty damning, until you realize that a US government body (the National Heart Lung, and Blood Institute) funded all 55 trials.

I suppose that raises two questions. First, has mandatory and open registration of clinical trials (and its associated scrutiny) halted a massive phenomenon of cherry-picking? Perhaps. Or second, has it stopped sloppy scientific methods that previously led to immense false positive bias? Possibly. But I believe it’s mostly down to the disappearance of low-hanging fruit as we neared the turn of the millennium. This was certainly the case for cardiovascular disease where, by 1970, the first β-blocker had only recently been discovered, and commercially available statins and angiotensin inhibitors were still over a decade away. The big wins were won well before 2000, and patients entering clinical trials by that date were better treated, making it harder for a new drug to beat standard of care.

However, I don’t think that the low-picking fruit argument applies to ophthalmology. We’re still in the new blockbuster phase for many drugs, and of course devices (like MIGS stents) are always going to be disruptive to pharmacotherapies. I therefore wonder if a similar analysis in ophthalmology might tell a different story