Stats on stats

How much statistical expertise is required to comprehend the ophthalmology literature? To find out, a recently-published study (1) surveyed the statistical techniques used in all articles published in 2012 in Ophthalmology, the Am. J. Ophthalmol., and Arch. Ophthalmol.

To get a handle on just how understandable the literature is, the authors “estimated the accumulated number and percentage of articles that a reader would be expected to be able to interpret depending on their statistical repertoire”.

With little or no statistical knowledge, a reader could interpret the statistical methods presented in two in every ten (20.8 percent) articles. To understand more than half (51.4 percent) of the articles, familiarity with at least 15 different statistical methods is required, rising to 21 different categories of statistical methods to comprehend the content of 70.9 percent of articles, and to 29 statistical methods to comprehend fully over 90 percent of articles. When the authors looked at the subspecialties of ophthalmology, articles that pertained to retina and glaucoma tended to use more complex analysis than those from the corneal subspecialty.

The study raises some questions. Is the journals’ readership sufficiently statistically numerate to be able to fully participate in scientific discourse? Are the reviewers? Do ophthalmologists receive adequately trained in statistical methods?

Adam Jacobs, statistician and director of the statistical consultancy firm, Dianthus Medical, has a different interpretation. “The research is framing the question in the wrong way,” notes Jacobs. “I don’t think it’s necessary for an ophthalmologist – or any other clinician – to be an expert in a wide variety of statistical techniques”, asking, “surely it’s the job of the study author to ensure that tricky statistical techniques are explained clearly so that non-experts can still understand what’s going on?”