Benchmarking Ophthalmology
Our cataloging of the major eye diseases is insightful, even though impact factor analysis is flawed.
In this issue we present benchmarking data for the field of glaucoma; last month, we did the same for cataract. The idea is to provide a series of reference points against which to judge performance and/or progress; today, benchmarking is being applied to everything from countries and corporations to schools and sports teams, as well as to scientific and medical sub-disciplines. It’s widely used by pharmaceutical companies to identify key opinion leaders in a given field or for ‘gap analyses’ for a given indication, to assess if their competitors missed a trick.
In The Ophthalmologist’s benchmarking analyses, we catalogue who published what and where; and we assess what kinds of research are being performed and what impact it makes. We do so because the findings may benefit clinical research and practice. They reveal where the attention (and, probably, funding) of the field has been focused, pointing out not only what is favored but also what is underrepresented. We don’t spoon-feed interpretations; instead, we present the data for you get to interpret as you see fit.
But one aspect of these analyses that troubles me is the Impact Factor (IF). Even Gene Garfield, who invented IFs isn’t in favor of using them to judge the value of scientific research. They are fine for “quick and dirty” assessments of journal quality, but a few ‘superstar’ papers can give a journal a high IF, even if the vast majority of papers are barely cited. It’s imprecise, but far better than nothing; just be aware of the limitations. Better metrics like Eigenfactors or SCImago are much more difficult to access.
As an aside, efforts to improve upon IFs led to the ‘recursive Impact Factor’ back in 1976 – citations from journals with high IFs attract a greater weighting than citations from low-IF journals. This was an early example of the type of algorithm, PageRank, that Google uses to rank their search results. And as any web developer will tell you, that’s the most important metric to come top of these days for visibility.
Almost nobody goes into medicine for “exposure”, they do it to heal the unwell. But if you’re competing for research funding, it truly is the case that life is a pitch. Here’s hoping that in the future, the impact and true value of your work is assessed by the best metrics possible.
I spent seven years as a medical writer, writing primary and review manuscripts, congress presentations and marketing materials for numerous – and mostly German – pharmaceutical companies. Prior to my adventures in medical communications, I was a Wellcome Trust PhD student at the University of Edinburgh.
