What's on Our (DR) Radar?

Diabetes is becoming a global epidemic; the number of people affected worldwide has risen from 108 million in 1980 to an estimated 463 million in 2019, with an estimated 700 million by 2045 (1). Diabetic retinopathy (DR) and its sight-threatening effects are expected to sharply rise along with the increasing population of diabetics. DR is already one of the leading causes of blindness in the working age group. Establishing robust DR screening programs, as achieved during the last two decades in the UK, has been shown to be effective in tackling this growing issue (2); however, the current model of DR screening is prohibitively expensive for all but a few highly developed countries.

Annual screening for every diabetic, performed by an ophthalmologist, is very expensive and inefficient. It is becoming increasingly clear that not everyone requires yearly screening. Some patients could be safely screened every 2–3 years, for example, presenting a significant cost-saving opportunity. One initiative to target this issue is Retina Risk – an algorithm for stratifying individual risk of developing sight-threatening DR, developed by a team led by Einar Stefansson, Professor at the Department of Ophthalmology at the National Hospital Reykjavik, University of Iceland. Using several inputs, including HbA1c level, Retina Risk is able to calculate if patients are likely to progress to vision threatening DR within the next six months or 1–3 years, then recommending an appropriate follow-up time based on the risk. The majority of diabetic patients do not have DR and, even if they do, only a fraction of them require treatment. Targeting this relatively small – although in absolute numbers very significant – group of patients is the main aim of screening.

The population of diabetics is steadily rising, but the number of ophthalmologists needed to treat them cannot rise fast enough to meet the demand. On average, low-income countries have fewer than four ophthalmologists per 1 million inhabitants; even in high-income countries the number is 76 (3). And so, even in high-income countries, thorough screening programs based on examinations by ophthalmologists are simply not feasible.

Where does AI come in?

AI may help us solve some of these issues, but it should not be mistaken for a cure-all solution. There are many deep-learning algorithms for screening of DR based on fundus images available today – and there are more on the horizon. (4) However, to be effective on a larger scale, these need to be used in pre-ophthalmological, community-based screening settings.

To reach a wider population that is not already being surveilled by ophthalmologists, AI DR screening solutions should be centered around primary care or diabetes clinics. With modern automatic fundus cameras, minimal training is required to allow staff, such as nurses, to take good quality fundus images in the majority of cases. And though it is clear that more staff would be required to capture retinal images overall, the personnel could also provide health education and advice.

We know that screening for sight-threatening DR using fundus images is not a flawless solution; some cases of isolated diabetic macular edema could be missed. We could of course train graders to take and classify OCT images, but fundus image-based screening remains the cheaper and still effective option. This reality was highlighted by the trial that led to the FDA clearance of IDx-DR – an AI based DR screening solution. Effectiveness of the AI system, basing only on two fundus images per eye, was compared with a professional grading center with access to not only stereo widefield retinal photographs, but also OCT. The software identified more than mild DR with 87.2 percent sensitivity and 90.7 percent specificity. 84 percent of center-involved DME cases were identified by the system. Inevitably, some cases will be missed, but human-based grading is also not infallible.

AI used for DR screening in the real world

Up to now, only the UK and Singapore have introduced comprehensive DR screening programs, mostly because of the considerable associated costs (2,4). In Poland, where we practice, the population of diabetics is around 3 million; from a screening point of view that’s 12 million images that must be analyzed. The upshot? Introducing comprehensive DR screening in Poland was not feasible. But introducing AI into this space has completely changed the mindset of the country’s DR specialists – the impossible has suddenly become realistic.

During 2018–2019, we conducted pilot studies in Poznań, Poland, starting with a single diabetes clinic with a population of 600 patients. We used pre-ophthalmic screening, working with diabetologists to screen at-risk patients; we wanted to screen as many diabetic patients as possible in the amount of time we had available to make the pilot program as effective as possible (5). Patients in the care of diabetic clinics are also likely to present more advanced stages of diabetes and, as a result, a higher likelihood of DR. In the future, general practitioners/family physician practices could be used to conduct the initial phase of the screening. After the first 600 cases, we then screened a further 1,000 patients. Afterwards, with lessons learned and teething problems largely resolved, we began a large-scale project, co-funded by the European Union, to screen over 40,000 diabetics in 30 diabetic clinics in the Polish region of Wielkopolska over three years. The project is still ongoing (6).

Our overall goal is to present large-scale data that would convince policy makers (in Poland: the Ministry of Health and the National Health Fund) that introducing a large-scale DR screening program in the future will be beneficial to the country’s healthcare system and that it should be considered a priority.

Wielkopolska has a population of 3.6 million people (around 10 percent of the country’s population) and a successful DR screening program in the region would be a good indication of positive screening outcomes across Poland.

Per aspera… ad astra

To conduct a project on such a large scale, we purchased fundus imaging equipment and trained specialist nurses to obtain consent, take images, upload them to the cloud, provide education, and care for patients involved in the project. The clinics and the staff responsible for capturing images have received financial compensation. Our aim is to perform around 1,500 exams every month; before the COVID-19 pandemic hit, we were averaging around 800-900 exams monthly. Providing support to so many clinics and trained graders at the same time has naturally proved to be a challenge. In this real-world study, we have had to rely on professionals with varying degrees of computer literacy and willingness to learn. We have had some nurses refuse to take part in the project as they considered this new training to be beyond their capabilities. In those cases, we had to source suitable locally-based personnel, such as trained optometrists outside the clinic.

Images were captured according to a standardized imaging protocol – with one disc- and one fovea-centered 45-degree image per eye – and submitted to an autonomous AI-based DR system (IDx Technologies) for automated image quality evaluation. The system returned results of i) “more than mild” DR not detected, ii) “more than mild” DR detected, or iii) vision-threatening DR detected. Patients with “more than mild” not detected results were given a recommendation to attend another screening in one year’s time; all other results recommended an urgent ophthalmology referral. The images were also reviewed by ophthalmic specialist graders, and, if inconsistencies with the AI algorithm were detected, patients were notified at their next diabetic clinic visit (in the case of a false-positive result) or by phone (if the result was a false-negative).

Notably, the AI system has previously received a class II certification in Europe as well as US FDA authorization and may be used as an autonomous screening device, meaning that the results can be relied upon for a referral decision without review by a clinician.

Our findings

Based on preliminary analysis of data gathered from the first 450 screening episodes, we determined that the IDx-DR system was able to analyze 78 percent of screening episodes, compared with 82 percent in the case of a human grader. Overall, according to a single clinician reference standard, sensitivity was 94 percent and specificity was 95 percent, with positive predictive values of 82 percent and negative predictive values of 99 percent. We found the system easy to implement and use. Automatically saved results were available for analysis within a minute. In the future, we would like to conduct a cost-effectiveness analysis, acquire additional data of patients’ DR risk factors, and expand the program beyond the high-risk population. So far, the program has proven to be very successful, with our accuracy results higher than those previously reported in the literature, including the FDA pivotal clinical trial (7-9). Moreover, based on our in-growing database of fundus images, recently we started testing other AI algorithms for DR screening. and we are open to collaborate on this subject with teams working on novel ideas.