RETFound Revolution

In the last year – amazingly – we have seen a rapid acceleration of developments in artificial intelligence (AI). This has been predominantly driven by the introduction of “foundation models,” such as ChatGPT, and through the advent of generative image models, such as Midjourney.

Current AI models are typically created using an approach called “supervised” learning. In this method, human experts are used to classify data. For example, an ophthalmologist might be asked to give a diagnosis for a retinal scan or identify individual disease features. The problem with this approach is that often hundreds of thousands, or even millions, of labels are required. This is both very costly (in terms of human expert time) and time-consuming. It also means that many current AI systems have been limited to applications where large, labeled datasets are available, such as in diabetic retinopathy screening.

It was in line with these advancements outside of healthcare that Moorfields Eye Hospital and the UCL Institute of Ophthalmology were inspired to develop RETFound – the first foundation model in ophthalmology – and one of the first in any medical specialty (1).

How did we do it? We first identified a new and exciting form of self-supervised learning called a “masked autoencoder” – an approach developed by Facebook AI Research in 2021. After adapting this approach for use with retinal images, color fundus photographs, and optical coherence tomography (OCT), we built a vision transformer neural network architecture consisting of 300 million parameters. We trained it on over 1.6 million retinal images using some relatively powerful graphical processing units (Nvidia A100s) using our Cloud-based data pipeline at Moorfields. Finally, we fine-tuned it for over 10 downstream clinical tasks, including diabetic retinopathy screening, age-related macular degeneration (AMD) prediction, and a wider range of oculomic tasks!

RETFound has been shown to perform better than other state-of-the-art comparison approaches (2). As well as being more robust on external validation, the platform works well across different ethnic groups and is very label efficient. The latter is of particular significance as we believe it will open up the possibility of developing clinical AI systems for less common retinal diseases, such as uveitis and inherited retinal disease.

You may be surprised to hear that RETFound is open source, meaning that other people can also enjoy its benefits. On a large scale, we are hopeful that it will become a cornerstone for global efforts in the fight against blindness. On a smaller scale, we would like to see others fine-tune RETFound for local applications to improve existing use cases, such as diabetic retinopathy screening. Could there be a RETFound-Brazil or a RETFound-Ghana?

Although RETFound has been trained on just under two million retinal images, there are more than 20 million images still in the pipeline at Moorfields. And so, as a natural next step, we would like to scale it up, greatly increasing its model size and complexity. We’re also planning to make RETFound truly multi-modal – first, with other forms of ophthalmic imaging, and secondly, with other forms of data – from text to genetic information.

Watch this space… The best is yet to come!

^{Credit: Image sourced from Unsplash.com}