Artificial intelligence (AI) has transformed ophthalmic imaging, but one persistent limitation remains: data. While common conditions such as diabetic retinopathy and glaucoma benefit from large, well-annotated datasets, rare eye diseases – collectively numerous but individually uncommon – remain underrepresented.
A new npj Digital Medicine study introduces a potential solution: a multimodal generative foundation model capable of synthesizing clinically realistic ophthalmic images from descriptive text prompts.
The model, EyeDiff, leverages diffusion-based text-to-image generation to produce lesion-preserving images across multiple imaging modalities, including color fundus photography, OCT, and fluorescein angiography. Unlike earlier generative approaches, which were typically limited to single modalities or specific tasks, EyeDiff is designed as a generalist system trained on more than 42,000 images spanning 14 modalities and over 80 disease categories.
At its core, the challenge EyeDiff addresses is data imbalance. In real-world datasets, rare diseases and minority subtypes are often sparsely represented, limiting the ability of deep learning models to learn meaningful features. Traditional solutions such as oversampling or conventional augmentation can mitigate this to some extent, but often at the cost of overfitting or reduced generalizability.
By contrast, EyeDiff generates synthetic images conditioned on structured clinical text, effectively creating “digital twin” datasets that preserve disease-specific features. As such, the model is capable of reproducing key pathological signs – such as bone spicule pigmentation in retinitis pigmentosa or the macular changes observed in diabetic retinopathy – with notable fidelity to real-world reference images.
Importantly, these images are not merely visually convincing. In blinded evaluations, ophthalmologists misclassified up to two-thirds of generated images as real, suggesting a high degree of clinical realism. Quantitative metrics also demonstrated strong alignment between text prompts and generated images across modalities.
The critical question, however, is whether such synthetic data improves diagnostic performance. Across 11 external datasets, EyeDiff consistently boosted diagnostic accuracy for both rare and more common eye diseases. Notably, diagnostic improvements were most pronounced in underrepresented classes. In early glaucoma detection, AUROC rose from 0.860 to 0.927 following augmentation, while performance in severe diabetic retinopathy and proliferative disease also improved significantly. These findings suggest that synthetic data may help address one of the most stubborn barriers in medical AI: the long tail of rare and heterogeneous conditions.
The findings indicate that generative AI could play a key role in enabling more robust, generalizable diagnostic systems – particularly in subspecialties where data scarcity is a limiting factor. Additionally, the integration of text-driven image generation opens new possibilities for education, simulation, and data sharing, especially in environments constrained by privacy regulations.
However, the study authors also note that caution is warranted. Synthetic images, they say, can still exhibit some noticeable differences from real visual data, such as deviations in color and lesion location, which could potentially introduce diagnostic bias. In addition, the current data used to train EyeDiff “still lacks sufficient population diversity,” possibly affecting the model’s generalizability. As such, further work is now needed on developing algorithms that can interpret prompts that are more nuanced, as well as expanding the dataset so that it allows for more diverse, real-time data, to address these issues.
A new npj Digital Medicine study introduces a potential solution: a multimodal generative foundation model capable of synthesizing clinically realistic ophthalmic images from descriptive text prompts.
The model, EyeDiff, leverages diffusion-based text-to-image generation to produce lesion-preserving images across multiple imaging modalities, including color fundus photography, OCT, and fluorescein angiography. Unlike earlier generative approaches, which were typically limited to single modalities or specific tasks, EyeDiff is designed as a generalist system trained on more than 42,000 images spanning 14 modalities and over 80 disease categories.
At its core, the challenge EyeDiff addresses is data imbalance. In real-world datasets, rare diseases and minority subtypes are often sparsely represented, limiting the ability of deep learning models to learn meaningful features. Traditional solutions such as oversampling or conventional augmentation can mitigate this to some extent, but often at the cost of overfitting or reduced generalizability.
By contrast, EyeDiff generates synthetic images conditioned on structured clinical text, effectively creating “digital twin” datasets that preserve disease-specific features. As such, the model is capable of reproducing key pathological signs – such as bone spicule pigmentation in retinitis pigmentosa or the macular changes observed in diabetic retinopathy – with notable fidelity to real-world reference images.
Importantly, these images are not merely visually convincing. In blinded evaluations, ophthalmologists misclassified up to two-thirds of generated images as real, suggesting a high degree of clinical realism. Quantitative metrics also demonstrated strong alignment between text prompts and generated images across modalities.
The critical question, however, is whether such synthetic data improves diagnostic performance. Across 11 external datasets, EyeDiff consistently boosted diagnostic accuracy for both rare and more common eye diseases. Notably, diagnostic improvements were most pronounced in underrepresented classes. In early glaucoma detection, AUROC rose from 0.860 to 0.927 following augmentation, while performance in severe diabetic retinopathy and proliferative disease also improved significantly. These findings suggest that synthetic data may help address one of the most stubborn barriers in medical AI: the long tail of rare and heterogeneous conditions.
The findings indicate that generative AI could play a key role in enabling more robust, generalizable diagnostic systems – particularly in subspecialties where data scarcity is a limiting factor. Additionally, the integration of text-driven image generation opens new possibilities for education, simulation, and data sharing, especially in environments constrained by privacy regulations.
However, the study authors also note that caution is warranted. Synthetic images, they say, can still exhibit some noticeable differences from real visual data, such as deviations in color and lesion location, which could potentially introduce diagnostic bias. In addition, the current data used to train EyeDiff “still lacks sufficient population diversity,” possibly affecting the model’s generalizability. As such, further work is now needed on developing algorithms that can interpret prompts that are more nuanced, as well as expanding the dataset so that it allows for more diverse, real-time data, to address these issues.
