Altered States

Though augmented reality (AR) has proved successful in enhancing the corporate user experience, it is yet to have a similar impact in the healthcare sphere, with most AR applications aimed at physicians, rather than patients. A team at the Keck School of Medicine, USC, is hoping to change that with a patient-focused design: a pair of AR glasses. Designed for visually impaired patients, the glasses operate on an intelligent AR system that overlays objects with four bright, distinct colors – blue, green, red (for far vision) and white (for close vision). These visual color cues help people with constricted peripheral vision interpret complex environments, allowing them to avoid obstacles in dim lighting. A nice idea, but how do the glasses perform in the real world?

In a trial of patients with retinitis pigmentosa (RP)-induced impairment – down to five degrees of vision – the AR glasses were found to improve mobility and grasp performance by 50 percent and 70 percent, respectively. the improvements are apparently gained by the device’s ability to render the structure of a room in real time – a process called simultaneous location and mapping – which translates information into a semitransparent overlay. In short, by highlighting potential obstacles with colored visuals, patients benefited from improved spatial understanding and depth perception.

Mark Humayun, Director of the USC Allen and Charlotte Ginsburg Institute for Biomedical Therapeutics, Co-Director of the USC Roski Eye Institute and University Professor of Ophthalmology at the Keck School, and Anastasios Angelopoulos, project lead in Humayun’s research laboratory at the Keck School, explain how – and why – the glasses were so effective. “Clinical results have shown that advanced RP patient mobility performance is highly dependent on contrast sensitivity. Thus edges are very important to RP patients, so an effective aid must both enhance edges when they are in the field of view, and also supplement the lack of edges when there are none in the field. We use color to perform this substitution.”

“It has long been known that object color is important for edge identification and motion tracking. A system which completely overlaps objects’ natural color would interfere significantly with these cues,” they continue. “Our approach splits the difference, co-opting some of the perceptual edge-enhancing properties of color for the purposes of depth detection while attempting to retain the object’s natural color which can be seen through the wireframe.”

The results offer hope to the one in 30 Americans over the age of 40 who experience a visual impairment that cannot be corrected with glasses, lenses, medication or surgery. Currently, there are few options available for RP, with most moderate cases relying on canes for mobility and electronic text-to-voice conversion tools for reading, leaving late-stage RP patients with only one treatment option: retinal implants. But the glasses have their limitations. The technology does not perform well in fast-changing environments, though this could change in the future with faster processors.

The second problem is cost. “The AR glasses currently cost $3500 – however, as these cutting-edge devices become more common, this price will drop due to scale. It is worth noting that even at the current price, the glasses are much cheaper than other similarly high tech visual aids,” explain Humayan and Angelopoulos. “In the future, we hope to evaluate different visual encodings besides pseudocolor to understand what depth map is optimal for helping blind people with navigation; we also hope to test in more types of blindness”.