Holding Court

You started out as an athlete – how did you become a scientist?

I originally came to the USA from Australia on a college tennis scholarship. I turned professional but ended up getting an injury. I started coaching and doing graduate studies at the same time. A lot of my doctorate focused on motor learning: what it takes to learn a skill. I was teaching a great athlete, a 15-year-old girl. One day, at the end of the lesson, we played a few games. She served, and I returned with a drop shot. She missed it and looked completely taken aback.

I did it again on the next point, and she moved but was too late to pick it up. I did it again on the third point and she threw her racket down, infuriated. What perplexed me was that if she had known where to look and what to look for, she would have been able to anticipate that shot seconds ahead of time. Even though she was faster and stronger than me, she wasn’t able to get to it – simply because she couldn’t see it early enough.

Around that time, I came across an article talking about a new technology called eye tracking. It was really primitive – lab-based with heavy wires everywhere – but had offered some interesting results. In a comparison of experienced versus inexperienced soccer goalkeepers, researchers found that the experienced goalkeeper looked at the rotation of the leg and the foot before the ball was kicked and was therefore able to predict its trajectory. And that’s similar to what I was seeing in tennis. In short, inexperienced players end up reacting to the ball, rather than being proactive.

How did that tie into your dissertation?

I began using eye-tracking technology to see where people were looking on the tennis court when they returned a serve. I found two things: first, skilled players looked at the same cues, every single time. Inexperienced players took in more information but didn’t know how to make sense of what they were looking at. Second, there were differences in where people looked based on their stress level. Stress actually reduces visual focus, almost like stage fright. When a person is stressed, their vision becomes narrow and inefficient.

This work led me to consulting for other sports, like Major League Baseball. I was brought down to spring training. Although it was a sport I was much less familiar with, I was able to give them a picture of where major league guys looked, which was a very narrow focus. The MLB players also looked at specific cues, or locations on the pitcher, in the same order, every time. In comparison, the minor league guys’ visual cues were much less focused and distracted. I showed the pictures (see below) to the staff and there was silence. Finally, the major league hitting coach, who’s now a good friend of mine, slammed his hand down on the table and said, “I ****ing told you guys this was important!” We still laugh about it now.

Is it true you consulted for the military?

Yes. If police officers or soldiers think there is a threat in a building, they do what is called a “stack” – a tactical team enters, each member looking in a different place to clear the space as efficiently as possible. I was asked to profile members of the stack to work out whether individuals should be first, second or third in the line-up, since each position requires different skills. But instead of just establishing where each person was looking, the project grew, and I was asked if it was possible to determine a person’s depth perception or dominant eye. The idea being, if we could determine their skills and deficiencies, they could potentially be trained to cross over to other positions. This made me take a broader look at vision. It wasn’t just about visual search anymore (looking at the environment for cues to determine what to do next), but about fundamentals of vision.

What happened next?

Around 2000, eye-tracking technology became lighter and cheaper as parts were more readily available. I began moving from a consulting model, which was very time and labor intensive, to processing information automatically. In 2012, I was actually playing tennis one night with my now business partner, Adam Gross, when we decided to create a product that automated my consulting work and took me out of the equation altogether! The goal being that this work was important, and more people needed access to a tool that could provide automatic feedback.

What was the biggest challenge when it came to building RightEye?

In the early days, the hardware wasn’t what we needed it to be – it was too expensive, too lab based – but then cloud-based processing came along and changed everything. What used to take me three months to analyze, can now be done in a quarter of a millisecond! I used 18 different metrics in my doctoral dissertation, RightEye uses over 1,000.

What’s the value of eye tracking?

Vision is the only sense that has a direct connection to the brain. Every part of the brain has some function in processing what we see – which is what makes eye tracking such an important tool. It allows us to map different areas of the brain, and target therapies to the exact location of the dysfunction. We can now identify specific problems, very early on, and improve or prolong a person’s quality of life.

What do you think the eye tracker of the future will be like?

Eye tracking is going the way of webcams – webcams started out as external devices and now they’re built into laptops. Eye tracking is already available as a free app for iPhones and other devices, and while it does not have the accuracy of a clinical eye tracker, it can do some screening. It is even available as a feature in some cars. If the driver’s eyes close for a period of time, the car will vibrate – what is known as haptic feedback – to wake them up. Eye tracking technology in the future will be built into every technological device you own from the phone and laptop to car, VR headset and even helmets.

One example of future use is to examine the effects of marijuana use on driving and to differentiate it from alcohol and other drug use. There is an opportunity to make eye tracking available to police to identify drug use, which is exciting.

You have been vocal about the social benefits of eye tracking. Can you give an example?

It can be as simple as identifying kids who have problems reading. Convergence problems directly affect early reading and learning. Many kids have never even had a basic vision assessment. In elementary school, they may be erroneously considered problem children because they (literally) can’t focus on a book. Being categorized as disruptive may put them on a certain track, and certainly stop them from getting services that may help. There are many children, particularly minority children, who go without the help they need because there have not been tests that readily identify such issues.

And the implications are huge. Misdiagnosis can increase medical and social welfare costs and stop a person from reaching their full potential. I’m an immigrant myself, and the thought of someone not being able to achieve their full potential because of something so minor to fix, but with a huge lifelong impact, is nothing short of tragic.

How can eye tracking help?

Eye tracking is an unbiased way to alleviate the physician assessment and treatment burden. As a technology, it’s much more specific than clinical observation. Tests tend to be based on white females and males, but aspects of vision are affected by ethnicity and gender. And that’s why it’s so important to have normative values specific to certain population groups – to avoid misdiagnosis. 

Is that your goal?

If I want to leave a legacy – and I do – it’s going to be for the social side of RightEye. Even if just one person benefits from the product, that’s enough. I know what value our information has. As of this moment, 266,402 people have used our software. That’s over 1.8 terabytes – 10 times 10 million data points – which blows my mind. We can mine that data to find trends and insights into human health and wellness that we could never have imagined without it.

What, if anything, would you change about your journey?

I recently gave a talk at my old high school about how you can’t have success without having failure. As much as I wish I could have done it faster, we couldn’t have gotten to where we are today without those setbacks. Just don’t make the same mistakes twice!

What does the future hold for RightEye?

Further automation, machine learning, artificial intelligence, big data analytics, object recognition and pattern identification are all being implemented now with more in the future. This enables us to learn at a greater rate and to really understand the data we have. We have the largest repository of eye tracking data in the world, and we want to set up infrastructure in a way that’s scalable, secure, informative and ethical.

Eye tracking is relatively new so it’s a bit like the Wild West out there right now.  Our goal is to set the standard for scientific and ethical use of the tool for the betterment of humankind. Facial recognition was unregulated for a long time and, as a result, ended up having multiple vendors, all with different levels of accuracy and accountability. Famously, facial recognition once misidentified an African American man as a criminal. The problem was that the technology was based on a white male model that didn’t differentiate from other minorities. It gave a false positive and changed that person’s life.

Before RightEye even puts out metrics or outcomes, we go through a rigorous scientific process to make sure they’re reliable, valid and accurate. We’re adhering to the highest scientific and ethical standards. It is that important to us. We want positive use of the RightEye tool for generations to come.