Rupe Hansra, OD, Vice President of Professional Affairs at Topcon Healthcare, joins professional co-editor Scot Morris, OD, in discussing the “black box” in AI and why eye care must embrace data-driven innovation. They argue that explainable AI, larger datasets and collaboration can shift eye care from reactive screenings to preventative care that improves eye and overall health.
What Is The AI Black Box Problem ?
Dr. Morris frames the “black box” as a central challenge of artificial intelligence in medicine – clinicians lack transparency on how AI reaches a decision. He explains that modern AI learns patterns by training on massive data sets and producing complex numerical “weights and connections” rather than something readable, like English or code. He says the results are “incredibly accurate but very non-transparent.”
Dr. Morris warns that AI accuracy alone is insufficient. “If an AI diagnoses a patient with a disease, the doctor needs to know why so they can verify it. We can’t have a life or death decision made by a system that says, ‘Trust me, I just know,’” he says. AI systems can also learn biased information as “fact,” with no ability to recognize or admit the bias. Without transparent explanations, users have little access to whether or not there is bias in the final decision or answer. Dr. Morris says, “An AI that’s right 99% of the time, but can’t tell you why it is right, is a tool that we, as humans, are never fully going to trust.”
Using XAI as a Solution
In the face of this uncertainty, how should clinicians respond? Dr. Morris positions explainable AI, or XAI, as the solution to that distrust. Researchers are “turning the black box into a transparent glass box,” Dr. Morris says.
He offers a practical example: an XAI model could explain a loan decision, saying, “I reject this loan specifically because the debt to income ratio was 5% too high.”
Explanation helps verify recommendations instead of just spitting out an answer. Dr. Morris urges clinicians to ask not only “does it work?” but “do we understand how it works?” He says AI models are “clinical decision support tools for us. It doesn’t remove the doctor from the equation.” That shift frames trust to be equally as important as accuracy in clinical decision support.
Retinal Imaging As A Window To Systemic Disease
Dr. Hansra says Topcon Healthcare is actively building the collaborative AI infrastructure needed to power oculomics, which uses retinal imaging to detect neurodegenerative disease and other risks.
“What we’ve realized at Topcon Healthcare is that [oculomics] cannot be done in a silo,” he says. Topcon re-launched the Institute for Digital Health and now has “data sharing partners across the world that are building in this data set,” he continues. He urges optometrists to responsibly share retinal imaging data, saying this will help “democratize that data” for researchers and pharma to find and vet the right patients.
Dr. Hansra says generative AI has accelerated oculomics over the past five years and is beginning to reveal signals for certain diseases like Parkinsons, Alzheimers, kidney disease and preeclampsia. “What we’re finding is that there may be PTSD and some other things that we can detect through the eyes, whether it’s your pupil sizes, movements, all of that stuff,” he says.
Dr. Morris and Dr. Hansra agree that explainable AI, larger real‑world data sets and cross‑disciplinary collaboration will guide the optometric field toward larger access to care. They say AI should be a trusted clinical decision‑support tool rather than an opaque authority.
“The doctors who embrace AI are going to replace the ones who don’t,” Dr. Morris said, and both doctors urge practitioners to lead the change rather than be left behind.
For more on this conversation, listen to this episode of Real Talk.
Author
