MindfuL™ technology to provide transparency and build user trust in machine learning systems

BAE Systems recently delivered software to the Defense Advanced Research Projects Agency (DARPA) as part of a contract under the agency’s Competency-Aware Machine Learning (CAML) program. The delivery of the MindfuL™ software is the first milestone in the program to improve the transparency of machine learning systems.

Transitioning artificial intelligence-based systems from decision-making tools into true partners requires users to trust in their machine counterpart. While machine learning technology has matured, these systems are unable to communicate context and confidence in their decisions – including task strategies, the completeness of their training relative to a given task, factors that may influence their actions, or the likelihood to succeed under specific conditions. To meet these challenges, BAE Systems provided its MindfuL solution, a system which will independently “audit” a machine learning-based system and provide the end user with insights to build trust in the technology. The first software release of the system provides a baseline capability to detect when the system encounters a new environment for which it has not been trained.

“The technology that underpins machine learning and artificial intelligence applications is rapidly advancing, and now it’s time to ensure these systems can be integrated, utilized, and ultimately trusted in the field,” said Chris Eisenbies, product line director of the Autonomy, Control, and Estimation group at BAE Systems. “The MindfuL system stores relevant data in order to compare the current environment to past experiences and deliver findings that are easy to understand.”

The program will produce statements like: “The machine learning system has navigated obstacles in sunny, dry environments 1000 times and completed the task with greater than 99 percent accuracy under similar conditions.” Or alternatively, “The machine learning system has only navigated obstacles in rain 100 times with 80 percent accuracy in similar conditions; manual override recommended.”

The MindfuL software was designed and built as part of a collaboration between BAE Systems and Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory. Under the terms of the approximately $5 million three-year CAML program awarded in 2019, BAE Systems’ FAST Labs™ research and development organization will demonstrate the technology in both simulation and prototype hardware later this year. The program will leverage BAE Systems’ autonomy portfolio and build on its 20-year history of pioneering autonomy technology. Work will be performed at BAE Systems facilities in Burlington, Massachusetts and Arlington, Virginia.