What information is available concerning DoD standards and/or methods for validating and implementing data derived from machine learning and artificial intelligence algorithms used in biometric identification applications?

Overview

A member of the Department of Commerce’s National Institute of Standards and Technology (NIST) requested information concerning U.S. Department of Defense (DoD) standards and/or methods for validating and implementing data derived from machine learning (ML) and artificial intelligence (AI) algorithms used in biometric identification applications.

Findings

According to subject matter experts at the Defense Forensics & Biometrics Agency’s (DFBA’s) Standards Branch, standards for AI and ML for biometrics have yet to be developed [1]. However, the International Organization for Standardization (ISO) has several biometric performance testing and reporting standards [2-8] that could aid in creating formalized standards related to the use of AI and other autonomous systems in biometrics identification practices. These include the following:

• Part 1: Principles and framework (ISO/IEC 19795-1:2006) [2]
• Part 2: Testing methodologies for technology and scenario evaluation (ISO/IEC 19795-2:2007) [3]
• Part 3: Modality-specific testing (ISO/IEC 19795-3:2007) [4]
• Part 4: Interoperability performance testing (ISO/IEC 19795-4:2008) [5]
• Part 5: Access control scenario and grading scheme (ISO/IEC 19795-5:2011) [6]
• Part 6: Testing methodologies for operational evaluation (ISO/IEC 19795-6:2012) [7]
• Part 7: Testing of on-card biometric comparison algorithms (ISO/IEC 19795-7:2011) [8]

Additionally, the U.S. Air Force Research Laboratory’s (AFRL’s) Human Performance Wing (711 HPW) is researching methods to improve the reliability of automated systems used in human detection, tracking, and characterization [9]. This research effort generated an ML algorithm used to streamline human recognition by full-motion video analysts in data collected from aerial assets [10]. This software package, based on developing and exploiting behavior understanding datasets, could be valuable in developing DoD-wide standards and best practices for AI/ML used in conjunction with biometrics identification.

Conclusion

Although the DoD has yet to develop and implement formalized standards for using ML and AI in biometrics identification operations, standards from the ISO, as well as ongoing research from AFRL 711 HPW, could assist in their creation.

References

1. McNeirney, V. “Standards for AI use in biometric identification.”  Private communication, DFBA, 3 October 2018.
2. ISO. “Information technology — Biometric performance testing and reporting — Part 1:  Principles and framework.” Retrieved from https://www.iso.org/standard/41447.html, 2016.
3. ISO. “Information technology — Biometric performance testing and reporting — Part 2:  Testing methodologies for technology and scenario evaluation.” Retrieved from https://www.iso.org/standard/41448.html, 2012.
4. ISO. “Information technology — Biometric performance testing and reporting — Part 3:  Modality-specific testing”. Retrieved from https://www.iso.org/standard/41449.html, 2015.
5. ISO. “Information technology — Biometric performance testing and reporting — Part 4:  Interoperability performance testing.” Retrieved from https://www.iso.org/standard/46329.html, 2014.
6. ISO. “Information technology — Biometric performance testing and reporting — Part 5:  Access control scenario and grading scheme.” Retrieved from https://www.iso.org/standard/51768.html, 2016.
7. ISO. “Information technology — Biometric performance testing and reporting — Part 6:  Testing methodologies for operational evaluation.” Retrieved from https://www.iso.org/standard/50873.html, 2012.
8. ISO. “Information technology — Biometric performance testing and reporting — Part 7:  Testing of on-card biometric comparison algorithms.” Retrieved from https://www.iso.org/standard/53059.html, 2016.
9. AFRL Human Performance Wing (HPW 711th). “HOAR Machine Intelligent Tools” (previously “Machine Intelligent Human Threat Models”). URED Project Abstract, EF005009, 10 August 2018 to 30 November 2016.
10. Cheng, Z., J. Camp, and T. Webb. “Human biosignatures for intelligence, surveillance, and reconnaissance (ISR).” AD1021880, Wright Patterson Air Force Base, OH, 30 November 2018.