In the Fall of 2020 the U.S. Department of Homeland Security (DHS) Science and Technology Directorate (S&T) hosted another Biometric Technology Rally to assess the capabilities of biometric algorithms for automated passenger identification in real-time screening systems. Due to the presence of COVID-19, the evaluation was expanded to include the identification of passengers wearing masks.
Similar to the 2019 rally Rank One Computing (ROC) emerged as a stand-out performer in the 2020 rally. The DHS-assigned alias for the ROC algorithm was “Owens” on unmasked faces and on masked faces it was “Pond”.
Rank One’s accuracy stood out when including errors from the acquisition systems:
- On masked faces ROC was the only algorithm to achieve above 90% accuracy on two different acquisition systems.
- On masked faces ROC had the second highest accuracy of any algorithm with 91.9%
When excluding acquisition errors, even more impressive accuracy metrics were achieved:
- On unmasked faces ROC achieved at least 99% accuracy with 3 of the 6 different acquisition systems
- On masked faces ROC achieved above 94% accuracy with 4 of the 6 acquisition systems. Rank One was one of only two algorithms to achieve such accuracy.
As with previous years, the Rank One algorithm demonstrated that it is perhaps the only U.S. developed technology that can be trusted for accurate and effective real-time screening applications.
In addition to the strong performance by Rank One, a few other vendors have recently proclaimed strong results in the DHS competition, including NEC, Innovatrics, and Corsight. However, despite strong accuracy performance from Rank One and certain other vendors, the difference in hardware efficiency between these solutions is a notable consideration.
Efficiency Matters
In terms of efficiency, while the 2019 DHS Rally had fixed limits on the efficiency of algorithm submissions (algorithms needed to perform template generation and matching in under 250ms), this year’s rally did not impose such restrictions. Thus, the matching algorithms submitted to this year’s rally had significant variations in hardware efficiency.
Hardware efficiency, and in particular template generation speed, is an important consideration when deploying real-time screening systems. In fact, some face recognition solutions will require spending more on additional hardware than the licensing fees for software itself. This is why efficiency metrics are critical to consider when planning and procuring an enterprise grade real-time screening capabilities.
Of the vendors who have publicly stated their participation in the 2020 DHS Biometric Rally, surprisingly, only Innovatrics participates in the industry-standard NIST FRVT Ongoing report. The following is the efficiency comparison between ROC and Innovatrics:
Based on the FRVT benchmark, the Rank One algorithm can generate templates a staggering 7.5x faster than the Innovatrics algorithms, in addition to being an order of magnitude more efficient in memory usage. In real-time screening applications, this difference in template generation speeds means ROC would cost 7.5x less CPU in hardware expenses to achieve the same application workflow.
While NEC has surprisingly never participated in FRVT Ongoing, despite being deployed in several U.S. national security systems, they have submitted to FRVT 1:N. The following is the efficiency comparison between ROC and NEC:
Similarly, the NEC algorithm requires more than 3x the CPU throughput as the ROC algorithm to generate templates for faces encountered in an image or video frame. The total throughput difference to both generate a template and search it against a large-sized gallery is more than 2x different. The memory usage of the Rank One algorithm is an order of magnitude less than NEC.
The company Corsight had never submitted their algorithm to NIST FRVT benchmarking at the time this article was published.
Comparison to Iris Recognition Algorithms
In addition to the 10 face recognition matching algorithms submitted to the rally, three different iris matching algorithms were submitted. While iris recognition pairs very well with face recognition algorithms, as a stand-alone biometric all three solutions performed significantly worse than the Rank One face recognition algorithms in the rally.
When including acquisition errors, the highest accuracy achieved on unmasked faces by any of the three iris algorithms was 80.4%. By comparison, the ROC algorithm achieved an accuracy of 98.1% in these same conditions.
When excluding acquisition errors the highest accuracy achieved on unmasked faces by any of the three iris algorithms was 97.9%. By comparison, the ROC algorithm achieved an accuracy of 99.4% in these same conditions.
When masks were present, the iris recognition algorithms were impacted further, with the best performing solution only achieving an accuracy of 67.3% including acquisition errors. By comparison, the ROC algorithm achieved an accuracy of 91.9% in these same conditions.
Summary
Rank One Computing once again emerged as one of the most accurate matching algorithms in the DHS Biometric Rally. Impressively high accuracies were achieved on both masked and unmasked faces. In addition to the accuracies achieved by the ROC algorithm, it continues to be a stalwart of efficiency in the NIST FRVT benchmarks. The combination of top-tier accuracy and hardware efficiency both put Rank One in a category of its own and save ROC’s partners and customers substantial amounts of money in hardware costs.