Big-screen biometric apps, stand aside: Off-the-shelf smartphones enable rapid suspect ID for warfighters, intelligence ops

Story

December 05, 2012

Editor's note: It sounds like something straight out of "Covert Affairs" or the "The Bourne Identity," but this time it's got a twist. Face biometric and facial recognition software hits up databases for their best shot at a face match, but it's not on the big screen nor at the local CIA or other intelligence agency office. Instead, biometric face recognition software is right in the operator's hands - via an off-the-shelf iPhone or Android-based smartphones - giving warfighters and intelligence operatives in the field the chance to autonomously identify suspects or persons of interest within seconds or minutes. Editor Sharon Hess recently caught up with Animetrics President and CEO Paul Schuepp to find out more. Edited excerpts follow.

 

MIL EMBEDDED: Can you tell me a bit about Animetrics, how long in business, number of employees, and your technology focus?

SCHUEPP: Animetrics is a small firm in Conway, New Hampshire, and we also have an office in Columbia, Maryland, kind of closer to the customer down there. We employ about 15 people, most of which are engineers and scientists. We specialize in face recognition and face biometrics software. We have been in business for nine years, and most of our work [is] with the Department of Defense and the intelligence agencies. And we’re doing more now with public safety and law enforcement.

MIL EMBEDDED: Your press release mentioned integrating your face biometrics and facial recognition software into smartphones for military use. Has that already started?

SCHUEPP: Yes, it has been happening for a couple of years now, but deployments are slow going because there is so much testing that goes on, and the agencies are trying to get their hands around [how] to use it best. But we have apps that run on the iPhone and on Android platforms.

There are really two main apps: MobileID, which is a one-to-many search lookup of a face when you take a picture from the device itself. And then the other one is CredentialME, which is a one-to-one verification application where you could use your face for authorization to continue in another app, kind of like a password.

MIL EMBEDDED: How can your face recognition software be used by the military and intelligence communities?

SCHUEPP: Realistically, the intelligence agencies and military are most interested in using the face biometric software in conjunction with other biometrics, to try to resolve identity [questions arising] from [those serving] in theater.

MIL EMBEDDED: Which other biometrics would they use them with?

SCHUEPP: Fingerprinting, of course, has been around for a long time and there are several mobile devices today that have that on them. The other one is iris [scanning]. Iris is a very strong biometric, strong in the sense that it is very accurate, but, of course, it is also very controlled. You have to get very close to the person to be able to use it and [need] special lighting. But those are the biometrics of choice. But if you could use all three, including face biometrics, you’re in really good shape relative to seeing if that person matches the database or authenticating that person on a watch list.

MIL EMBEDDED: What about retina scanning?

SCHUEPP: They really don’t do retina scanning. Retinal was one of the original technologies that used an infrared. They didn’t like infrared because it could be potentially harmful to your eye. But the iris scanning is very safe, and it is basically looking in your eye with a light just like an eye doctor does, and it recognizes the pattern of the iris.

MIL EMBEDDED: What’s the advantage of the face biometric technology then, particularly when it’s on a smartphone?

SCHUEPP: The face biometric is a very good tool for trying to find out who the person is that you are confronted with. And you can be a little bit away from them by using the camera, and not have to be awfully close. Face biometric software on a smartphone is not intrusive: Nothing touches the person like [doing] a finger[print], and that is one of the desirable attributes. You can be 6 to 10 feet away, take a picture, and look up that person to see if they are in the database that you are checking.

MIL EMBEDDED: So anyone using your software on a smartphone would just take a photo like they would with any ordinary cell phone?

SCHUEPP: Yeah, exactly. So now here’s the military’s appeal: [The military has] basically a portable networking system. They are able to use the same kind of phone that you and I use, like the iPhone or an Android[-based mobile device] to communicate with. But it is on a secure DoD special cellular system, and they use that now to access their central databases for a lot of information. So one of those is getting to a biometric database. As you can imagine, there are a lot of people, perhaps millions, on a watch list that the military and also the intelligence community are on the lookout for, once you think about what is going on in Afghanistan and other parts of the world.

MIL EMBEDDED: How does the face biometric software on the smartphone match up the faces after you take the photo?

SCHUEPP: There is a definite process. On the face recognition side, a photo is taken. Basically, you have to have a digital image, and that digital image is internally scanned by the computer to find a face. That is the first step, and that’s a whole algorithm in itself, a technology in itself, called “face detection.” Once [the software] finds the face, then it finds more attributes about the face and basically reduces that face, that photo, to a vector of numbers. The numbers are determined by the algorithms, the technology that represents things like texture of the face. We also recognize the 3D nature of the face so from the 2-dimensional image, the facial pose is determined. [Face biometric software] actually can perceive through a process called “computational anatomy” to create the 3D model. And our software also recognizes the varied lighting on the face.

So those are the three major components: the geometry; the texture, which is all the colors in your skin; and then the lighting. Those variables will all reduce to a set of numbers, and they call that a “face biometric template.” And that template is used to compare to other templates that are previously stored in a large database, and then the statistical process [starts]. That takes that probe, that picture you just took, that template that is created, and determines how close it is to any of the other templates that are in that database. And if your statistical score is high enough, then it is deemed to be either a match or something that could be a match.

MIL EMBEDDED: OK. So once someone takes a photo of a suspect with a biometric software-enabled smartphone, how long do the results take?

SCHUEPP: Well, that is the exciting part. The systems that are being built now are set up to be very fast – we’re talking minutes to get back the result, maybe even seconds but usually minutes. Before this, the warfighter would have to send in a picture and he would have to wait a couple of days to get back the results from headquarters. Back in the States, the BIMA (Biometric Identity Management Agency, part of the Army Provost Marshal General) agency’s main location is in Clarksburg, West Virginia, and it had a long turnaround time. So the purpose of these systems being built now is to really improve that turnaround time to be very fast.

MIL EMBEDDED: What kind of transmission were they using with the old way that took two days?

SCHUEPP: It was a digital photo, but it was just protocol and sometimes I’m not even sure I understand why it took so long. But the systems were disconnected, and that is all being streamlined now. Of course, there was always the expedited way of doing things that the military can handle quite well. But as the normal everyday business, the way they had it set up wasn’t practical but now that’s changing.

MIL EMBEDDED: When using your system with or without the iris scanning and the fingerprinting, the process is completely automated?

SCHUEPP: Yes. We call it “lights-out face recognition.” No people are involved except for the operator taking the picture.

MIL EMBEDDED: What’s the percentage of identification accuracy when using face biometric software like yours on a smartphone or mobile device?

SCHUEPP: We test that all the time and in the face recognition business, there is a degree of variability, because the face is very uncontrolled compared to a finger or an iris. If you have a perfect facial image with good lighting on it and the person is looking at you frontal, you can [achieve] statistics of 99.9 percent identification rates.

But when the facial image varies from that, in other words you get poor lighting or lower resolution or especially an angulated face looking away from you, now the identification rate drops dramatically. But I am happy to say it is much better than it used to be. At Animetrics, our face biometric software is able, as long as two eyes can be seen, to identify angled faces at about a 92 percent identification rate.

MIL EMBEDDED: Is there a specific smartphone camera requirement though when using biometric face software?

SCHUEPP: There are some minimum requirements. Technically, when you take a picture of someone, [the software likes] to see 65 to 100 pixels between the eye centers. So what that means is if you have a 2 megapixel camera, you have to be within about 6 feet. I am not even sure you could buy a 2 megapixel camera anymore. But the iPhone 5 now [has an 8 megapixel camera]. With that, you could take a picture and be 25 feet away from a face, no problem, and get good resolution to get an ID.

There is a trade-off though: The higher resolution you use, the more data has to be transmitted, and sometimes that will make your response a little slower. The 2 megapixel camera would be much faster – it would be like seconds because of the less data in transit, depending on your network speed.

MIL EMBEDDED: For an 8 [megapixel] photo, how long would transmission take?

SCHUEPP: If you have a 4G network like AT&T or Verizon, it would only take 1.5 seconds, but if you’re in a 3G network or 2G network, it is going to take several seconds.

MIL EMBEDDED: Let’s move on to how the face biometrics-enabled smartphone accesses the database after a photo is taken. Is it a database that the military or intelligence community has and maintains, or is it just a database on the phone itself?

SCHUEPP: A little bit of both. There is no secret to the fact that the DoD has the large database, and they keep track of basically people who are outside of the U.S.; there are no U.S. civilians in the database – that is a separate system. That is the kind of system you have for the FBI, for people who are wanted, etc. But the military has a large database. BIMA, which I previously mentioned, maintains these very large databases with people of interest to the military.

The other process that happens is database access by a [system or] systems distributed by different military control centers in the field offices. We call them a forward operating base, and those forward operating bases have access to those databases. That’s what can be accessed by mobile devices such as the Android smartphones.

MIL EMBEDDED: Is there any range of distance for database access, for example, could someone in Afghanistan use this to access these databases you have referred to in the States?

SCHUEPP: They could access them if they are authorized to. Everything is set up on the secure, secret networks that the military uses. They are very, very secure beyond your imagination, and so if the Marines or if Special Forces are authorized to use them, then they can. But to me, [this kind of access] is almost like having a weapon: They really have to be authorized for use.

MIL EMBEDDED: Are you referring to SIPRNet and NIPRNet?

SCHUEPP: That is exactly the type of networks they would be used on.

MIL EMBEDDED: Your press release also said something about this being used in conjunction with a cloud system?

SCHUEPP: Yes. Everything we’ve built is based on a cloud architecture, which allows your capability to expand, to be able to have capacity of very large databases, and have capacity to support thousands of people sending in pictures/photos at once while expecting a good response time.

MIL EMBEDDED: So the military is putting some of its databases in the cloud?

SCHUEPP: The military is, definitely. The DoD is definitely going with a cloud architecture. They have been building that now, and all their information systems are starting to go cloud.

MIL EMBEDDED: What role does the cloud play in accessing the biometric information?

SCHUEPP: The cloud has two purposes: the database and the processing. The good thing about the database: It provides for high capacity and redundancy. To build a real cloud system, you have all kinds of backup and redundancy should something break. So it protects your data.

On the processing side, you can enable tens or hundreds of computer nodes to support your processing, so you can get very high throughput and with high capacity.

MIL EMBEDDED: Do all of your customers use the cloud?

SCHUEPP: We have customers who are not cloud based and they operate fine. It’s all about what the requirements are for what they are trying to do. For example, we have a system installed that runs a jail management system. It uses face recognition to keep track of the prisoners. So that was designed so that you’d be able to handle 100,000 prisoners. The department, in this case the sheriff’s department, that purchased the jail management system bought a certain-sized capacity computer system and their face recognition system works fine. Now if they are going to expand, that jail system expands, it doubles, they are going to have to supply more computer power and more storage. And they are going to have to create the specifications, the throughput and capacity needs. They are going to have to design that. If they had it on a cloud system, they could just turn it on like you and I turn the water on, like just give me more.

MIL EMBEDDED: Some of your face biometrics applications are described as a “localized” application on mobile devices. Are they localized to the phone?

SCHUEPP: Yes. We are calling that an “embedded system.” We are now able to do face matching and have a database actually on the device itself, not necessarily in the cloud, or it could be in both. So if you lose your communications or you don’t want to use your communications, we now have the technology that can do that locally right on the device itself.

MIL EMBEDDED: So how does the localized database update?

SCHUEPP: Your face database will get updated when you synchronize. It is kind of similar from a conceptual point of view to when you update your mail with Google or Microsoft Outlook. You have your email on your computer locally and then when you plug into the network, it automatically synchronizes and updates your main Gmail system somewhere out there. Same concept.

MIL EMBEDDED: What are the pros and cons of using a localized database?

SCHUEPP: The disadvantage is that you might not have access to the entire database. But one advantage is that localized is faster; the other advantage is, if you don’t have comms at all because you’re off the grid – as you can imagine soldiers often are – they can still have the same functionality for their mission.

MIL EMBEDDED: Are these COTS smartphones that are used with face biometric software? Can I use it on my iPhone that I am using right now?

SCHUEPP: Yep you could. But we don’t have it available as a consumer app to download. We sell it separately, directly to enterprises or directly to government agencies. The reason is that it accesses a face recognition system [that accesses] a database of faces, like the Department of Motor Vehicles or the FBI most wanted list or sex offender lists. So those are usually government-owned databases. So it’s not necessarily available to the consumer.

But the new wave of the future is for the military is to use more COTS, like Android, Motorola [smartphones], and so on. And in some cases, we’re seeing that law enforcement and the military are using Apple phones now and iPads.

MIL EMBEDDED: What about the Raptor ID smartphone mentioned in your PR?

SCHUEPP: We [have entered into] a partnership with Raptor ID, and they make a military ruggedized Android device [RaptorOne], which has three biometrics on it – the face, the fingerprinting, and the iris scanning – but it is also very rugged to the point where if a tank rolled over it, it wouldn’t break. It is purpose-built for the military’s use in the field.

MIL EMBEDDED: So that’s a proprietary smartphone?

SCHUEPP: Yes. But it’s only proprietary in a sense like anything is proprietary: It’s their device. But what makes it not proprietary is that it uses Android and that it runs on a standard mobile phone network like GSM cellular. And the military has its own protocols of cellular networks, and that will be available with that device from Raptor. [Editor’s note: For more info on Raptor ID’s RaptorOne ruggedized biometric smartphone, go to: www.raptor-id.com/products/raptorone.]

MIL EMBEDDED: Are any of the military services or intelligence agencies using your face biometric software on smartphones right now?

SCHUEPP: There are several projects involved with different military agencies doing just that. I’m not at liberty to disclose who they are and how they are using them, but the general sense, as I have kind of already explained, is that I don’t know that any are deployed today as a program of record. In other words, it’s in production. In most cases, they are being used as prototypes and are still under test at this stage.

I think as we go into 2013, you are going to see more announcements of deployments with the mobile device in the military. DARPA has done a lot with mobile. Now a soldier in the field or a battalion on their iPhones can download interesting and productive apps that are designed for the military from the DARPA app store.

MIL EMBEDDED: What will the military segment of the biometric smartphone market need next year, in three years, in five years?

SCHUEPP: We are looking forward to a lot of things as we go into 2013. The face biometric software will become recognized as a necessary instrument and tool for use in intelligence. It is going to be the way to keep people safe, safer than using guns actually. If you know who people are, then you know how to handle situations. If you don’t know who they are, you’re in trouble. So the buzzword in the military is about resolving identity. If you can resolve the identity, then security and order follow.

In the future five years from now, three years from now, what is needed is for the military to deploy this technology and use it. And that’s easily said but not easily done. Let’s face it, the Army, the Navy, all those guys – very large organizations, a lot of data, large systems – take a lot of coordination, a lot of program management to get systems in play and to complete testing and training.

Paul Schuepp is CEO and President of Animetrics. He joined Animetrics in 2003 to lead the firm’s transition from a pure technology company to a recognized provider of facial biometric solutions. He has 30+ years of experience in information technology and telecom, primarily in sales and business development, at companies such as Lucent Technologies, Ascend Communications (now Lucent Alcatel), Stratus Computer, and IBM. Paul is a frequent guest speaker at biometric events and serves on the board of trustees for the New Hampshire High Technology Council. Contact him at [email protected].

Animetrics 603-447-5600 www.animetrics.com