https://www.eff.org/pages/face-recognition

Face Recognition

Face recognition is a method of identifying or verifying the identity of an individual using their face. Face recognition systems can be used to identify people in photos, video, or in real-time. Law enforcement may also use mobile devices to identify people during police stops.

But face recognition data can be prone to error, which can implicate people for crimes they haven’t committed. Facial recognition software is particularly bad at recognizing African Americans and other ethnic minorities, women, and young people, often misidentifying or failing to identify them, disparately impacting certain groups.

Additionally, face recognition has been used to target people engaging in protected speech. In the near future, face recognition technology will likely become more ubiquitous. It may be used to track individuals’ movements out in the world like automated license plate readers track vehicles by plate numbers. Real-time face recognition is already being used in other countries and even at sporting events in the United States.

Criminal Defense Attorney?

Check out our legal resources on face recognition

How Face Recognition Works

Source: Iowa Department of Transportation

Face recognition systems use computer algorithms to pick out specific, distinctive details about a person’s face. These details, such as distance between the eyes or shape of the chin, are then converted into a mathematical representation and compared to data on other faces collected in a face recognition database. The data about a particular face is often called a face template and is distinct from a photograph because it’s designed to only include certain details that can be used to distinguish one face from another.

Some face recognition systems, instead of positively identifying an unknown person, are designed to calculate a probability match score between the unknown person and specific face templates stored in the database. These systems will offer up several potential matches, ranked in order of likelihood of correct identification, instead of just returning a single result.

Face recognition systems vary in their ability to identify people under challenging conditions such as poor lighting, low quality image resolution, and suboptimal angle of view (such as in a photograph taken from above looking down on an unknown person).

When it comes to errors, there are two key concepts to understand:

A “false negative” is when the face recognition system fails to match a person’s face to an image that is, in fact, contained in a database. In other words, the system will erroneously return zero results in response to a query.

A “false positive” is when the face recognition system does match a person’s face to an image in a database, but that match is actually incorrect. This is when a police officer submits an image of “Joe,” but the system erroneously tells the officer that the photo is of “Jack.”

When researching a face recognition system, it is important to look closely at the “false positive” rate and the “false negative” rate, since there is almost always a trade-off. For example, if you are using face recognition to unlock your phone, it is better if the system fails to identify you a few times (false negative) than it is for the system to misidentify other people as you and lets those people unlock your phone (false positive). If the result of a misidentification is that an innocent person goes to jail (like a misidentification in a mugshot database), then the system should be designed to have as few false positives as possible.

How Law Enforcement Uses Face Recognition

Source: Arizona Department of Transportation

Law enforcement agencies are using face recognition more and more frequently in routine policing. Police collect mugshots from arrestees and compare them against local, state, and federal face recognition databases. Once an arrestee’s photo has been taken, the mugshot will live on in one or more databases to be scanned every time the police do another criminal search.

Law enforcement can then query these vast mugshot databases to identify people in photos taken from social media, CCTV, traffic cameras, or even photographs they’ve taken themselves in the field. Faces may also be compared in real-time against “hot lists” of people suspected of illegal activity.

Mobile face recognition allows officers to use smartphones, tablets or other portable devices to take a photo of a driver or pedestrian in the field and immediately compare that photo against one or more face recognition databases to attempt an identification. In San Diego, for example, a program called TACIDS (Tactical Identification System) allows law enforcement officers from nearly 25 agencies to stop people on the street, use their tablets or mobile phones to take photographs of them and run the images against the county’s mugshot database.

Face recognition has been used in airports, at border crossings, and during events such as the Olympic Games. Face recognition may also be used in private spaces like stores and sports stadiums, but different rules may apply to private sector face recognition.

Supporting these uses of face reconition are scores of databases at the local, state and federal level. Estimates indicate that 25% or more of all state and local law enforcement agencies in the U.S. can run face recognition searches on their own databases or those of another agency.

According to Governing magazine, as of 2015, at least 39 states used face recogntion software with their Department of Motor Vehicles (DMV) databases to detect fraud. The Washington Post reported in 2013 that 26 of these states allow law enforcement to search or request searches of driver license databases, however it is likely this number has increased over time.

Databases are also found at the local level, and these databases can be very large. For example, the Pinellas County Sheriff’s Office in Florida may have one of the largest local face analysis databases. According to research from Georgetown University, the database is searched about 8,000 times a month by more than 240 agencies.

The federal government has several face recognition systems, but the database most relevant for law enforcement is FBI’s Next Generation Identification database which contains more than 30-million face recognition records. FBI allows state and local agencies “lights out” access to this database, which means no human at the federal level checks up on the individual searches. In turn, states allow FBI access to their own criminal face recognition databases.

FBI also has a team of employees dedicated just to face recognition searches called Facial Analysis, Comparison and Evaluation (“FACE”) Services. The FBI can access over 400-million non-criminal photos from state DMVs and the State Department, and 16 U.S. states allow FACE access to driver’s license and ID photos.

Given the large number of DMV databases using face recognition and the number of Americans whose photos are in the State Department’s database of passport and U.S. visa holders, Georgetown University has estimated close to half of all American adults have been entered into at least one if not more face recognition databases.

Who Sells Face Recognition

MorphoTrust, a subsidiary of Idemia (formerly known as OT-Morpho or Safran), is one of the largest vendors of face recognition and other biometric identification technology in the United States. It has designed systems for state DMVs, federal and state law enforcement agencies, border control and airports (including TSA PreCheck), and the state department. Other common vendors include 3M, Cognitec, DataWorks Plus, Dynamic Imaging Systems, FaceFirst, and NEC Global.

Threats Posed By Face Recognition

Face recognition data is easy for law enforcement to collect and hard for members of the public to avoid. Faces are in public all of the time, but unlike passwords, people can’t easily change their faces. We are seeing increased information-sharing among agencies. Cameras are getting more powerful and technology is rapidly improving.

Face recognition data is often derived from mugshot images, which are taken upon arrest, before a judge ever has a chance to determine guilt or innocence. Mugshot photos are often never removed from the database, even if the arrestee has never had charges brought against them.

In spite of face recognition’s ubiquity and the improvement in technology, face recognition data is prone to error. In fact, the FBI admitted in its privacy impact assessment that its system “may not be sufficiently reliable to accurately locate other photos of the same identity, resulting in an increased percentage of misidentifications.” Although the FBI purports its system can find the true candidate in the top 50 profiles 85% of the time, that’s only the case when the true candidate exists in the gallery. If the candidate is not in the gallery, it is quite possible the system will still produce one or more potential matches, creating false positive results. These people—who aren’t the candidate—could then become suspects for crimes they didn’t commit. An inaccurate system like this shifts the traditional burden of proof away from the government and forces people to try to prove their innocence.

Face recognition gets worse as the number of people in the database increases. This is because so many people in the world look alike. As the likelihood of similar faces increases, matching accuracy decreases.

Face recognition software is especially bad at recognizing African Americans. A 2012 study [.pdf] co-authored by the FBI showed that accuracy rates for African Americans were lower than for other demographics. Face recognition software also misidentifies other ethnic minorities, young people, and women at higher rates. Criminal databases include a disproportionate number of African Americans, Latinos, and immigrants, due in part to racially biased police practices. Therefore the use of face recognition technology has a disparate impact on people of color.

Some argue that human backup identification (a person who verifies the computer’s identification) can counteract false positives. However, research shows that, if people lack specialized training, they make the wrong decisions about whether a candidate photo is a match about half the time. Unfortunately, few systems have specialized personnel review and narrow down potential matches.

Face recognition can be used to target people engaging in protected speech. For example, during protests surrounding the death of Freddie Gray, the Baltimore Police Department ran social media photos through face recognition to identify protesters and arrest them. Of the 52 agencies analyzed in a report by Georgetown Center on Privacy and Technology , only one agency, the Ohio Bureau of Criminal Investigation, has a face recognition policy expressly prohibiting the use of the technology to track individuals engaged in protected free speech.

Few face recognition systems are audited for misuse. Of 52 agencies surveyed by Georgetown that acknowledged using face recognition, less than 10% had a publicly available use policy. Only two agencies (the San Francisco Police Department and the Seattle region’s South Sound 911) restrict the purchase of technology to those that meet certain accuracy thresholds. Only one—Michigan State Police—provides documentation of its audit process.

There are few measures in place to protect everyday Americans from the misuse of face recognition technology. In general, agencies do not require warrants, and many do not even require law enforcement to suspect someone of committing a crime before using face recognition to identify them.

The Illinois Biometric Information Privacy Act requires notice and consent before the private use of face recognition tech. However, this only applies to companies and not to law enforcement agencies.

EFF’s Work on Face Recognition

We support meaningful restrictions on face recognition use both by government and private companies. We testified about face recognition technology before the Senate Subcommittee on Privacy, Technology, and the Law, as well as the House Committee on Oversight and Government Reform Hearing on Law Enforcement’s Use of Facial Recognition Technology. We also participated in the NTIA face recognition multistakeholder process but walked out, along with other NGOs, when companies couldn’t commit to meaningful restrictions on face recognition use.

We have consistently filed public records requests to obtain previously secret information on face recognition systems. We even sued the FBI for access to its face recognition records.

In 2015, EFF and MuckRock launched a crowdsourcing campaign to request information on various mobile biometric technologies acquired by law enforcement around the country. We filed an amicus brief, along with the ACLU of Minnesota, demanding the release of emails regarding the Hennepin County Sheriff Office’s face recognition program that were requested by one local participant in the project.