Facebook's Facial Recognition Software Is Different From The FBI's. Here's Why

When someone tags you in a photo on Facebook, it's often a nice reminder of a shared memory. It lets your whole social network see what you've been up to or where you've been.

Well, to three men from Illinois, this feature takes on a much more sinister capacity. They argue that when someone tags you in a photo on Facebook without your consent, Facebook is breaking the law — and a federal judge has allowed the case to proceed.

Facebook is hardly the only facial recognition technology that exists, but this company in particular is being challenged because its capabilities are so powerful. In Europe and Canada this month, privacy advocates won a victory when Facebook launched its photo app, Moments, without facial recognition scanning.

"There are more [facial recognition] algorithms and techniques than there are companies," says Jonathan Frankle, staff technologist at the Georgetown Center on Privacy and Technology. But with its huge database of images, Facebook's algorithm has a leg up on most others in that it is constantly being taught how to improve. Every time you tag a photo, you're adding to an enormous, user-driven wealth of knowledge and data.

Every time one of its 1.65 billion users uploads a photo to Facebook and tags someone, that person is helping the facial recognition algorithm. The tag shows the algorithm what someone looks like from different angles and in different lights, Frankle says. If you give Facebook a face to identify, it has fewer photos to parse through, because it's only looking at photos of you and your friends.

Facebook, according to the company, is able to accurately identify a person 98 percent of the time. Compare that with the FBI's facial recognition technology, Next Generation Identification, which according to the FBI, identifies the correct person in the list of the top 50 people only 85 percent of the time. Facial recognition in the Google Photos app is prone to error as well — the company came under fire last year when its system tagged two African-Americans as gorillas.

Part of why the FBI's technology has such a large margin of error is that its database usually only has a photo taken straight on — a mug shot, or in several states, a driver's license photo. The software has to look through a huge database to find a match, and each photo is often of a different person. Grainy security footage can be problematic.

"It's much harder for face recognition to work when you're trying to identify one person from a very large database versus one from a very small database, which is what Facebook is doing," says Jennifer Lynch, staff attorney at the Electronic Frontier Foundation.

Another reason Facebook is a target for privacy advocates is that its database — so carefully updated and tended — is tempting to the FBI. Law enforcement officials can issue a warrant for any information available on Facebook, including tagged photos.

"The FBI has said publicly that they do not put these photographs in the facial recognition database, but there is nothing in the law to prevent them from doing that," Lynch says.

Be careful about conspiracy theory rabbit holes, though. "People don't have a good intuition for what is and isn't possible," Frankle says. "And a lot of times until you've written code and tried to do this yourself it's hard to have a real visceral sense for just how hard some of this stuff is."

So next time you tag a friend on Facebook, go ahead — just remember that you're helping to train one of the most powerful facial recognition systems in the world.