Defeating Big Brother with Glasses: An Attack on Facial Recognition Neural Networks*-HzBnvRt8KVJkLGc

Original Source Here

As deep learning techniques continue to advance, image recognition systems are becoming more and more powerful. With this power comes great reward — helping diagnose disease from x-rays and self-driving cars are just two examples. But there is also potential for harm, particularly concerning facial recognition. In the future, it’s possible that surveillance cameras with state-of-the-art facial recognition technology could pop up on every street corner, effectively eliminating any privacy we still have. Fortunately, some researchers are already coming up with ways to counteract deep learning based facial recognition. I would like to highlight one interesting method — using an adversarial attack in the form of specially colored glasses to confuse facial recognition algorithms.

The Approach

The idea behind this approach stems from a known weakness of neural networks, which is that they can be weak to small perturbations to their inputs. What this means is that I can take an input (in our case, a picture of someone’s face), modify it just a little bit, and then the neural network will no longer recognize the face. In order to successfully use this kind of attack, the researchers first had to decide on a modification that was both practical and inconspicuous. Why inconspicuous? One conspicuous way to avoid facial recognition is to wear a mask that covers the entire face. Unfortunately, most people on the street don’t do that, so wearing a full mask would make you stick out like a sore thumb, completely missing the point of not drawing attention to yourself. This is why the researchers decided on glasses — they are very common among the public, not to mention being easy to wear and cheap to manufacture.

After deciding on glasses as the adversarial attack mechanism against neural networks, the researchers arrived at meat of the problem: figuring out what color to make each pixel of the glasses. But before they could do that, first they had to create a neural network based facial recognition system to test their attack on. They did this by using transfer learning — taking a cutting edge, well known open source facial recognition network (call this Network A), and using that as a base to train two of their own networks (call these networks B and C). Networks B and C were trained to recognize a selection of celebrities, as well as the researchers themselves. Once networks B and C were trained, the researchers moved on to developing their glasses algorithm.

Let’s restate the problem at this point. We have a glasses frame that we want to color in such a way that when worn, networks B and C misrecognize faces. On the surface, this seems kind of impossible — each pixel in the glasses frame can have (using the RGB color scheme) 255³ possible colors. With 224×224 face images and the glasses covering 6.5% of the face, there are 3261 pixels to optimize over. In other words, there are way too many combinations ((255³)³²⁶¹) to try to brute force. Fortunately, gradient descent works well for this problem. The algorithm the researchers came up with is simple. First, initialize the pixels in the glasses to all be the same color. Then, calculate the loss — the normalized sum of the probabilities that the faces in the training set are classified correctly by networks B and C (remember, a correct classification is what we don’t want with our attack). Next, take the gradient of that loss with respect to each pixel in the glasses and do a gradient descent update for each pixel. Repeat these steps until the error falls below some specified number.

Reese Witherspoon + glasses is misclassified as Russel Crowe. Photo from original paper.


This basic algorithm works, but the researchers made a few modifications to make the glasses produced more robust and practical. First, they realized that glasses move around slightly in real life (for example when walking), as opposed to staying in the exact same location on the face all the time. To account for this, the researchers modified the cost function to account for a set of perturbations for each face input image, as opposed to just the base image by itself.

Second, they realized that the optimal glasses produced by their algorithm could look very strange. For example, one pixel could be light yellow, the one to the right of that pixel could be dark brown, the one to the right of that could be neon green, etc. Strange colors like these might draw unwanted attention to the wearer. The researchers mitigated this by introducing a smoothness criterion to the cost function, essentially rewarding glasses that had colors that changed smoothly over many pixels, instead of sudden color changes.

Finally, the researchers considered the printability of the glasses. Typical printers can only print a subset of the full RGB color space, so it makes sense to reward glasses that are made of printable colors. To do this, the researchers included a “printability score” in the cost function that penalized unprintable pixels. After including these three modifications (robustness to slight movement, smoothness, and printability) to the original cost function, the researchers arrived at their final, practical algorithm.


The researchers tested both the basic algorithm and the modified, practical version. For the basic algorithm, they chose at random from networks B and C twenty people the networks had been trained to recognize, computed the optimal glasses for each person, and ran the networks again to see if they were still recognized. They did this for three different images of each of the twenty people in the test set. Amazingly, they achieved a 100% misclassification rate, meaning every single one of the glasses was effective in fooling the neural networks. For the practical algorithm, the researchers only tested it on pictures of themselves. Here, the results were not perfect but still good, with an overall misclassification rate above 95%.

Before we get too excited, however, we should discuss a few major problems. First, because of the extremely limited testing data, it’s hard to say whether or not this will actually work in practice. Also, these results assume that we have access to the inner workings of the neural networks that we are trying to defeat, which may be unrealistic in the real world depending on the situation. Finally, even though the researchers took steps to make the algorithmically generated glasses less strange-looking, they still don’t really look like something a normal person would wear. Because of these issues, there is definitely work to be done before this approach makes it to production.

However, we should applaud the researchers for coming up with a creative approach to combat facial recognition. This research is a proof-of-concept that cheap adversarial attacks actually work against surveillance technology. In the coming years, I hope that more research like this will allow us to maintain our privacy in an ever-photographed world.


Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

%d bloggers like this: