Seventh Annual International Symposium on Digital Ethics →

Surveillance on Aisle 3

  • AuthorHolly Richmond
  • Published Monday, November 11th, 2013
  • Comments1

You’re grocery shopping as usual when one of the shelves starts talking to you. “Would you like to try our new fudge-center Chips Ahoy cookies?” it says. “They’re very popular with women in their late 20s like yourself.

Ugh, what? This creepy encounter sounds like something out of “Minority Report,” but it could be coming to your favorite supermarket in 2015. The Washington Post reported in mid-October that new, sensor-laden shelves will analyze shoppers’ facial features in grocery stores in order to sell them more junk food.

The Washington Post explains:

“The shelf, which is hooked up to Microsoft’s Kinect controller, will be able to use basic facial features like bone structure to build a profile of a potential snacker. … 

“The company expects the shelf to help funnel more of the right products to the right consumers, and even convince undecideds to commit to an impulse buy by offering well-timed in-store commercials or coupons when the embedded weight sensor learns they’ve picked up an item. The move is almost certain to make it more difficult to resist junk foods.”

What the heck? Supposedly these smart shelves won’t store actual photos of your face, but they’re still troubling on a gut level and an ethical level … not to mention that the information is vulnerable to hackers.

Supermarkets have a hacker problem

Shopping in person seems inherently more anonymous than shopping online. If you use cash, no loyalty card and the self-checkout, the only record of your visit could be the store’s security camera. It’s not like you’re handing over your home address, credit card number and other personal information, like you do when buying something from an e-commerce site. Shopping online makes you more vulnerable, right?

Maybe not. California shoppers got an unpleasant surprise when criminals put “sniffers” on credit/debit machines in 20-some Lucky supermarkets two years ago, recording at least 80 shoppers’ card numbers. Arizona chain Basha’s was hit early this year, with more than 400 customers reporting fraudulent use of their credit or debit card. Finally, the Schnucks family of stores in the Midwest was the victim of a hacker who used a “malicious computer code” to steal customers’ payment info.

So without carrying a hefty wad of cash with you everywhere, most purchases have some risk attached. But until now, at least we had the reassurance that the ubiquitous ad-targeting of the Internet couldn’t follow us into the store.

Privacy is disappearing and these shelves would only contribute

In this day and age, you have to go to extreme measures to avoid being tracked. A recent piece on Fast Company about living anonymously tackled this very issue. Titled “Think you can live offline without being tracked? Here’s what it takes,” the piece explores how people are grasping at privacy in a post-NSA-scandal world. Bottom line? To be truly anonymous, you basically can’t drive, need a fake credit card and must cut off friendships. Living in 2013 means conceding that corporations and the government are virtually following you, ostensibly for your good (at least some of the time).

But just because everything is tracked nowadays doesn’t mean it’s acceptable — or that these encroachments have to become increasingly invasive. These interactive shelves seem like a creepy new way to erode our quickly disappearing privacy. Are the shelves the end of shopping anonymously? In reality, shopping anonymously ended when stores installed security cameras. That said, consumers can stand up and protest when our personal data is used for less-than-innocuous reasons.

Mondelez’s history makes this more troubling

It would be one thing if pro-social causes were secretly using personal data — like how New York City reads your toll pass at random times to compile real-time traffic data. Situations such as that are different (and possibly acceptable) compared to using personal info to encourage unhealthy eating. Mondelez International is the company behind the shelves, and it owns a slew of well-known brands including Oreo, Nabisco, Teddy Grahams, Cheese Nips and Trident. So you won’t be pressured to buy more apples and zucchini, just preservative-laden, artificially flavored snacks.

And Mondelez doesn’t have the best track record to begin with. In May, consumers filed a class action lawsuit against the company for its claims that Fig Newtons are “made with real fruit,” when the cookies actually contain “mechanically processed fruit purée, which is not ‘real fruit,’” according to the plaintiff. (However, a judge dismissed the lawsuit this fall.)

Earlier this spring, the International Union of Food workers filed a formal complaint against Mondelez International for multiple human rights violations, such as union-busting and firing a worker after a machine chopped his thumb off. Incidents such as these seem to indicate that Mondelez cares more about profits than fairness to employees or transparency for customers.

Stores should use informed consent

Whether it’s Mondelez or any of the snack food companies that will inevitably follow suit and implement face-tracking shelves, it’s absolutely vital for them to use informed consent. As Treehugger writes, “Customers deserve to know when they’re the target of a marketing campaign.” Adds Robin Shreeves on Mother Nature Network:

“It’s important that we’re aware of insidious marketing schemes like this. It’s good to know when you’re being targeted; it helps you to make informed decisions. I also think that as consumers, we do have the power to influence the stores we shop in. If this is something that makes you uncomfortable, let stores know.”

Sure, informed consent could be a headache for snack giants using these shelves, and may even bite into their profits. But better — and far more ethical — to be up front with consumers rather than face their wrath later, right? It doesn’t have to be hard, either. A simple handwritten sign in a neighborhood boutique lets me know to smile, I’m on camera. A sign posted at supermarket entrances explaining the facial analysis software wouldn’t be enough to turn all hungry shoppers away, even if it did drive away more privacy-conscious ones.

 The Microsoft Kinect[ion]

 Another reason informed consent is so important in this case is that Mondelez’s shelves use Microsoft Kinect technology, adding another layer of ethical concern. I can’t help but wonder whether Mondelez International will be sharing the consumer data with Microsoft. At least Microsoft recently issued the following statement, after rumors that it made the Xbox One with advertising in mind:

“[We] will not target ads to you based on any data Kinect collects unless you choose to allow us to do so. Furthermore, we will give you a clear explanation of what is collected and how it will be used. Importantly, we do not collect your personal information to share or sell to third parties, and you are fully in control over what personal data is shared.”

Whether or not you believe Microsoft, at least that statement is a step in the right direction. Let’s hope Mondelez follows suit.

Holly Richmond lives in Portland, Oregon, and writes for and MoveOn. Find her at

One Response to “Surveillance on Aisle 3”

Leave a Reply