Grocers and other retailers are increasingly offering self-checkout and even no checkout. Tim Kridel explores how that trend — including its drawbacks — creates opportunities for pro AV.
If you walk into a store, fill your cart and then just walk out, it’s shoplifting — unless it’s one of a growing number of supermarkets, convenience stores and other retail outlets offering what’s broadly called “frictionless shopping.”
One example is Amazon Go grocery stores [pictured above], which launched in the US in 2018. Surveillance cameras and other technologies identify each item as shoppers remove them from the shelf and place them in their bag. When they’re finished, shoppers simply walk out instead of using a self-checkout or staffed register. The system then automatically charges their Amazon account and emails a receipt.
Earlier this year, the chain expanded to Europe with a London store under the Amazon Fresh brand. The company’s director told the BBC that additional London stores are planned.
Interest in frictionless shopping isn’t limited to outsiders and upstarts such as Amazon. Established EMEA retailers are increasingly exploring and implementing the technology. One example is Tesco, which plans to open its first frictionless supermarket later this year.
Frictionless shopping vendor AiFi has done work for the Albert Heijn chain in the Netherlands, as well as Zabka, the largest convenience store chain in Poland.
Aifi system in action at Choice Market
“They’re expanding aggressively, opening 1,000 stores a year,” says Steve Gu, co-founder and CEO. “We signed a very large contract with them. We did a very successful pilot with Carrefour two years ago. Now we’re in a contract with them.”
Department stores next?
In August, The Wall Street Journal reported that Amazon is considering expanding its bricks-and-mortar retail presence to department stores. Time will tell whether that happens, but it could set the stage for using frictionless shopping technology in other retail sectors.
For now, frictionless shopping is largely a supermarket phenomenon. The grocery business is known for having margins as thin as potato chips. So a big part of frictionless shopping’s business case is minimising overhead costs, including those related to employees.
“When you're talking with retailers, the primary business driver for investing in something like frictionless payment is the headcount,” says Steven Keith Platt, Retail Analytics Council research director.
Another, more recent business driver is Covid-19. No checkout queue means less time around other shoppers. Retailers could use frictionless shopping to attract consumers who are concerned that they’ll get infected at the store. Also, between the fear of getting sick and increased unemployment benefits, many former and prospective employees are staying home.
“It's pretty tough right now to find people to work,” Platt says. “So today, it's a realistic answer when somebody says: ‘It's not only saving labour. It's we can't get labour.’”
Just walk out
In 2019, Tesco invested in Trigo, one of several vendors specialising in the technologies used to enable frictionless shopping. Another vendor is Amazon, which used its Go experience to develop Just Walk Out, a turnkey platform that it’s licensing to other retailers.
But Just Walk Out could prove to be a tough sell, for a couple of reasons. An obvious hurdle is that Go and Fresh compete with many potential customers.
“We see that in our cloud video solutions, where most retailers will not adopt any type of cloud solution that's based on an Amazon Web Services platform simply because they don't want to be supporting their competition,” says Net Payne, March Networks chief sales and marketing officer. “So I think there's going to be a little bit of hesitancy to provide Amazon with an entry point into your location and let them capture the data that is essentially your business lifeblood.”
That concern might explain why the Just Walk Out FAQ — which Amazon provided in lieu of an interview — includes a question about the types of shopper data it collects. The answer: “We only collect the data needed to provide shoppers with an accurate receipt. Shoppers can think of this as similar to typical security camera footage.”
The FAQ also doesn’t have any pricing information, either, but many retail trade publications cite analyst estimates of US$1 million (€849,000) per store.
“It's going to be interesting to see how many choose that, not only because it's coming from Amazon, but also from I'm hearing about the cost of implementing it,” Platt says.
Off the shelf
Amazon and many of its vendor competitors are mum about the nuts and bolts of the technologies they use to enable frictionless shopping. Just Walk Out’s FAQ doesn’t get more specific than “computer vision, sensor fusion and deep learning.” Trigo also declined an interview.
One exception is AiFi, which focused on affordability when developing its Orchestrated Autonomous Store Infrastructure & Services (OASIS) platform.
“Our clients say we have to make the system very affordable to deploy,” Gu says. “We reduced the cost by 10X compared to Amazon and other competitors’ solutions.”
AiFi does that partly by using off-the-shelf cameras rather than bespoke models to feed OASIS, which then uses machine vision and other artificial intelligence (AI) to understand what it sees. The ability to use conventional cameras is key because a high density is necessary to identify every shopper and every product on every shelf in every aisle.
“We really want to see everything inside the shop so we can know precisely what that person is grabbing and how many items,” Gu says. “Most of the cameras are traditional IP cameras that you can buy for US$50 (€43) apiece. As long as they comply with certain protocols, such as RTSP, and 1080p, we’re fine.”
OASIS also can use some or all of a store’s existing surveillance cameras if they’re in the right positions. But those likely would need to be augmented to provide seamless coverage — a fact of life for any frictionless shopping platform that relies on cameras rather than other types of sensors, such as RFID tags and sensors.
Credit: Zyn Chakrapong/Shutterstock.com
“New applications for security cameras can drive requirements for the number of cameras, locations of cameras and camera fields of view,” says Dan Reese, Bosch vertical marketing manager. “For example, while today’s facial recognition algorithms can work with off-angle images, the best performance is still achieved with direct camera-facing shots.
“And for just-walk-out technologies, complete or near-complete coverage of the store floor helps to ensure seamless tracking through their shopping journey. For checkout scanner monitoring, a high-resolution shot of the scanning area is required, which can drive mounting requirements for these cameras.”
All of these installation and use case scenarios highlight some of the business opportunities for AV integrators in the emerging frictionless shopping market. Many major chains have contractors — including AV firms — that they regularly use for remodels and new construction. AiFi can work with those.
“Otherwise, we’ve been working with some systems integrators,” Gu says. “We partner with them to install the cameras. We have partners in the US and Europe. We’re always looking for great partners to work with.”
Package deal
The technologies deployed for frictionless shopping can be used for additional applications. For example, these platforms always know exactly how many people are in the store. That’s particularly valuable for complying with pandemic restrictions — especially if automation means few employees are around to enforce them.
“If there are too many people in the shop, we can close the gate, and no one can enter,” Gu says.
March Networks also is seeing interest in this use case.
“A large Italian supermarket chain that was one of the early adopters of our health compliance [solution],” Payne says. “They are using that for occupancy. It's counting people [and] giving them a red, yellow or green as to whether you can let more people in.”
A view of every product on every shelf means frictionless shopping platform can be used for inventory management, too. Retailers also could use that data to refine their stock-out prediction models.
Other use cases could benefit AV firms that offer digital signage products. Retailers could use platforms such as OASIS to serve personalised ads based on purchase histories, shopper-provided preferences and more. Those could be triggered when the shopper approaches the area where a particular product is shelved.
Retailers also could use platforms such as OASIS to understand whether and how people respond to ads on digital signage. Granted, that use case has been around long before frictionless shopping. But the high camera density and AI-powered capabilities could ferret out new insights, such as whether a shopper stopped because the ad was so compelling or because his phone just happened to ping at that moment.
“You know that not in a generic sense but a pixel-perfect sense,” Gu says. “You know precisely what that person is doing.”
Problems mean more opportunities
As automation grows, so does the risk of theft and fraud. For example, fewer employees — or even none, in the case of what’s called “ghost” stores — could make shoplifting more tempting. For some retailers, those risks could be a barrier to adoption, which is why vendors also have identified ways to thwart leakage.
“If someone takes a drink, and drinks it inside the shop, we can detect that,” Gu says.
Lower levels of automation also come with risks. For example, retailers such as Walmart are increasingly replacing staffed checkouts with self-service stations.
“I know of examples where people are putting barcodes on their rings,” says March Networks’ Payne. “Then they scan something over the self-checkout, but it reads the barcode on their ring instead of the product that's in their hands.”
AI-powered video analytics are one potential solution.
“Self-checkout is increasing the use of cameras in various form factors and for various purposes,” says Bosch’s Reese. “Examples include cameras monitoring the scanner feeding object recognition software — integrated with the point-of-sale (POS) system — and even cameras built into the POS system showing the shopper’s image together with the scanned items data.”
frantic00/Shutterstock.com
The surveillance system also could use facial recognition to check each self-checkout shopper against a database of known shoplifters. This strategy depends on what a particular country allows.
“The deployment of face recognition has been slowed due to privacy concerns and laws prohibiting its use,” Reese says. “However, some retailers have seen significant success when deployed in locations where it is allowed.
“Global databases of offenders are available, as are databases from Organized Retail Crime Associations (ORCAs). Retailers can choose to access these databases or use only internal databases created from their own surveillance systems.”
For AV firms, automation’s emerging risks also create more business opportunities.
“I think retailers are really trying to limit risk by putting enough cameras in place that they have the high-quality video they need from multiple angles,” Payne says. “Then they can do the forensic analysis when issues come up.”
Finally, in some cases, friction is preferable, such as a traditional checkout station for people who can’t use the new technologies. That means an old type of fraud will be around for a while.
“Sweethearting is an employee giving away goods to a ‘sweetheart’ customer by either skipping the scan altogether or scanning an item of lower value,” Reese says. “High-resolution cameras integrated with the POS system can help detect sweethearting. AI-based object classification will further improve these systems.”
Top photo credit: MariaX/Shutterstock.com