Ravi Annavajjhala – CEO, Kinara Inc.
getty
Checkout-free shopping, which promises greater benefits for both shoppers and retailers, is the new rage in brick-and-mortar stores. Pioneered by Amazon, this trend encompasses a rapidly growing number of retailers around the globe, including Carrefour, Sainsbury’s, 7-11, Circle-K and others. Shoppers pick up what they need and simply walk out with funds deducted from their bank accounts and a receipt sent to their smartphones.
This convenience improves retailers’ profitability by using the shoppers’ data to better understand their needs, in real time, leading to enhanced inventory management, minimized lost revenue and lower labor costs.
The foundation of a seamless, checkout-free shopping experience is a set of sophisticated, compute-intensive AI models that map a customer’s journey through a store in order to answer a simple question: “Who bought what?”
The Business Case For Checkout-Free Shopping
Shoppers want, indeed expect, fast, easy and frictionless ways to shop. Checkout-free shopping offers that by enabling shoppers to select what they need and simply leave the store. This offers increased profitability for retailers by enabling them to accurately record and minimize lost revenue while delighting their customers with a convenient shopping experience over which they have full control.
The decreasing costs of sensors, cameras and computing systems, along with the considerably improved accuracy of AI models, make checkout-free shopping solutions a reality. However, challenges for scaling the solution remain.
While AI models can analyze multiple video feeds from cameras deployed throughout a store, the hardware they run on requires intensive processing power and lots of it. That’s expensive in terms of the power required to run the tracking devices because it can drive energy bills significantly higher due to the power consumed by the GPUs and servers that are constantly operating during retail hours and sometimes around the clock.
Hurdles To Large Scale Deployment Can Now Be Overcome
An average convenience store spans roughly 2,000 square feet. With overhead cameras installed every 10 square feet, each store would require 200 cameras. These cameras then generate video streams at 1080p resolution and 30 frames per second.
That’s a lot of cameras and a lot of data storage.
One approach to resolve this issue would be to upload the video feeds to the cloud. However, uploading video feeds from 200 cameras is extremely expensive and requires vast amounts of network bandwidth that has to be 100% available. Any glitch in network reliability, however momentary, could adversely impact the accuracy of the results. Cloud-based deployment can be used to prototype solution ideas, but at the moment, it is not yet the real solution that will lead to more widespread use.
Meeting The Challenge Through Energy Efficiency
Until recently, such challenges made full-scale deployment of checkout-free shopping cost-prohibitive. Because many checkout-free shopping deployments still utilize the cloud or high-cost servers, it’s difficult to replace incumbent and traditional solutions. However, silicon technology has evolved with more energy-efficient alternatives that are closer to the edge, or in some cases, right inside the cameras. These alternatives, in the form of edge AI processors or accelerators, offer more than just better energy efficiency.
Latency is another very important factor. For example, when sending data to the cloud for processing, applications typically experience latencies greater than 100 milliseconds; compare this to sub-5 millisecond latencies on edge, which enable real-time edge analytics.
From a performance standpoint, any edge AI processor suitable for the job of camera-based inferencing must be capable of running, in real-time, a series of AI models of varying complexity. For example, a baseline checkout-free shopping application could run four AI models to analyze video feeds from the cameras.
1. The person detection and tracking model tracks individuals’ movements as they browse throughout the store, picking up and dropping off from one camera zone to the next.
2. The person reidentification model identifies each individual walking into a store. Due to GDPR compliance and other privacy requirements, the system must blur each person’s face. The system should, therefore, use other, detailed attributes such as attire, gender and height to identify each person.
3. The pose estimation model determines whether the person picked up or returned a product.
4. The product classification model identifies each product that’s been picked up or put back.
By weaving these separate customer actions together into a whole, retailers can accurately identify and charge each customer for what they selected and departed with. No other interaction is required.
Walk in. Get what you want. Walk out.
Edge AI Processors For Lowest CAPEX At Scale
In general, retailers can anticipate several challenges with adopting these models. If they are new to this technology, one of the first challenges will also be one of the first steps: collecting sufficient data to train the AI models effectively.
Retailers and/or system developers that have “dumb” cameras in their stores have already been collecting infinite amounts of video data for this purpose, and that video data must be labeled, which is often a huge project, although there are AI-based tools that help with this process.
AI model selection is another significant challenge. It’s interesting to note that very few developers create their own AI models. It’s almost unheard of because a) it requires a special class of data scientist, and b) there are a wealth of freely available models that serve the purpose, so the developer’s job is to train the already existing models.
In selecting the right edge AI processor, assuming that all factors such as performance, power and price have been met, the most important thing is ensuring that the company’s development tools are sufficient. No one wants to hand-build models. At a minimum, the processor vendor’s tools must be able to take an off-the-shelf model and optimally compile it to the associated architecture. This can’t be overstated. While there are many AI processor vendors, one of the most important questions is whether those tools are easy to use.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?