At MishiPay we started because we hated waiting in a line. Now that shoppers never have to wait in a line, what’s next? What if they never miss out on the best deals? What if they always find the right product at the right time inside the store? Rich data on the shopper’s behavior inside the store have helped us understand more about preferences and last year we started experimenting with product recommendations while the shopper is browsing — but this time, in the physical store!
Product recommendations in the ecommerce world have been a “must-have” for every platform with any significant success. Amazon claims that 1 in 3 products sold are coming through recommendations. However, in the physical store the lack of an interface to serve the recommendations, became a blocker. Traditionally recommendations are made by staff who will recommend products based on your queries or based on what you are looking for. This lacks access to your past purchases and of course is bound by the staff’s knowledge as well as personal biases.
However, the concept of recommending things while the shopper is browsing is universally accepted. MishiPay thus became the first Scan & Go provider to launch product recommendations during the browsing journey inside physical stores with stunning results.
Our business domain enables unprecedented data science, not only in product recommendation but also in shopper behavior, preferences, loyalty as well as on the retailer’s side with inventory management, pricing and many other fields.
We capture data every time a shopper makes a transaction, we know every detail such as price, promotions, name, variations, quantities on every SKU tagged to every single store. In addition to that, we know the timestamp of every product scan, the order of the scans, deletions of products: all of this creates a digital mapping of the shopper experience where we can position the shopper and calculate the time between scans and the movement in the physical store. On top of that, shoppers sometimes delete product A and immediately scan product B, which reveals shopper preferences, something impossible to compute at scale in a physical store, if done manually by the staff or using the traditional POS which wouldn’t be able to capture this information as it isn’t in the shopper’s hand when browsing. We can then add this browsing behavior to their recorded preferences to coupons and discounts, and their loyalty to stores, brands and retailers.
Let’s walk through the custom journey to understand this better.
1 – Shopper enter the store and can access the Scan & Go App via Webapp or by downloading the App. Our Webapp is the go-to device since it requires 0 effort, in 4 seconds you are ready to scan your first product.
2 – If the shopper is new, then our algorithm doesn’t take any prior history into account. If the shopper is returning, then we immediately load the historical transactions to the engine.
3 – The shopper makes the first product scan. The product recommendation request is processed by an algorithm that takes into account: the product (barcode), the store, the date, the inventory available, the price, historical data and placement inside the store.
The engine now needs to produce ranked product recommendations — this needs to be fast as the shopper has scanned the product and now will look into other areas and continue browsing.
It is easy to assume that this looks like a collaborative filtering problem. As such, we do use some standard collaborative filtering algorithms (e.g. those who have bought what you have scanned have also bought …).
We do also get help from all past historical transactional data and this helps us solve the cold start problem. In our case, we don’t have enough explicit data (shopper asking for specific products or rating them, but we do take historical data into account) as well as current basket details.
Mixed-effects modelling is also a solution that can help, given the longitudinal aspect of the problem, i.e. as the shopper is shopping with MishiPay we increase the level of data we capture. There are many latent features that also take place, plus the addition of taking the retailer’s own data into consideration.
Special cases also exist: in fashion products, seasonality is playing a big role as the models need to be retrained for new product data. On top of that, barcode and SKU levels need to be taken into account, if product A was scanned and it is a black Medium t-shirt, then the product recommendations will need to be for the same size — hence if Medium size is not available for the recommended products, then the recommended product will not appear.
But on top of that we also need to take into account the inventory of the recommended products, as we don’t want to recommend a great accessory that is out of stock. Also, location of recommended products is critical — the great cheese that will go with the pasta might be in aisle 5, while the shopper is in aisle
Once this API response is provided, the shopper is greeted with a carousel card that provides options. Given the modern smartphone screen sizes, the maximum products that the carousel can present is 3.
That means that the first 3 products will be critical to be shown to the shopper. If none of them make any sense, the feature loses its delightful essence and becomes spam or starts feeling like an ad.
The final UI was not released until multiple iterations.
On the one hand, we tried not to reinvent the wheel in terms of serving the recommendations. Scan & Go is already a new concept to most customers and requires focus and user-education in a “shopping session” timespan. Adding new features only complicates the journey.
On the other hand, we had to take several limitations into consideration. From the attention span of the customer all the way to confusion around which product is added and what to do next, many variables were taken into consideration until we had minimized error rate and satisfaction.
Therefore the rank of the recommended products is critical. In order to optimize this we assign a cost to every ranked position. This cost is stored in a matrix across sessions the specific products have appeared. If the product is successfully added by the shopper, then the cost becomes 0. Else, the product carries the cost. The higher the cost the lower the ranking.
This allows us to show more relevant products first. This particular ranking also creates its own feedback loop, products with high cost eventually fade out of the recommendation engine.
Therefore we re-run the product recommendations in order to ensure all recommendations are up to date in terms of: seasonality, inventory availability, and ranking.
Algorithms are great, but humans can provide expertise that sometimes is lost in these calculations. This is where local knowledge and information is provided. Store staff can provide their own set of recommendations. This process helps us in terms of understanding the process of creating the product recommendations as well as identifying “hidden” patterns.
As mentioned before, the shopper’s location is important in order to recommend products that can be seemingly near the shopper at the point of the recommendation.
Not all shoppers are in the same state of their shopping journey when they see a product recommendation. There are differences in terms of new app shoppers (education of basic scan & go is still ongoing), repeat shoppers (they know how to use the basic features), first scan, second scan or approaching the ABS. This helps us understand the effectiveness that showing product recommendations will have — we don’t want to miss out on showing a great recommendation on your first scan but this can’t come at the detriment of shopper satisfaction and confusion. Similarly, we don’t want to show a recommendation every time the shopper scans an product — in a grocery environment where the Average Basket Size can easily surpass 50 products, showing 50 recommendations is not a good “recommendation”.
On top of that the use case is affecting the product recommendations. Imagine you are in an airport where you just want to quickly buy a water bottle as your gate is closing. Even the best product recommendation cannot ensure a conversion. When the shopper journey is limited by time or is limited by focus and budget then we need to ensure that unnecessary product recommendations won’t be shown.
The goal of this first piece on in-store product recommendations is mainly to share some of the exciting things we are working on at MishiPay. Of course, our production is more complicated and there are many more elements that have not been addressed (cross-retailer pollination of product recommendations algorithm, deep-down on shopper journey and success rate breakdown by retailer, session time and differences among our 13 live regions and many more). Perhaps also to feed the academic conversation around product recommendations in a completely new environment with its own rules and regulations. We definitely want to inspire creativity to the readers and can’t wait to hear from you.
Let us explain how our technology can help you increase your average basket size.
Let us explain how our technology can help you increase your average basket size.
Vice President of Product @ MishiPay
Follow him on LinkedIn
For more related studies - Engineering Medium
Share this with your colleagues 35% larger baskets and five-star ratings: How MishiPay delivers ROI for both shops and shoppers MishiPay, global leaders in the
Share this with your colleagues Learn how MUJI combines MishiPay Scan & Go and Product Recommendations, enabling them to benefit from Amazon Go levels of
MishiPay Platform MishiPay Self Checkout Kiosk With an 80% smaller footprint, you can capture 100% of the store transactions. Get in touch Delivering Value For
