We’ve become familiar to seeing the phrase “because you shopped for similar items” when we shop online, or “because you watched The Crown…” when we browse for what to watch on a streaming service.
These sorts of recommendations – sometimes called market basket analysis – have been closely linked to the success of Amazon and Netflix, the online giants who created their categories, e-commerce and streaming entertainment, respectively. The Amazon and Netflix apply market basket techniques, purportedly using the classic modeling technique, apriori, to drive some part of their recommendation engines.
But market basket analysis is not synonymous with a recommendation engine, which includes a broader array of techniques. Most importantly, other industries can also perform market basket analysis to address other business problems.
Why? Because market basket techniques don’t care what’s in the basket or even if there is a basket: they discover association patterns on any items in a set. Items in an online cart and titles in a queue are just examples of applying market basket analysis. Goods in a market basket, SKUs in the manufacturing and distribution parts of a supply chain, and even symptoms, drugs, and types of adverse drug events are problems that can be addressed with the same techniques.
The Classic Use Case – Market Basket Analysis for Retail
As the name suggests, techniques like the apriori algorithm were built with retail in mind. Retail has many transactions with many items per transaction. Traditionally, no information on the buyer was available, and certainly not preferences. So, the classic problem only finds relationships between items in transactions in a massive data set.
The brilliance of the algorithm is its simplicity: it generates three stats that can be used to understand and apply the results of the analysis:
- Support: the frequency of one transaction containing an item over all transactions
- Confidence: the likelihood that the purchase of a first item contains a second, or the frequency of transactions with both items over the frequency of transactions with the first item
- Lift: the increase in the first item when the second is in a transaction, which is calculated using confidence and support
These measures provide a lot of benefit, in theory. In practice, retailers also apply other constraints like shelf space, end caps, and what is likely to be mere coincidence due to high frequency items.
Therefore, market basket analysis for retail is constrained by what can and cannot be acted upon or used to derive insight. Target’s famous “pregnancy prediction” is an example of both finding insight on co-occurrence of products and how taking action can lead to concern by customers. Nevertheless, market basket analysis remains a valuable first step in understanding purchase patterns.
Optimizing the Supply Chain with Market Basket Analysis
Basket analysis also extends to the supply chain. Consider a warehouse like many across the country: rows of shelves contain palettes of carefully marked goods. To meet consumer expectations, the right goods need to be picked from the shelf, packed into packaging, and shipped to the customer. The same pick-pack-ship principle applies in a distributed example, when you can’t warehouse inventory.
If items are bought together frequently, organize your warehouse and your shelves to reduce the time needed to gather items for orders. The same principle applies to production of discrete products: if you have 10,000 SKUs for ball bearings, association algorithms can help you to forecast demand.
Using Market Basket Analysis for Adverse Drug Events
Of course, a general technique like market basket analysis could be applied to just about any items-in-a-set problem. Adverse Drug Events (ADEs) are negative reactions that people have to substances like drugs. ADEs are reported by pharmaceutical companies, health care providers, and consumers in government reporting systems.
In this case, the “transactions” are reports of ADEs. The items are the drugs taken, as well as other information like symptoms. Market basket analysis simply uses the same calculations to identify patterns of co-occurrence in the reports to provide insights about what drugs and symptoms go together. To take action, of course, you find “baskets” of items that can be prevented.
Syntelli Solutions data science experts are happy to help you get started with market basket analysis or other techniques that solve real business problems.
Contact us to learn more.
