Dude, where’s my yogurt? Algorithms and competition in UK retail
- Competitve landscape; every retailer has ~12 direct competitors. Customers are very price-sensitive.
- Compare a product against all competitor products (within a category), and use a number of metrics to determine similarity, including a
Jaccard Index over words in the product name.
- Pairs of similar products are tagged by humans and sent to a data warehouse used for competitive intelligeence queries.
- Use human labor to classify products into categories so fewer product pairs require human intervention (only compute pairs within a category).
- Possible ML improvements but these aren’t implemented yet.
- Competitor data comes from external sources; essentially you pay people to scrape this data for you or actually go into stores and record pricing information.
- Didn’t go into implementation specifics: what stack are they using, how is this deployed, etc.