Dude, where’s my yogurt? Algorithms and competition in UK retail

Video URL

  • Competitve landscape; every retailer has ~12 direct competitors. Customers are very price-sensitive.
  • Compare a product against all competitor products (within a category), and use a number of metrics to determine similarity, including a Jaccard Index over words in the product name.
  • Pairs of similar products are tagged by humans and sent to a data warehouse used for competitive intelligeence queries.
  • Use human labor to classify products into categories so fewer product pairs require human intervention (only compute pairs within a category).
  • Possible ML improvements but these aren’t implemented yet.
  • Competitor data comes from external sources; essentially you pay people to scrape this data for you or actually go into stores and record pricing information.
  • Didn’t go into implementation specifics: what stack are they using, how is this deployed, etc.