Loss Prevention in Retail using Computer Vision

Industry: Retail


Our client is a large Grocery retailer in US. In their departmental stores, items left on the bottom of the
shopping cart during checkout is a major source of revenue loss.

The “Bottom of Basket (BoB)” loss or shrinkage as it is called, runs into billions of dollars.

So, our client approached us for a solution to identify "Bottom of Basket" items through a Deep
Learning based solution.

Project Journey


The objective of this Deep Learning Program was to identify whether the Shopping cart image is empty or not.

If not empty, to understand what kind of products are left behind.

Input Data Set
  • Around 30,000 images out of which 27,000 were empty and remaining non-empty
  • Batch of 128 images were created taking 50% each from Empty and Non-Empty images
  • Training data – Random sampling of 70% of the dataset
  • Validation data – 10% of the training dataset
  • Test data – 30% of the dataset

AI & Deep Learning Models used:

  • Tensorflow was used as the AI(Artificial Intelligence) library
  • VGG (Visual Geometry Group) 16 pretrained on ImageNet Dataset
  • Convolutional Neural Network (CNN) was used to build the Neural Network

Solution Design & Methodology:

  • Abzooba classified over 30,000 images using Convolutional Neural Networks (VGG16) pre-trained on the ImageNet dataset.
  • Fine-tuned top most 4 layers, out of which 3 were fully connected layer and one convolution layer.
  • The data was highly imbalanced, out of the 30,000 images tagged around 27,000 was of empty class and remaining 3,000 was non-empty class.
  • Over-sampling of the rare class or non-empty class was done.
  • Batch of 128 images were created, out of which 64 images were from the empty class (total had 27K images) and remaining 64 was from the non-empty class.
  • The batches were passed through the whole training framework as described above.
  • We achieved an accuracy of 95-96% for both the classes.


  • Tighter shrink control translated into better annual profits – between $45,000 to $60,000 per departmental store.
  • Front end productivity also increased as transactions move faster through the lane.