Robust and affordable retail customer profiling by vision and radio beacon sensor fusion

Abstract The development of reliable and precise indoor localization systems would considerably improve the ability to investigate shopper movements and behaviour inside retail environments. Previous approaches used either computer vision technologies or the analysis of signals emitted by communication devices (beacons). While computer vision approaches provide higher level of accuracy, beacons cover a wider operational area. In this paper, we propose a sensor fusion approach between active radio beacons and RGB-D cameras. This system, used in an intelligent retail environment where cameras are already installed for other purposes, allows an affordable environment set-up and a low operational costs for customer indoor localization and tracking. We adopted a Kalman filter to fuse localization data from radio signals emitted by beacons are used to track users’ mobile devices and RGB-D cameras used to refine position estimations. By combing coarse localization datasets from active beacons and RGB-D data from sparse cameras, we demonstrate that the indoor position estimation is strongly enhanced. The aim of this general framework is to provide retailers with useful information by analysing consumer activities inside the store. To prove the robustness of our approach, several tests were conducted into a real indoor showroom by analysing real customers behaviour with encouraging results.