Sustainable shopping through real-time visualization of product information on mobile devices
Unsustainable production practices and excessive consumerism are major drivers of the climate and biodiversity crises, which are two of the most significant challenges of our time. Although more and more people are concerned with the environmental and social impacts of their purchasing habits and consequently want to consume more sustainable, surprisingly few do so. This paradox is called the “attitude-behavior-gab”, whereby approximately 65% of consumers claim that they want to shop more sustainable, but only 26% manage to do so [1]. One of the main reasons is the lack of access to product information on site, which would allow a consumer to distinguish between sustainable and non-sustainable products quickly. Consequently, sustainable shopping is associated with an additional effort that many consumers are not willing to make and thus maintain their non-sustainable purchasing habits.
The goal of this project is to help consumers overcome this initial hurdle in order to promote sustainable shopping. All necessary product information should be brought directly to the consumer with minimal effort. This is achieved through an augmented reality smartphone application that provides sustainability information of products on a supermarket shelf in real-time. Sustainable products should be visually highlighted, allowing consumers to distinguish them from non-sustainable products at a glance without having to read the product packaging. This should facilitate a sustainable shopping experience with minimal additional effort.
The primary technical challenge of this project is the development of a state-of-the-art product recognition pipeline that can run in real-time on mobile phones with limited computational resources while being able to reliably recognize tens of thousands of products in-the-wild. This requires the development of new and efficient neural network architectures for product recognition as well as the creation of a comprehensive product database for model training and on-device indexing.
Project Partner
Funding
This project is partly funded by the Vienna Business Agency (grant 3540290).
Publications
Strohmayer J. and Kampel M., “Real-Time Supermarket Product Recognition on Mobile Devices Using Scalable Pipelines,” 2023 IEEE International Conference on Image Processing (ICIP), Kuala Lumpur, Malaysia, October 2023, pp. 420-424, doi: https://doi.org/10.1109/ICIP49359.2023.10223137.
Strohmayer, J., Kampel, M. (2023). Domain-Adaptive Data Synthesis for Large-Scale Supermarket Product Recognition. In Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14184. Springer, Cham. https://doi.org/10.1007/978-3-031-44237-7_23.
Strohmayer J. and Kampel M., “Data Synthesis for Large-scale Supermarket Product Recognition”, In the Proceedings of the Austrian Association for Pattern Recognition (OAGM) Joint Austrian Computer Vision and Robotics Workshop, November 2021, St. Pölten, Austria, doi: https://doi.org/10.3217/978-3-85125-869-1-11.