Events that made my day – creating a visual diary

… or the “emotional camera” project

 

This project aims at developing techniques and algorithms to be used with a wearable “always-on” camera, which will automatically identify and select the best frames representing the personal “highlights” of the day.

“Always-on” wearable cameras are already on the market and can capture every waking moment of one’s life, but most of the enormous amounts of data they produce are neither necessary nor interesting. In this project we aim to tackle one of the great challenges for photography of the future, namely how to find what we care about in the flood of videos/images. We aim at selecting the best personal snapshots of the day based on aesthetic and emotional features.

For this we make the following two assumptions:

• The most important personal moments(“highlights”) of the day are those where one is emotionally involved
• The resulting images illustrating these moments (“best frame”) should be aesthetically pleasing

To select just a few highlights from the vast amounts of images taken by the camera throughout the day, we propose the following filtering process:

• The first filter level selects the image sequences from periods of time when the user was (emotionally) aroused. To detect emotional arousal we will utilize wearable bio-sensors that the user will wear alongside the camera.
• The second filtering level will then automatically select the best single images from each scene based on their aesthetics, interestingness and originality.
• The resulting images will be post-processed to amplify their emotional context.

These can be then saved in the form of a visual diary or shared with others on blogs and social network sites. A user feedback and machine learning loop will enable the system to learn the user’s personal aesthetic preferences, his/her daily rituals (to avoid repetition in the pictures), and to fine tune the system’s interpretation of the bio-signals.

Publications

Will follow…