The following topics are part of our collaboration with Cogvis Gmbh, For processing deep learning networks, Cogvis as well as CVL have powerful hardware available which can be used. All Master students get an internal company supervisor who is responsible for the topic and the guidance of the students. A renumeration and/working place is possible.
Topic 1: BEHAVIOUR DETECTION AT LUNCH TO ASSIST PEOPLE WITH DEMENTIA
Problem Statement: Nursing staff members spend a lot of time on routine tasks, such as assisting in basic activities of daily living (toileting, eating, bathing, transferring, grooming, etc.), ensuring the safety of patients, or walking tours and providing nursing care to their patients. The task of eating lunch is part of the daily routine, and takes a high amount of nursing time. The goal is to increase people’s autonomy and digitize nursing assistance in this area. For people with cognitive impairment, this currently means that a caregiver has to guide them or at least support them. This control measure may restrict people’s autonomy, is unpleasant for both sides and in institutions it also binds care personnel.
Goal: Developing an action recognition system to detect specific behaviors at lunch using 3D data (range images). Data acquisition as well as analyzing all activities which are part of a lunch taking place at a table are foreseen. Specific behaviours should be recognized and evaluated accordingly?
Tasks:
- Study the State–of–the–Art on foot recognition and foot intake
- Select a methodology and implement it
- Provide Results and Evaluation
Keywords: Computer Vision, skeleton tracking, 3D Vision, Dementia
Topic 2: BEHAVIOUR DETECTION IN A BATHROOM ENVIRONMENT TO ASSIST PEOPLE WITH DEMENTIA
Problem Statement Nursing staff members spend a lot of time on routine tasks, such as assisting in basic activities of daily living (toileting, bathing, transferring, grooming, etc.), ensuring the safety of patients, or walking tours and providing nursing care to their patients. The visit of a toilet is part of the daily routine, and takes a high amount of the nursing time. The goal is to increase people’s autonomy and digitizing nursing assistance in this area. It is important that everyone can use a toilet as comfortably and barrier-free as possible. For people with cognitive impairment, this currently means that a caregiver has to escort them during toilet visits or at least wait in front of it. This control measure may restrict people’s autonomy, is unpleasant for both sides and in institutions it also binds care personnel.
Figure: Range Image from a bathroom
Goal
In this thesis, a system should be developed that recognizes certain actions to identify when assistance on the toilet is needed. 3D sensors will be used for private sphere protecting action recognition using behaviour recognition algorithms already available but trained for other application areas. If irregularities (e.g. forgetting to wash hands) or incidents (e.g. falling) are detected using this method, assistance is offered. This assistance can either be provided (1) by alerting nursing staff, which intervenes accordingly, or (2) through interaction (e.g. audio instructions, interactive display). Additional functionality to integrate would be to calculate the frequency of the toilet visits, which can be used to determine whether enough water is consumed by a patient. This is an important indicator for nursing staff to avoid the threat of dehydration, especially for people who have difficulty expressing themselves. Thus, the main tasks of the thesis are:
- Developing an action recognition system to detect specific behaviours on the toilet
- Develop an interaction tool (alarm notification and dialogue communication between user and system in order to guide them through the toileting process)
This thesis is part of the DIANA project.
Publication: Ballester, I., Mujirishvili, T., Kampel, M. (2022). RITA: A Privacy-Aware Toileting Assistance Designed for People with Dementia. In: Lewy, H., Barkan, R. (eds) Pervasive Computing Technologies for Healthcare. PH 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 431. Springer, Cham. https://doi.org/10.1007/978-3-030-99194-4_20
Topic 3: SYNTHETIC DATA GENERATION USING DEEP LEARNING FOR EARLY REHABILITATION IN INTENSIVE CARE
UNITS (ICU)
Problem Statement: For patients treated in ICUs due to severe injuries (accidents, heart transplants, etc.) it is crucial to perform physical exercises in bed (early rehabilitation). Patients cannot leave the bed and without early rehabilitation the loss of muscle mass can pose another risk. Also, a result of early rehabilitation actions is the minimization of long–term effects. To motivate the patients in doing their exercises, physiotherapy should be supported by gamification. The idea is to control a game using the
limb movements of the patient lying in bed. Therefore, a pose estimation algorithm of the person lying
in bed is the basis. Deep Learning Algorithms needs a huge amount of training data. As access to ICUs is
limited and thus recording sessions are hardly possible, the necessary amount of training data is difficult
to gather. To overcome this problem the ICU scene is simulated allowing for the generation of the
synthetic data needed. For the pose estimation algorithm to be robust it is crucial that the data used for
training is as close to real recordings as possible. This means that not only the simulated scene needs to
be accurate but also that the correct noise model needs to be applied to the “perfect” renderings of the
simulation.
Figure: (left) Simulation scene of a person lying in an ICU bed (Software: Blender). (right) real depth image with noise of a person lying in bed, color-coded.
Goal: The goal is to generate a synthetic dataset in a simulation (blender–python) mimicking the setup of
ICUs as close as possible. Using state–of–the–Art Neural Networks like General Adversarial Networks, the
noise model of the camera used in the real world should be identified and applied to the simulated 3D
renderings. A basic simulation including the used camera, person models, pose data, the bed and the simulation script is already available. To decrease complexity of the simulation and make the latter pose estimation more robust to different surroundings, a bed detection algorithm could optionally be implemented. This algorithm cuts out the bed and thus removes complex objects normally present in ICUs.
Tasks:
- Get familiar with existing simulation pipeline
- State of the Art research
- Improvement of existing simulation pipeline
- Generating an advanced noise model by using for example real world images and General
- Adversarial Networks (GANS).
- Evaluation of generated synthetic data
- Bed detection algorithm using classic computer vision methods or deep learning methods(optional)
- Develop Pose Estimation (optional)
Keywords: Computer Vision, Noise mode, AI, General adversarial network, Blender, Rehabilitation
Topic 4: ESTIMATION OF A PATIENT’S POSE IN AN INTENSIVE CARE UNIT FOR THE GAMIFICATION OF EARLY
REHABILITATION EXERCISES USING DEEP LEARNING
Problem Statement: For patients treated in ICUs due to severe injuries (accidents, heart transplants, etc.) it is crucial to perform physical exercises in bed (early rehabilitation). Patients cannot leave the bed and without early rehabilitation the loss of muscle mass can pose another risk. Also, a result of early rehabilitation actions is the minimization of long–term effects. To motivate the patients in doing their exercises, physiotherapy should be supported by gamification. The idea is to control a game using the limb movements of the patient lying in bed. A pose estimation algorithm based on depth images of the person lying in bed is the basis.
Figure: (left) Synthetic depth image of a person sitting in an ICU bed with the joint in question (here: head) marked by a cross. (right) Deep believe maps are heatmaps of the algorithm’s believe of the joint in question’s location, here: the head. – A possible output of Deep Learning Pose Estimation Networks.
Goal: The goal is to develop a robust Pose Estimation Algorithm using state–of–the–art Deep Learning methods to detect small movements of a patient lying in bed. A promising network architecture is for example, the Stacked Hour–Glass Neural Network. The input is a depth image either recorded from a
real scene or synthetically generated in a simulation.
Work:
• State of the Art Research
• Gather dataset (real and synthetic data provided)
• Implement Pose Estimation Algorithm
• Evaluation
• Improve synthetic data generation pipeline (optional)
• Deployment of algorithms (optional)
Keywords: Computer Vision, Pose estimation, AI, Deep Learning, Rehabilitation
Topic 5: DEPLOYMENT (INFERENCE) OF DEEP LEARNING POSE ESTIMATION MODELS ON DIFFERENT HARDWARE
FOR THE USE IN INTENSIVE CARE UNITS (ICUs).
Problem Statement: For patients treated in ICUs due to severe injuries (accidents, heart transplants, etc.) it is crucial to perform physical exercises in bed (early rehabilitation). Patients cannot leave the bed and without early rehabilitation the loss of muscle mass can pose another risk. Also, a result of early rehabilitation actions is the minimization of long–term effects. To motivate the patients in doing their exercises, physiotherapy should be supported by gamification. The idea is to control a game using the limb movements of the patient lying in bed. Therefore, a pose estimation algorithm using depth images of the person lying in bed is the basis. The developed algorithms then need to be deployed on appropriate hardware with an interface to the game.
Figure: (left) Prototype setup of the game developed in this project (the game is not our part, we do the pose estimation). (right) Nvidia Jetson Nano single chip-computer with an onboard GPU. A possible hardware for inference.
Goal: Development of robust inference software with an interface to the game and the deployment on hardware with respect to compute power, costs, and reliability. The inference should be robust and reliable. The hardware might be a Windows i5 shuttle plus compute stick, a Jetson Nano or Raspberry PI single chip computer, etc. .
Work:
• State of the Art research
• Inference software with interface to the game used for early rehabilitation
• Evaluation of different hardware
• Deployment of inference software
• Implementation of Pose Estimation Deep Learning Network (optional)
Keywords: Computer Vision. Inference, AI, Deep Learning, Software Embedding, Raspberry PI, Pose
Estimation
Topic 6: NOVEL APPLICATIONS USING ULTRA WIDE BAND (UWB) TECHNOLOGY FOR THE USE IN HEALTH CARE
INSTITUTIONS.
Problem Statement: To date Health Care institutions like elderly homes rely on expensive resources to monitor patients. Monitoring is important as special situations like falls, cardiac problems etc. need fast attention. Computer Vision technologies like the cogvisAI 3D Sensor target exactly this problem by providing cost efficient monitoring and alarming in case of specific events. However, vision methods still have limitations. The localization of people not being in the 3D sensor’s field of view cannot be determined, or nurses cannot be differentiated from patients. To overcome such limitations the
trending technology UWB should be assessed to identify novel features for the support of existing 3D Vision technologies.
Goal: Identification and implementation of novel features using UWB for the use in Health Care Institutions. These could be person localization, person identification (nurse, patient), or any other
useful feature.
Tasks:
• State of the Art research on UWB and 3D Vision in Health Care
• Implementation of novel UWB enabled feature
• Evaluation of implemented features
Keywords: Computer Vision, UWB, Health Care
Contact: martin.kampel@tuwien.ac.at