Student Topics

Self-Supervised 4D Point Cloud Feature Learning for Activity Recognition

Master Practical Training Project

Status: not available
Supervisors: Irene Ballester, Martin Kampel

Problem Statement

This project aims to address the challenges associated with the expensive and time-consuming annotation of 3D data by exploring a self-supervised approach for the extraction of 4D spatio-temporal features from dynamic point cloud data. Specifically, the project investigates the prediction of the temporal order of shuffled point cloud clips as an auxiliary task, inspired by [1], but with a special focus on working with datasets from real-world scenarios. Different models for 4D point cloud processing will be applied to a large dataset to evaluate the suitability of these methods to improve the performance of action recognition methods on smaller 3D datasets (MSRA and own curated dataset).

Goal

To assess the feasibility and effectiveness of a self-supervised method for learning 4D spatio-temporal features from dynamic point cloud data coming from real-world scenarios, with a focus on improving the performance of action recognition models on smaller 3D datasets.

Tasks

  1. Literature review on the state of the art of self-supervised methods for dynamic point clouds
  2. Implement different models for 4D point cloud feature learning
  3. Implement self-supervised method
  4.  Evaluation on MSRA dataset and own curated dataset
  5.  Preprocess and annotate real-world data
  6. Report
[1] Wang, H., Yang, L., Rong, X., Feng, J., & Tian, Y. (2021). Self-supervised 4d spatio-temporal feature learning via order prediction of sequential point cloud clips. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 3762-3771).

Contact
Irene Ballester
irene.ballester@tuwien.ac.at