Person Detection and Data Diversity

Status: available
Supervisor: Roman Pflugfelder

Problem Statement

Visual person detection is a foundational building block in almost every application from pedestrian detection in surveillance and autonomous driving, to person perception in robotic systems, to human localisation in user-centric computing. Person or pedestrian detection has made a significant leap in the last years, by re-discovering the connectionist view on the problem.

The current approach is to model the localisation and classification of persons by a convolutional neural network (CNN). Such networks are trained end-to-end with real images and given labels, i.e. with rectangles enclosing the visible persons. One way to improve current state of the art is to use more diverse data capturing persons in scenes typical for the considered application. Imagine future autonomous driving where scenes are enormously diverse, from city traffic, to highway driving to rural roads, on all continents all around the globe. Here the question is if diversity is constrained by common features all scenes share in natural and man-made environments and if this diversity can be captured by large but finite datasets.


Compose image datasets, train an existing deep neural network and compare the results with existing benchmarks. Implement the neural network for person detection.


The thesis can be combined with a preceding Informatik Praktika.

  • Review literature
  • Create training and validation dataset based on previous work
  • Implement training and test algorithms
  • Test results on CityPersons dataset
  • Optional: Improve algorithms for better results
  • Written report/thesis and final presentation


  • Basic knowledge in computer vision
  • Basic experience in Matlab, C++, Python
  • Interest in Machine Learning, maths, statistics
  • Interest in GPU programming


  • S. Zhang, R. Benenson, B. Schiele. CityPersons: A Diverse Dataset for Pedestrian Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  • S. Ren, K. He, R. Girshick, J. Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 39 no. 6, p. 1137-1149, 2017.
  • G. Sperl, R. Pflugfelder. Person Classification with Convolutional Neural Networks. Master’s Thesis, TU Vienna, 2016.