Fusing Multispectral Images and Chemical Material Analysis

Status: available
Supervisor: Simon Brenner


In the last 10 years the Computer Vision Lab participated in a series of interdisciplinary research projects on the analysis of ancient manuscripts. While we were concerned with the enhancement of degraded text based on Multispectral Imaging (MSI), colleagues from the Academy of Fine Arts performed chemical meterial analysis of the same manuscripts via X-Ray Fluoprescence (XRF), Fourier Transform Infra Red Spectroscopy (FTIR) or Raman Spectroscopy.

Multispectral Imaging: In this example, a palimpsest (a text which has been erased to recycle the parchment) is revealed.
X-Ray Fluorescence spectra of different materials.

We are now in the possession of both MSI- and material-data for dozens of manuscripts, which is, to our knowledge, quite unique. Now the time is ripe to develop approaches to fuse these different types of data, which have so far only been treated individually.


While the MSI data consist of raster images, the material data are point measurements of various spectra. So far, the results of chemical material analysis were delivered accompanied by a color image, in which the measurement points were indicated, like so:

1. Development of a data format

A first step would be the definition of a data format that allows the integration of point measurements of XRF/FTIR/etc. spectra with MSI images. The format must provide the information where a measurement was made (image coordinates), the size of the measurement point, and the spectral data that was measured there.

2. Development of a nomacs plugin

A plugin for the open source image viewer nomacs (that was developed at CVL) for the visualization and/or editing of material data (see objective 1). C++ skills required.

3. Data Analysis

Find correlations between MSI-signatures and material spectra and investigate the possibility of estimating material from MSI images. Keyword machine learning. Also possible: can point measurements be extrapolated across pages/manuscripts based on MSI data?

Depending on the type of project (MA/BA/PR), one or more of the above named tasks can be treated.



  • Interest in the topic
  • Reliability