Multi-frame Superresolution

Multi-frame Superresolution

Resolution of array-based infrared cameras is limited by a large pixel spacing (required for thermal isolation) combined with a small size array. Providing a sufficiently wide field-of-view results in severe undersampling of the digital image. Undersampling causes aliasing, which manifests itself through rough block-edges and by corruption of image detail. Undersampling not only hampers visual inspection, but also reliable detection, segmentation and subsequent analysis.

This project will develop a method to obtain super-resolution in undersampled image sequences acquired by a vibrating infrared camera. The vibrating sensor yields randomly shifted (and rotated) instances of the same scene.

Proper fusion of a series of aliased frames will increase the spatial resolution in exchange for a lower temporal resolution. This process will boost the image quality and reduce or complete solve the aliasing. The output sequence allows reliable detection, segmentation and analysis of the scene.

We will develop a software prototype of the system and a mathematical framework for modeling aliased image sequences. The robustness of the proposed algorithm is a key issue. The system should indicate the quality of the output sequence in combination with a confidence value for each result. This allows monitoring of the system's performance and reliability. The performance should degrade gracefully (or switch back to the input image sequence) to guarantee continuity in the information sequence. The algorithms should have a clear perspective on real-time implementation.

We distinguish two major components: image registration and image fusion. To obtain high flexibility, all time-variant information (displacement vector, rotation angle, motion vector, etc.) that is needed to obtain super-resolution has to be extracted from the low-resolution image sequence. The method should be robust, reliable, and degrade gracefully for increasing noise levels. Image fusion should automatically balance between resolution enhancement and signal-to-noise ratio improvement.

Project leader: prof.dr.ir. L.J. van Vliet

This project is supported by "Het ministerie van Economische Zaken" as part of the IOP program  "Inovatiegerichte Onderzoeksprogramma Beeldverwerking". IOP beeldverwerking is ondergebracht bij Senter.