Mathematics, Computation and Geometry of Data

Volume 1 (2021)

Number 1

Subsampled turbulence removal network

Pages: 1 – 33

DOI: https://dx.doi.org/10.4310/MCGD.2021.v1.n1.a1

Authors

Wai Ho Chak (Department of Mathematics, Chinese University of Hong Kong)

Chun Pong Lau (Department of Mathematics, Chinese University of Hong Kong)

Lok Ming Lui (Department of Mathematics, Chinese University of Hong Kong)

Abstract

We present a deep-learning-based approach to restore turbulence-distorted images from turbulent deformations and space-time varying blurs. Instead of requiring a massive training sample size in deep networks, we propose a simple but effective data augmentation method to firstly make deep learning approach feasible to solve turbulence problem with data scarcity. Then we employ the proposed Turbulence Removal Network (TRN), which is the Wasserstein generative adversarial network (GAN) with a $\ell_1$ cost and multiframe input to freshly restore the degraded image under atmospheric turbulence. Finally, we novelly explore the possibility to introduce a subsampling algorithm in the deep network to filter out strongly corrupted frames and enhance the restoration performance. We also investigate the viability to significantly reduce the demand of a huge number of turbulence-distorted frames in our deep network TRN without losing the quality of the reconstructed image. Experimental results demonstrate the effectiveness of the subsampling algorithm by significantly enhancing the image quality without requiring a large number of frames in deep learning.

Keywords

turbulence, data augmentation, subsampling, deep learning, WGAN

We would like to thank Mr. M. Hirsch and Dr. S. Harmeling from Max Planck Institute for Biological Cybernetics for sharing the real chimney and building video sequence. We also thank Dr. Joseph M. Zawodny from NASA Langley Research Center for sharing the moon video sequence.

Lok Ming Lui is supported by HKRGC GRF (Project ID: 402413).

Received 3 November 2019

Published 7 September 2021