Annals of Mathematical Sciences and Applications
Volume 3 (2018)
Special issue in honor of Professor David Mumford, dedicated to the memory of Jennifer Mumford
Guest Editors: Stuart Geman, David Gu, Stanley Osher, Chi-Wang Shu, Yang Wang, and Shing-Tung Yau
A survey of exemplar-based texture synthesis
Pages: 89 – 148
Exemplar-based texture synthesis is the process of generating, from an input sample, new texture images of arbitrary size and which are perceptually equivalent to the sample. The two main approaches are statistics-based methods and patch re-arrangement methods. In the first class, a texture is characterized by a statistical signature; then, a random sampling conditioned to this signature produces genuinely different texture images. The second class boils down to a clever “copy-paste” procedure, which stitches together large regions of the sample. Hybrid methods try to combine ideas from both approaches to avoid their hurdles. The recent approaches using convolutional neural networks fit to this classification, some being statistical and others performing patch re-arrangement in the feature space. They produce impressive synthesis on various kinds of textures. Nevertheless, we found that most real textures are organized at multiple scales, with global structures revealed at coarse scales and highly varying details at finer ones. Thus, when confronted with large natural images of textures the results of state-of-the-art methods degrade rapidly, and the problem of modeling them remains wide open.
texture synthesis, exemplar-based, image statistics, images patches, Julesz conjecture, convolutional neural networks
We thank Rafael Grompone von Gioi for valuable corrections and suggestions, Arthur Leclaire and Yang Lu for their feedback and for helping produce some of the experiments, and the anonymous referees for valuable advice.
Received 18 June 2017
Published 27 March 2018