Generative Adversarial Networks Based Reconstruction and Restoration of Cultural Heritage

Abstract

Cultural heritage takes an important part in defining the identity and the history of a civilization or a nation. Valuing and preserving this heritage is thus a top priority for governments and heritage institutions. Through this paper, we present an image completion (inpainting) approach adapted for the curation and the completion of damaged artwork. Our approach uses a set of machine learning techniques such as Generative Adversarial Networks which are among the most powerful generative models that can be trained to generate realistic data samples. As we are focusing mostly on visual cultural heritage, the pipeline of our framework has many optimizations such as the use of clustering to optimize the training of the generative part to ensure a better performance across a variety of cultural data categories. The experimental results of our framework were validated on cultural dataset of paintings collected from Wiki-Art and the Rijksmuseum. We used the divide-and-conquer strategy by clustering the training data into different small clusters containing similarly looking images to train smaller Specialized DCGANs. The training has been made on five painting categories containing 2000 paintings each, which took an average of 6.1 training hours. Training the Specialized DCGAN on 1200 paintings from one of the clusters took 3.4 training hours. The inpainting results of the Specialized DCGANs are clearly better in quality than the results of a DCGAN trained on mixture of paintings or on painting category.