Towards a Hierarchical Multitask Classification Framework for Cultural Heritage

Abstract

Digital technologies such as 3D imaging, data analytics and computer vision opened the door to a large set of applications in cultural heritage. Digital acquisition of a cultural assets takes nowadays a couple of seconds thanks to the achievements in 2D and 3D acquisition technologies. However, enriching these cultural assets with labels and relevant metadata is still not fully automatized especially due to their nature and specificities. With the recent publication of several cultural heritage datasets, many researchers are tackling the challenge of effectively classifying and annotating digital heritage. The challenges that are often addressed are related to visual recognition and image classification. In this paper, we present a novel approach of hierarchical classification for cultural heritage assets. The metadata structural differences that exist between cultural assets motivated us to design a classification framework that can efficiently perform the classification of multiple types of assets. Our approach relies on several deep learning classifiers, each of them is assigned the task of classifying a certain type of assets. The classification framework starts the labeling process by first determining the asset type. The asset is then assigned to a specific classifier in order to be annotated with data fields related to its type. As a preliminary step, we successfully designed a general cultural type classifier and a specific type classifier for paintings. Our approach is currently achieving interesting results and is set to be improved by the integration of more asset types.