HANDWRITING RECOGNITION IN ARABIC HISTORICAL MANUSCRIPTS

Abstract

Document Analysis and Recognition significantly impact humanitarian studies by revealing information hidden in historical document collections worldwide. This research area merges the sciences of computer vision and machine learning. This PhD dissertation aims at recognizing text in Arabic historical handwritten documents by learning and extracting visual representations inside these manuscripts. The proposed approaches presented in this dissertation have the primary purpose of creating effective systems to deal with challenges linked to Arabic handwriting recognition, particularly in ancient manuscripts with old handwriting. The use of Convolutional Neural Networks (CNNs) to tackle the Arabic handwriting recognition challenges is an integral part of this dissertation. Several architectures for developing high-performing features are suggested. A model based on CNN and Gated Recurrent Units (GRUs) is used to recognize a wide range of handwritten Arabic subwords extracted from historical documents. Because recent research has shown that typical CNNs' learning performance is limited as they are homogeneous networks with a simple (linear) neuron model, a further improvement in the handwriting recognition models using non-linear neuron models is implemented. Operational Neural Networks (ONNs) are recently proposed as heterogeneous networks with a non-linear neuron model. Even with compact architectures, they can learn highly complex and multi-modal functions. This PhD dissertation investigates the use of Self-Organized Operational Neural Networks (SelfONNs) for handwriting recognition and the generalization capabilities of non-linear neuron models, i.e., if deep discriminative features can be created. An investigation of an adequate level of non-linearity of the Self-ONN layers to provide extensive information on the Self-ONN performance under various topologies is presented. With such a novel approach, superior performance is achieved on a historical Arabic dataset and state-of-the-art performance is gained with a significant performance gap overall recent methods on an English dataset. Furthermore, a novel method for disambiguating undotted Arabic characters is presented. While the method is useful for handwriting recognition systems dealing with Arabic manuscripts with ancient undotted letters, it also improved the visual recognition performance on current Arabic handwritten documents with dotted and diacritized characters.