Towards adaptive multimedia system for assisting children with Arabic learning difficulties

Abstract

Children with learning difficulties (LD) are increasing dramatically in the Arab world. Such children require quick intervention especially during the early childhood years. This paper presents a dynamic multimedia system for helping children with LD to overcome their learning problems. We introduce an approach to automatically convert modern standard Arabic children's stories to the finest representative images that can efficiently illustrate the meaning of words. Specifically, first, we apply natural language processing techniques to analyze the text in stories and we extract keywords of all characters and events in each sentence. Second, we apply an image captioning process through a pre-trained deep learning model for all retrieved images from our multimedia database as well as the Google search engine. Third, using sentence similarities, most significant images are retrieved back by selecting top highest similarity values. The proposed system aims to better enhance understanding, communications, and thinking skills for children with LD in elementary schools and special education centers.