Understanding Simple Stories through Concepts Extraction and Multimedia Elements

Abstract

Teaching children with intellectual disabilities is a challenging task. Instructors use different methodologies and techniques to introduce the concepts of lessons to the children, motivate them and keep them engaged. These methodologies include reading texts, showing pictures and images, touching items, taking children to sites to see and understand, and even using the taste and smell senses (i.e., hot, cold). The objective of this work is to develop a system that can assist the children with special needs to improve their understanding of simple stories related to the animals and foods domain through multimedia technology. We use formal concepts analysis and a simple ontology to extract the keywords representing characters, actions, and objects from the story text and link them with the corresponding multimedia elements (i.e., images, sounds and clips). These elements are retrieved by querying Google database using Google APIs features. The instructors would have to validate the obtained results and select what is appropriate for the children in the classroom. The system allows the instructor to input the story text and get as output the multimedia story.