AUTOMATED DETECTION AND ASSESSMENT OF DEVELOPMENTAL DYSGRAPHIA: ADVANCED MACHINE LEARNING FRAMEWORKS AND MULTIMODAL ANALYSIS

Abstract

Dysgraphia, a learning disability affecting handwriting, presents significant challenges in diagnosis due to its complex interplay of cognitive, motor, and perceptual impairments. Existing automated approaches predominantly rely on tablet-based data, which effectively capture handwriting kinematics but do not fully reflect natural writing conditions or biomechanical factors. Moreover, current methods lack multimodal analysis, severity grading, and publicly available datasets, particularly for assessing dysgraphia severity. This research introduces a novel conceptual framework for automated dysgraphia assessment and support, integrating multimodal handwriting analysis with biomechanical assessment to enable a more comprehensive evaluation. While the study presents a holistic framework, it primarily focuses on advancing and validating key components, contributing novel methodologies that lay the foundation for future integration. The first contribution is an optimized online handwriting analysis method that reduces feature dimensionality (by 85%) while maintaining state-of-the-art accuracy. It incorporates both On-Surface and In-Air handwriting attributes, improving computational efficiency. The second contribution extends to offline handwriting analysis, utilizing deep transfer learning on the first publicly available handwriting image dataset for dysgraphia detection. This study also demonstrates the benefits of task-specific feature fusion in improving classification performance. Extending beyond single-modality approaches, thiswork introduces the first multimodal dysgraphia detection method, combining online handwriting with offline handwriting images using conditional feature fusion. This multimodal approach improves classification accuracy by 12-14% over unimodal methods, setting a new benchmark for dysgraphia detection. Additionally, we introduce the first publicly available dataset for dysgraphia severity grading, alongside novel meta-learning-based severity assessment algorithms that integrate online and offline handwriting data for more reliable severity tracking. Furthermore, we developed a hardware system integrating force-sensing resistors (FSRs) and surface electromyography (sEMG) sensors to assess grip patterns and muscle activation during handwriting. This novel system introduces grip force and muscle activation biomarkers for the first time in automated dysgraphia assessment, offering objective insights into handwriting-related motor patterns. Preliminary analysis on a limited dataset demonstrated 95% accuracy in classifying dysgraphia-related biomechanical patterns, suggesting its potential as a complementary assessment tool. By integrating multimodal handwriting analysis, novel severity grading techniques, and a novel biomechanical biomarker assessment, this research lays a strong foundation for advancing dysgraphia diagnosis and intervention.