Measuring students' AI competence: Development and validation of a multidimensional scale integrating educational psychology perspectives

View/ Open
Publisher version (Check access options)
Check access options
Date
2025Author
Ahmad, ZubairSultana, Almaas
Latheef, Nafilah Abdul
Siby, Nitha
Sellami, Abdellatif
Abbasi, Saddam Akber
...show more authors ...show less authors
Metadata
Show full item recordAbstract
The rapid integration of Artificial Intelligence (AI) into K-12 education has created a pressing need to evaluate students' capacity to engage with AI effectively and responsibly. As outlined in UNESCO's AI Competency Framework, AI competence is a multidimensional construct encompassing cognitive (knowledge and skills), affective (values and motivation), ethical (principles and responsibilities), and technical (practical and design-based) dimensions. Without systematic assessment, educators lack the evidence needed to ensure students possess not only technical proficiency but also the critical thinking, ethical awareness, and motivation required in AI-driven learning environments. To address this gap, the present study develops and validates the AI Competence Scale (AICS), a multidimensional instrument grounded in UNESCO's framework and educational psychology principles. The AICS conceptualizes AI competence across four interrelated dimensions-Human-Centred Mindset, Ethics of AI, AI Techniques and Applications, and AI System Design-capturing the interplay between students' knowledge, values, reasoning, and applied skills.Data were collected from 608 high school students in public secondary schools in Qatar, following a multi-stage process: expert reviews for content validity; exploratory and confirmatory factor analyses for construct validity; reliability testing via Cronbach's alpha and composite reliability; and convergent, discriminant, and nomological validity using AVE, HTMT ratios, and structural equation modeling (SEM). The results confirmed strong psychometric properties, with SEM revealing a significant positive association between AI competence and students' interest in AI-related careers. The validated AICS provides educators, researchers, and policymakers with a theoretically grounded and empirically robust tool for assessing students' readiness for ethical and informed engagement with AI, supporting curriculum design, targeted interventions, and evidence-based policy in K-12 AI education.
Collections
- Educational Research Center [122 items ]
- Mathematics, Statistics & Physics [810 items ]
- Research of Qatar University Young Scientists Center [216 items ]