DEVELOPING THE DETERMINANTS TO ASSESS THE AI LEARNER OUTCOMES

Abstract

Within the current educational milieu, incorporating Artificial Intelligence (AI) into secondary education is deemed imperative, serving as a requisite measure to equip students with fundamental aptitudes essential for navigating forthcoming societal and professional landscapes. This assertion stems from acknowledging the prevalent influence of AI technologies across diverse sectors, necessitating an adept workforce equipped with the requisite competencies to effectively engage with and harness the transformative potential of AI innovations. With its diverse and complex implications, the ever-evolving landscape of AI poses significant educational hurdles in thoroughly evaluating AI learner outcomes (AI-LO) of students, specifically high school learners. This challenge results in a notable scarcity of validated instruments tailored for assessing the proficiency of secondary students in AI education. The current study addresses this gap by developing determinants that can effectively gauge the AI-LO among secondary students subjected to formal AI curricula. Employing the Delphi technique, the study adopts a comprehensive and consensus-driven approach encompassing surveys, reviews, item generation, pilot studies, and psychometric analyses to discern these determinants. Through collaborative efforts, a panel of experts identified and refined the AI-LO scale with determinants across three overarching categories: AI foundations, Understanding and using AI, and Ethics of AI, encompassing requisite knowledge, skills, and values/attitudes essential for students to achieve AI education outcomes. Statistical techniques, like exploratory factor analysis (EFA), were employed to determine the assessment factors. The results demonstrated robust psychometric properties of the developed scale, providing evidence for its factorial structure, construct validity, and internal consistency. Within the AI foundations category, determinants encompass foundational concepts such as data literacy, programming, and algorithms. Understanding and using AI determinants focused on practical application skills, including AI techniques, technologies, and development, particularly in robotics. Finally, ethics of AI determinants address ethical considerations, responsible AI usage, and societal implications of AI technologies. The findings of this study have a broad impact on educators, researchers, and policymakers interested in improving students' interactions with AI technology within educational contexts. This research contributes to the progression of empirical inquiry into AI in education by providing a validated tool for assessing students' AI learner outcomes. This, in turn, enables evidence-based decision-making and promotes the development of AI interventions tailored to the needs of learners, thereby fostering a more student-centered approach to AI integration within educational settings.