Constructing and testing the psychometrics of an instrument to measure the attitudes, benefits, and threats associated with the use of Artificial Intelligence tools in higher education

Abstract

Under the acceleration in the body of information regarding AI technology and the paucity of instruments that assess the views and reactions of consumers, we have constructed this instrument to measure the attitudes, benefits, and threats (ABT) toward using Artificial Intelligence (AI) tools in higher education. Google Form was used in August of 2023 to collect data from students and teachers at higher education institutions in 11 Asian and African countries. After the ABT instrument obtained a sufficient score in content validity, additional statistical analyses were done. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed. This study included 503 participants who are familiar with AI tools. Over 56% have Bachelor’s degrees and 35% have Master’s or Doctoral degrees. The most popular AI tool was ChatGPT. One model out of six models created for the factor structure of the 35 items that measure attitudes, benefits, and threats was chosen. The selected model provides the highest explained variance (55.6%). The CFA, using AMOS software, demonstrated that the fit indices were satisfactory for the adopted model. Attitude (15), benefits (6), and threats (14 items) are the three factors of the model. The CFA supports the EFA with the ABT three-factor structure model. The high factor loadings and communalities suggest that the factors are reliable and valid measures of the attitude, benefits, and threats toward AI tools among highly educated personnel.