Evaluating the utility of ChatGPT in enhancing parental education and clinical support in hypospadias care

Abstract

IntroductionHypospadias is a prevalent congenital anomaly that requires effective parental education. Current online resources often exceed recommended readability levels, potentially hindering understanding. This study evaluates the utility of ChatGPT in providing accurate, clear, and actionable information towards parental education about hypospadias. MethodsA structured set of questions was posed to ChatGPT 4.0 covering diagnosis, treatment options, and postoperative care. Responses were quantitatively assessed using the Patient Education Material Assessment Tool for Printable Materials (PEMAT-P) to measure understandability and actionability. Qualitative evaluations were conducted by six pediatric urologists who rated the information for accuracy on a scale from 1 (completely accurate) to 4 (completely inaccurate). The Fleiss' Kappa statistic was calculated to assess inter-observer agreement among the urologists. ResultsThe quantitative assessment yielded understandability scores between 84 % and 92 % (average 88 %), while actionability scores ranged from 37 % to 70 % (average 51 %). In the qualitative assessment, 41 % of responses were deemed completely accurate, with 35 % considered accurate but inadequate, and 24 % rated as inaccurate. The overall Kappa value was 0.607, indicating substantial agreement among reviewers regarding the accuracy of the information provided by ChatGPT. ConclusionChatGPT can effectively convey information about hypospadias, but enhancing the actionability of its responses is crucial. Inaccuracy is still a main concern in using AI-generated search engine. Future updates should include more accurate and reliable responses and visual aids addition may support parents in navigating their child's care.