The Design and Implementation of Survey Quality Control System for Qatar's First National Mental Health Survey: A Case Study

Abstract

All World Mental Health (WMH) surveys apply high standards of data quality. Most published quality control procedures for these surveys to date were in relation to face-to-face interviews. However, due to the social restrictions that emerged from the COVID-19 pandemic, telephone interviews are the most effective alternative for conducting complex probability based large-scale surveys. In this paper, we present the quality control system implemented in the World Mental Health Qatar (WMHQ) survey - the first WMH survey conducted during the COVID-19 pandemic in the Middle East. The objective of the quality control process was to acquire high data quality through reduction in random errors and bias in data collection. The QC design and procedures in this study were adapted to telephone survey mode in response to the COVID-19 pandemic. We focus on the design of the quality control indicator system and its implementation including investigation process, monitoring interviewers' performance during survey fielding, and applying quality-informed interventions. The study team investigated a total of 11,035 flags triggered during two waves of survey data collection. The most triggered flags were related to short question administration duration and multiple visits to the same survey questions or responses. Live monitoring of the interviews helped in understanding why certain duration-related flags were triggered and interviewing patterns of the interviewers. Corrective and preventive actions were taken against interviewers' behaviors based on the investigation of triggered flags per interviewer and live call monitoring of interviews. While in most cases, the interviewers required refresher training sessions and feedback for improving their performance, several interviewers discontinued work due to low productivity and high number of triggered flags. The specific QC procedures implemented in the course of the WMHQ survey were essential in successfully meeting the target number of interviews (N=5000). The QC strategies as well as the new indicators customized for the telephone interviews contributed to the flag investigation and verification process. The QC data presented here shed light on the rigorous methods and quality monitoring process in the course of conducting a large-scale national survey on a sensitive topic during the COVID-19 pandemic.