Perspectives in Arabian healthcare Structured Multi-Skill Assessment ( SMSA ) in pharmacy : A contextual adaptation for authentic assessment for colleges of pharmacy and beyond

There is a need for a contextual adaptation model of the Objective Structured Clinical Examination (OSCE) procedures and assessment methods into diverse contexts with great cultural diversity that is both affordable and achievable. This manuscript aims to describe the principles, organizational structure and theoretical framework of the Structured Multi-Skill Assessment (SMSA), a modified version of the OSCE and a performance-based assessment method developed at the Qatar University College of Pharmacy (QU CPH) that addresses contextual and cultural considerations when used in undergraduate pharmacy curricula. Cite this article as: Kheir N, Awaisu A, Ndoye A, Wilby KJ. Structured Multi-Skill Assessment (SMSA) in pharmacy: A contextual adaptation for authentic assessment for colleges of pharmacy and beyond, Avicenna 2015:5 http://dx.doi.org/10.5339/avi.2015.5 http://dx.doi.org/ 10.5339/avi.2015.5 Submitted: 29 March 2015 Accepted: 02 June 2015 a 2015 Kheir, Awaisu, Ndoye, Wilby, licensee Bloomsbury Qatar Foundation Journals. This is an open access article distributed under the terms of the Creative Commons Attribution license CC BY 4.0, which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. School of Pharmacy, Qatar University, Doha, Qatar *Email: nadirk@qu.edu.qa


INTRODUCTION
Formative assessments in health professions education traditionally rely on methods such as multiplechoice and essay questions. 1,2These traditional methods of assessment may not adequately evaluate essential clinical competence, professional skills and measure cognitive learning abilities. 1,2,3 disparity has also been reported between high performance achievers in the classroom and those in clinical settings. 2The use of performance-based assessment methods, such as the objective structured clinical examination (OSCE), chiefly provides optimal solutions to these issues and has been adapted as summative into health curricula worldwide.OSCE was originally introduced by Ronald M. Harden in the 1970's and was used in the assessment of clinical competence among medical students. 4,5Today OSCE has become a common performance-based method across many health professions in a variety of different settings and contexts.Similarly, there is widespread use and interest in using OSCEs within pharmacy education. 6Notably, OSCEs are a component of high-stake entry-to-practice licensing examinations in Canada (Pharmacy Examining Board of Canada Qualifying Examination Part II) and the USA 6 -10 , have been used to assess the clinical and counseling skills of undergraduate students in the UK, 11,12 and have been used in assessing undergraduate pharmacy students clinical competences in some developing countries. 13 -15he OSCE that was originally adapted from the medical profession focuses on assessing clinical skills, but there is limited literature providing guidance for contextual adaptation into diverse cultural settings. 16,17Simply adapting the traditional OSCE construct, including process and assessment components, may compromise validity in contexts differing from North American or European settings.For example, OSCEs are designed to assess patient-practitioner interactions, with a strong emphasis on communication skills, usually using checklists or global rating scales with rubric descriptors.These assessment methods commonly emphasize the importance of verbal and nonverbal communication (including facial expressions), demonstration of empathy, and interview organization and coherence.In the Middle Eastern setting (among others), however, interpretation of these components may significantly differ, as cultural and contextual communication norms vary greatly.A student communicating effectively in the Middle East (strong voice tone, deemphasized nonverbal communication) may be evaluated poorly according to traditional OSCE methods.This is especially true for female students who wear face coverings, which makes assessment of facial expressions and eye contact invalid.It is also our experience that similar contextual challenges exist within settings in Africa and Asia, including resource implications inhibiting effective and valid adaptation of performance-based assessments.Therefore, there seems to be a need for a contextual adaptation model of traditional OSCE procedures and assessment methods into diverse contexts with great cultural diversity that is both affordable and achievable.
This manuscript aims to describe the principles, organizational structure and theoretical framework of the Structured Multi-Skill Assessment (SMSA), a modified version of the OSCE and a performancebased assessment method invented at the Qatar University College of Pharmacy (QU CPH) that addresses contextual and cultural considerations when used in undergraduate pharmacy curricula.Principles discussed are relevant to a variety of health disciplines, including interprofessional education initiatives.It was also our objective to provide an affordable performance-based assessment model for educators working within resource-constrained settings.

THEORETICAL FRAMEWORK
Scholars have argued that authentic assessments are more appropriate for learners to develop and apply professional skills. 18 -20Further, authentic assessments allow students to integrate knowledge for higher order thinking and learning, especially in professional disciplines.Wiggins defines authentic assessment as assessment activities in which students must use knowledge to fashion performances effectively and creatively and where the tasks are either replicas of or analogous to the kinds of problems faced by adult citizens and consumers or professionals in the field.It appears from this definition that authentic assessment helps students deal with the complex issues and ambiguities of the real word by addressing multiple domains and competencies.Authentic assessments also allow learners to reflect on their practices and learning. 18ailored as an authentic assessment, the SMSA uses high impact educational practice principles that can promote the development of integrated skills and knowledge which are transferable and applicable to real life situations.Therefore, the SMSA aims at providing students an opportunity to develop integrated and multidimensional knowledge and skills that would allow them to address the different aspects of problems and issues encountered during their professional practices.The strength of an authentic assessment like the SMSA is that it does not only measure the correctness of a student's response, but also the thought processes involved in arriving at the response.This is supplemented with the post-evaluation time (usually few minutes) that is dedicated for feedback and dialog between the actor/patient, student, and the grading faculty member (or at times, practicing clinicians).
Another principle of authentic assessment is the involvement of the student as an active learner rather than a passive one.Further, the student's active participation in an assessment promotes higher level of awareness of expectations, which in turn can reinforce motivation and resourcefulness to learn. 21,22he SMSA requires students to be active participants in the process, and it focuses attention on how students arrive at their answers through observation, feedback, and in some cases, audio-or video-taping of the student while performing their activities.In that sense, it requires students to demonstrate the knowledge or skills needed to obtain a correct answer.The SMSA is also followed by a session dedicated to providing feedback.Typically, this is supported by a review of the taped recordings.
In the SMSA, a student is assessed against standard criteria decided a priori that demonstrate best practices.Hence, decisions about whether a person is competent are based on the evidence they provide to the assessor.In that sense, the SMSA is used as a tool to monitor and promote student learning.The SMSA is also participatory, where the students are aware of how and on what knowledge and skills are to be assessed.Assessment is therefore conceived of as both an evaluative tool and a learning activity.

HISTORY OF THE STRUCTURED MULTI-SKILL ASSESSMENT (SMSA) METHOD
As part of our teaching and assessment innovations at QU CPH, we developed a modified performancebased assessment technique that we called the Structured Multi-Skill Assessment (SMSA), which shares similar basic principles and mechanics with an OSCE, but designed to provide contextual validity when used as part of an undergraduate pharmacy curriculum.The SMSA is therefore the CPH's modified version of the traditional OSCE, used primarily for formative evaluation of student performance.It takes into account in its objective, development, and structure, the spirit of professional skills and competencies required of an undergraduate pharmacy student, competency and professional practice standards required of a pharmacy graduate in our local context, utilization of available limited resources, and the need for continuum of patient care activities as they occur in real-life.
The term SMSA was originally coined by Nadir Kheir, a professor of pharmacy practice at QU CPH, in his attempt to contextualize a pharmacy-oriented professional skill-based assessment technique that can evaluate pharmacy skills including patient counseling and education, data gathering and historytaking, drug therapy problem identification and resolution, and communication with patients and other health care providers.Both SMSA and OSCE are similar competency-based assessment methods and consist of standardized tasks that cover a number of competencies using objective and structured grading sheets.However, the SMSA has been adapted to our context by strict focus on providing feedback by multiple mechanisms (see below).Typically, stations are few (three to four) and replicate a progression of student skills from the first professional year to the final year.For example, students in the first professional year may complete four stations regarding the same patient case in progression from data gathering, developing a recommendation through utilization of resources, patient counseling, and health record documentation.This model allows students to focus on skill development with fewer time constraints and allows faculty to minimize resources by incorporating both interactive and non-interactive stations.Throughout the following professional years, however, complexity increases to cumulate with students completing independent, unrelated stations as part of their summative course assessments.
Contextual adaptation requires orientation for faculty but also for students.The SMSA was first introduced in the Spring Semester of 2007-2008 academic year to first professional year pharmacy students.Prior to its first use, an orientation session aimed at introducing the concept to the students to help familiarize them with the new assessment methodology was provided.From the second professional year onwards, the SMSA was offered once in each semester, with increasing complexity as the student's competency increases and the curriculum progressively becomes more advanced.In response to student and faculty feedback to improve the assessment process, a strategy of conducting a mock-SMSA was established as an orientation when introducing the SMSA to the first and second professional years.The mock-SMSA allows first year students to 'practice' the SMSA for the first time and receive immediate feedback by standardized patients (SPs) or/and faculty members.W Typically, standardized or simulated patients (or actors) are not expected to provide feedback after each student performs in a station W Typical, standardized or simulated patients (or actors) are required to provide supervised feedback to students when SMSA is used to supplement teaching Usage W Used in the context of both low-and high-stake examinations including entry-to-practice and re-licensure examinations 6 -12 W Used in the setting of low-stake assessments such as undergraduate students summative or formative assessments W When used in high-stake context (i.e.re-licensing examinations), the OSCE is highly labor-intensive and requires a significant investment in time and resources 7 -9,13 -16 W Because it is not usually intended for high-stake settings, the SMSA is less demanding and can be used in resource-limited environments Purpose W Primarily used as an assessment and licensing tool W Used for both teaching and assessment purposes

ADAPTATIONS FROM AN OSCE TO AN SMSA
Although SMSA and OSCE share many commonalities, some important differences exist between the two.Table 1 presents contextual adaptations found to be beneficial when developing and implementing the SMSA over multiple cyclic iterations.One adaptation deemed essential to our context is the provision of feedback by multiple sources for interactive stations.First, students receive informal feedback from assessors.This focuses on analytical performance, including positive and negative points regarding how the student solved the station.Next, and perhaps the most important contextual adaptation, is that the standardized patient provides feedback regarding the student's communication skills.We deem this an essential component of the SMSA process because the cultural diversity of Qatar challenges students to communicate with patients from a variety of cultural backgrounds.As such, Standardized Patients (SPs) s are chosen to reflect the resident population of Qatar (Arab, South Asian, Western, African, etc.).Lastly, students receive formal feedback in terms of grades and notes from station assessors (interactive stations) and evaluations from faculty on non-interactive stations.

STRUCTURE AND DEVELOPMENT OF AN SMSA
As the purpose of the SMSA is to assess the students' level of professional competency, it contains an adequate sample of the competencies and skills to be assessed.This is achieved through developing blueprint of the tasks performed (Figure 1).The blueprint includes the competency domains that are required to be assessed, the scenario within which the competency domain would be assessed, the specific skills to be assessed within the domain, and the station type (interactive or static).
The SMSA includes three types of stations: (1) Clinical/role station involving interaction between a student and a standardized patient or someone posing as healthcare provider; (2) Technical/practice station involving the performance of a technical procedure, often using a piece of equipment (e.g.prescription check, measuring blood pressure, teaching inhaler technique); (3) Static station which does not involve any physical interaction but may involve interpretation of data, searching for drug information, writing a care plan, or documentation within the health record.
Stations are developed using a rigorous process of case-writing and validation.A full list of requirements in each station is also developed (e.g.equipment; scripts for simulated patients; technical or administrative assistance).Each of the SMSA stations includes clear instructions to students, examiners and simulated/standardized patient.Table 2 provides a summary of typical SMSA stations.

DISCUSSION
It can be argued from the overview above that SMSA as implemented is in agreement with the principles of good authentic assessment practices.The SMSA addresses a multitude of competencies and skills that are all main components of, and prerequisites for, the provision of pharmaceutical care.These include a wide range of communication skills, consultation skills, interviewing skills, problemsolving and analytical skills, drug information provision skills, and care planning skills.In addressing multiple competencies, the SMSA supports most of the American Association for Higher Education's (AAHE) nine principles of good practices for assessing student learning. 23For example AAHE principle 2 states that "assessment is most effective when it reflects an understanding of learning as multidimensional, integrated . . .." (p.1).Further, the SMSA allows to integrate different skills and competencies learned in different courses, and therefore equips the student with an integrated and holistic approach to issues and problems at hand.In addition to involving students, members of an SMSA team support the respective professional skills course coordinator in developing the blueprint, designing stations, scheduling the activity, invigilating stations and carrying out the full spectrum of the SMSA activities.Through this team-based approach, the SMSA is also setting the stage for effective, holistic and durable curriculum changes that would be beneficial to students and faculty members.As in many universities, faculty members have diverse training and backgrounds, which warrants a team-based approach to ensure local validity of station development.As indicated in another AAHE principle "assessment fosters wider improvement when representatives from across the educational community are involved" (p.1).This dimension of the SMSA is also in line with another principle by Nicol who stated that good assessment encourages interaction and dialogue around learning (peer and teacher-student). 24It is through this dialogue that learners can receive high quality education which helps students learn best. 25Other scholars 26,27 have investigated the important role of feedback in the assessment and learning processes.
While no formal evaluation of the SMSA was conducted so far, the general feeling of faculty members and students alike is that the SMSA activity provides a platform for hands-on application and evaluation of skills that are best assessed through such a formative and practical method.The SMSA seems also to promote assessment for learning principles rather than assessment of learning.As indicated by Gavriel (2013), assessment for learning provides opportunities for dialogue between teachers and learners in order to promote deeper learning. 28The Assessment Reform Group (2002) defines assessment for learning as a "process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there" (p.3).In fact, by allowing students to receive feedback on the spot as well as means to reflect on their performances, the SMSA is a tool that promotes deep reflection on the learning.For example, and as indicated above, the SMSA places strong emphasis on the thought processes that lead to an answer rather than just focusing on whether a response is correct or wrong.The activity also provides course coordinators and faculty members involved in teaching students with an opportunity to discover areas of strength in the students' performance of simulated practice situations, and areas that require attention.
There remains important work to be done in respect of the SMSA at our college.For example, we need to explore, probably through qualitative data, the students' perceptions and attitudes towards this assessment method.There is also a need to develop a training program to be provided to SPs to ensure consistency in 'feedback'.Finally, we need to evaluate the effectiveness of the instant post-SMSA feedback strategy on student performance in subsequent SMSA activities, and on their summative OSCE assessments.

CONCLUSION
The SMSA is an authentic assessment adaptation of the OSCE and other performance-based assessment strategies that can be applied into other disciplines (both health-related and non-healthrelated).It takes into consideration in its development and structure, the spirit of professional skills and competencies required of an undergraduate pharmacy student, while utilizing minimal human and financial resources.

CONFLICT OF INTEREST
None to declare.

Table 1 .
Comparison between an Objective Structured Clinical Examination (OSCE) and a Structured Multi-Skill Assessment (SMSA).

Table 2 .
Components of a Typical Structured Multi-Skill Assessment (SMSA) Station.