Postgraduate orthopedic training in Saudi Arabia: A need assessment for change

Introduction

Since its establishment in 1993, the Saudi Commission for Health Specialties (SCHS) has made significant improvements in postgraduate medical education in Saudi Arabia. The SCHS has structured accredited training programs based on the North American model.^[1] In 1996, a residency training program in orthopedic surgery was established by a group of educators who graduated from North American, European, and British training programs.^[1] The 5-year orthopedic residency program was designed by the SCHS with a complete set of standards for its delivery. Trainees are expected to achieve the same core learning objective during their training rotations regardless of the training hospital. However, the means by which these standards are to be applied and achieved are left to the discretion of the individual training program.

There are three main orthopedic training programs in Saudi Arabia, geographically distributed to provide orthopedic training across the country. The three programs are the Central region, the Eastern region, and the Western region programs. University Hospitals, Ministry of Health Hospitals (MOH), and Military Hospitals are three major health-care providers that provide residency training in each regional program. Orthopedic training is designed as a joint program based on 3-months rotations distributed regionally between different teaching centers according to SCHS accreditation standards. The regional program mandates for all trainees to rotate between different hospital categories within the region to ensure adequate exposure throughout their training.

There is a limited number of positions for the surgical residency training programs in Saudi Arabia, where some surgical programs can accommodate only 10% of applicants for training.^[1] Furthermore, there is a predictable increase in the number of medical graduate students after the recent establishment of more than 20 new medical schools since 1999.^[2] This mandate the expansion of current surgical training programs to accommodate the growing numbers of graduates to serve the growing population. These program expansions cannot be done appropriately without assessing the current programs and current performance that would help to direct development of future programs.

Appropriate assessment of the trainees' learning needs will help in planning, monitoring, and improving the educational process to ensure the relevance of the educational activities and the consistency of the training process.^[3],[4],[5] Literature addressing learning needs of surgical training in Saudi Arabia is limited. Some authors identified inadequate surgical skills training, theoretical teaching, and inadequate supervision as major challenges that need to be addressed.^[6],[7] Others recommended developing a new medical education provision, that is, high quality, cost-effective, and able to address the needs of learners and stakeholders.^[1],[8],[9] It is possible that learner needs assessment and continuous program evaluations may help to identify areas of improvement in the program and reduced dropout rate. Khairy^[10] emphasized the need for change in the surgical training process in Saudi Arabia using a survey as a need assessment tool to obtain feedback from trainees and their educators.

The main purpose of this study is to assess the trainees' learning needs and challenges in order to promote educational effectiveness, efficiency, and quality of orthopedic training programs. The primary objective was to identify trainees' perceptions of the content and adequacy of orthopedic training and its impact on the trainees' perceived satisfaction with the program in the acquisition of knowledge and skills. The secondary objective was to assess the consistency in the educational process of orthopedic training programs across Saudi Arabia.

Materials and Methods

An electronic questionnaire survey was conducted for all trainees in Saudi orthopedic training programs. The survey was designed to answer the primary question through comprehensive questionnaires to identify the trainees perceived needs. The results between different levels of trainees, different training programs, and different types of teaching centers were compared to assess the consistency in the educational process.

Survey design

We conducted a consensus survey of all orthopedic trainees (the postgraduate year “[PGY]” 1—5) in Saudi orthopedic training programs based on published guidelines.^{[11],[12],[13],[14]} We defined PGY1, 2, and 3 as junior trainees; while PGY4 and 5 were considered senior trainees. An English version of the survey, consisting of 36 items, was made available securely online and was structured with a combination categorical 5-point Likert-type scale. The questionnaire covered demographic data, learning objectives, theoretical and skills training, assessment and feedback, training center, and general training-related areas. We pretested the survey and obtained face and content validity by asking three orthopedic clinical fellows and three orthopedic educators to review the questionnaire and provide input and feedback. Subsequent changes to areas of concern and ambiguity were made accordingly. Further, the questionnaire was piloted on five orthopedic trainees to obtain approximate timing and transparency.

Implementation and data collection

Following the approval of the study by the Research Ethics Board and agreement from the orthopedic program from Saudi Arabia, a list of all trainees in orthopedic programs was obtained. We contacted the trainees electronically by sending direct invitations. After the first invitation, four reminders were sent on a weekly basis to improve the response rate, and then, no further invitations to nonresponders were made. The purpose of the study was disclosed clearly to the study participants in the cover letter before enrollment into the study. Participants' decision to complete the questionnaire was an indication of their consent to participate. After removing identifying information and replacing it with number coding for individuals and places to ensure their confidentiality, the raw data were used for statistical analyses.

Data analysis

Responses were collected online and converted to Microsoft Excel^® spreadsheet. We used descriptive statistics (i.e., frequency, percentage, mean, median, mode, and standard deviation) to summarize the results for all collected data as appropriate. Since the questionnaire was based on a 5-point Likert scoring system, the data for the survey were collected and analyzed categorically. These categorical variables were summarized as frequencies and percentages. We compared these variables between the groups in terms of region, level of training, and training center where appropriate using the Pearson Chi-square test for independent proportions. Linear regression analysis was used to compare orthopedic residency programs with regard to the percentage of cases done as a primary surgeon, level of satisfaction, and learning objective determination by the training program. Univariable and multivariable linear regression was performed to assess the influence of the included variables on the response variables: satisfaction, knowledge improvement, and skills improvement. These variables were considered as continuous responses. Based on the univariable linear regression, those variables with a statistical significance (associated P < 0.05) were included in the multiple linear regression with no selection method. A statistical software package (SAS, Version 9.2, SAS Institute, Cary, NC, USA) was used for all statistical analyses. All reported P values are two-tailed with an alpha of 0.05.

Results

Demographic data

Of the 104 trainees invited to participate, 71% (74/104) participants completed the survey. The demographic characteristics of the respondents are detailed in [Table - 1]. The response rates were 59% (32/54) from the Central region's program, 79% (23/29) from the Western region's program, and 90% (19/21) from the Eastern region's program. Across all programs, 78% (26/33) of senior trainees and 68% (48/71) of junior trainees responded to the survey. We categorized the responses into two groups based on the training level as 48 (65%) junior trainees and 26 (35%) senior trainees (P = 0.21).

Table 1: Demographic data

Learning objectives

Sixty-three percent of the respondents did not receive a standard set of rotation-specific learning objectives or come up with their own [Table - 2]. Of note, only 26% of the senior trainees showed that they set their own learning objectives before starting a new rotation; however, this did not reach statistical significance when compared to junior trainees (P > 0.5). On their reflection about their learning achievements at the end of each rotation, only 10% of the respondents agreed that their achievements met their expectations.

Table 2: Responses by the level of training

Knowledge and skills training

Most of the respondents perceived the suboptimal experience as a primary surgeon [Table - 2]. More interestingly, the reported number of surgeries performed as a primary surgeon by the senior trainees was not significantly different as compared with the junior trainees (P = 0.09), [Figure - 1]. However, a statistically significant difference was observed between regions, where the least surgical exposure as a primary surgeon was seen in the Western region's program (P < 0.01). Half of the respondents felt that technical skills training was not well structured, and a similar number of respondents faced competition in performing surgical cases most of the time, which was a constant finding across level and region of training (P > 0.05).

Figure 1: Number of cases as a primary surgeon compared between regions of training

Assessment and feedback methods

Only 20% of the respondents agreed that they received formal feedback from their supervisors at the end of their rotation, which was a consistent finding across the levels of training [Table - 2]. However, there was a significant difference between the regions, which showed that the Central region's trainees had the best chance to get formal or informal feedback (P = 0.02). Similarly, 30% of all trainees never had the chance to discuss their end-of-rotation evaluation, and this was significantly higher in the Western region's program, in which 57% of the respondents never had this chance (P < 0.01). More than 70% of the respondents agreed that there was no formal process by which they could give their feedback and reflection about the training center or their supervisors. Nevertheless, 90% of the participant believed that giving the trainees the opportunity to evaluate the training center or supervisors will improve their training process. On their self-reflection, only one-third of the trainees agreed that end-of-rotation evaluations reflect their performance and 45% of trainees felt that they have learned what they need to learn by the end of the rotation.

Training centers

Most of the trainees (64%) agreed that the best chance for them to perform the largest number of procedures as a primary surgeon was at MOH hospitals [Figure - 2]. This was an invariable finding across the level and region of training. In addition, exposure to well-structured surgical training was perceived more at MOH hospitals in the Western region's program as compared to university hospitals for their colleagues in Eastern region's program, while military hospitals were chosen by 44% of the participant in Central region's program. Although more than 50% of the respondents agreed that well-structured theoretical education was perceived mainly at the university hospitals, this was not observed in the Western region's program (p 0.0015). When asked in which training center the hospital service negatively interfered with their training, the responses were variable between the regions in which MOH and military hospitals were selected by half of the trainees in the Central and Eastern regions, respectively.

Figure 2: Result of training opportunity compared between training centers

General training related area

Trainees were asked to rate their exposure to different aspects of both orthopedics and related training. In examining general learning opportunities such as educational requirements, orientation, and its influence on the training process; fewer differences were noted in the perception of participants as compared between the level of training and region of training [Table - 2]. Most of the trainees agreed that they were not fully aware of their program curriculum design and educational requirements. When asked about the availability of educational recourses and materials, about half of the respondents agreed that they are easily available, where the best access was seen in Western region's program, but it did not reach statistical significance (P = 0.7). In addition, 50% of junior trainees said that they never had the orientation before starting their rotations, which was a constant finding across the training programs.

Only 10% of the respondents had the chance to do an elective rotation to address their learning needs. The chance for trainees to perform elective rotation was more in the Central and Eastern region's programs in which 15% of trainees had this chance as compared to 0% in the Western region (P = 0.016). Fifty percent of the respondents agreed that their academic half days are protected during their training. This was best seen in the Western region's program (P = 0.046).

About half of the trainees agreed that their knowledge and clinical performance improved at the end of each rotation. Of all participants, 20% (15/74) showed that they are satisfied with their training and 46% (34/74) demonstrated neutral response (neither satisfied nor dissatisfied) [Table - 2] and [Figure - 3]. The dissatisfied trainees 34% (25/74) were significantly higher as compared to satisfied trainees (15/74) (P = 0.001). The dissatisfaction was seen more in the Eastern region's program (50%); however, this did not reach statistical significance as compared to their colleagues in other regions (P = 0.31).

Figure 3: Satisfaction rate compared between level and region of training

All variables were tested in a univariable fashion using univariable linear regression to assess factors that have a significant impact on participants' training satisfaction. Among all the tested variables, we identified 14 variables associated with a P < 0.05. The other variables were not significant and were not candidates to be included in the multiple regression. Performing the multiple regression model with no selection method (all significant variables were included), the only variable that was independently significantly (P = 0.01) affecting satisfaction level was the amount of surgical exposure. Achievement of learning needs that were expected by trainees and improvement of surgical skills performance at the end of rotation were identified as independent factors that have an impact on acquisition of knowledge and skills.

Discussion

Throughout the world, there is a substantial need for change in surgical education to shift from process-oriented to outcome-oriented education.^[15] This study provides representative data describing the current status of orthopedic training in Saudi Arabia from the trainees' perspective that may help to design strategies for change. Program planning was identified as an essential element for effective teaching and learning.^[16] Therefore, our needs assessment results could be used as indicators of the training programs evaluation to set appropriate strategies to improve the training outcomes.

Learning objectives

As the concept of the outcome-based program is growing, setting clear, specific, measurable, and achievable learning objectives that address all aspects of training is essential in the training process. In this study, we observed significant lack of learning objectives set by training programs, training centers, or trainees themselves. Sixty-three percent of trainees never had a set of objectives that they should achieve by the end of their rotations, which was constant finding across training programs and level of training. Developing a clear set of learner-centered objectives could be used as an indicator to measure and monitor the training outcome.^[17],[18] It will further help in program evaluation to identify the educational achievement and detect educational gap if any. In addition, it will play an important role to ensure consistency in the training process. Moreover, teaching the trainees to develop their own objectives and create their learning contracts will promote their ability to focus their efforts to achieve the desired outcome.

Teaching and mentorship

We observed that supervision and mentorship had been identified as a concern of the trainees across the training programs. Similar findings have been observed in a large survey of orthopedic trainees in the US, where only 44% out of 506 respondents were satisfied with their mentoring environment.^[19] This finding may be attributed to increased clinical demands on the academic faculty and lack of incentives for training. It has been shown that the success of the educational process in residency training is a mutual responsibility of both the trainees and educators to generate a bidirectional exchange of benefits.^[20] Therefore, creating a safe educational environment and effective communication with a rational distribution of responsibilities based on a clear set of objectives for each rotation is essential for success. This will further help program directors in monitoring and evaluating the training process.

As the suboptimal theoretical and skills teaching raise to be a trainees' concern, trainee themselves could play an essential role in the education process. Developing teaching skills in trainees was found to be an effective educational tool in training program,^[21],[22] and senior trainees were identified by their peers in training programs as important teachers for them.^[23] Furthermore, developing teaching skills could play important roles in teaching values, communications, and professionalism.^[24] Therefore, the implementation of strategies to develop trainees' teaching skills will help to create more teaching opportunities in busy practice environments. This will enhance the capability of the trainees to work as professional surgeons and efficient teachers once they graduated. Finally, the implementation of incentive systems based on measurable academic productivity would play a role in increasing educators engagement in the training process and creating more teaching opportunities.^[25]

Surgical skills training

The development of essential psychomotor skills requires optimal surgical exposure of the trainees to enhance their competency to work independently on graduation. We identified suboptimal exposure to surgical cases as an independent predictor of trainees' dissatisfaction. Similarly, many national reports have identified insufficient surgical skills training as a major concern of trainees, which will reflect on their future surgical competency.^{[1],[6],[7],[10]} Similar challenges have been shown in needs assessment in developing and developed programs, where the trainees have suboptimal exposure as primary surgeons.^[26],[27] This phenomenon is attributed mainly to decreased surgical volume, concerns for patient safety, increasing medicolegal concerns, and increasing clinical demands on the academic faculty.^[28]

As few respondents perceived enough surgical exposure, more structural surgical skills education is required to improve trainees' satisfaction and training outcome. It has been shown that there is significant variability in surgical skills acquisition among trainees due to the difficulty of ensuring that all trainees have been exposed to the same core procedures.^[29] Inadequate exposure in specialized areas seems like a universal problem, and some programs have developed special training programs to address this.^[30] A set of core surgical procedures during each rotation that all trainees should accomplish by the end of the rotation is required to improve essential psychomotor skills acquisition and trainees' competency. Objective tools such as log book and portfolio may be used as a registry that measures trainees' surgical exposure, performance, and degree of supervision.^{[31],[32],[33],[34]} Although these tools will measure performance quantity rather than quality, it may help to monitor surgical exposure and ensure consistency of training. In addition, teaching with simulation is proposed as an essential element of surgical skills training to face challenges such as patient safety, operating room efficiency, and cost.^{[15],[35],[36],[37],[38]} Similar to other authors,^[1],[10] we emphasize the need to create simulation-training models to develop essential surgical skills in Saudi Arabian trainees. Developing surgical skills with simulation will prepare trainees for operating room, relief concern on patients' safety and efficiency of operating rooms to increase their opportunity to perform more cases.^[36],[37]

Training centers

The teaching centers that deliver the curriculum and provide training have significant diversity in their scopes and missions. This diversity is probably advantageous as it allows trainees to have exposure to a variety of cases and teaching experiences. However, it is challenging for the program directors to maintain close monitoring over the training process. We observed significant variability of perceived trainees' achievement during their rotations in different centers. Each training center should focus their efforts to maximize its educational potential to meet trainees' expectations. To ensure consistency, training centers should develop a set of rotation-specific objectives that is firmly linked to the general standards and requirements of training programs and adaptable to the trainees' needs. In addition, program planners should develop general objective measures that are acceptable and applicable to all training centers to help in the training process monitoring.

Assessment and feedback

We observed that few respondents agreed that their evaluations reflect their performance. In addition, most trainees perceived inadequate formal and informal feedback. Reliable, valid, and feasible assessment and evaluation system is a powerful learning tool, and it helps to increase the accuracy to measure trainees' performance if used appropriately. As a successful completion of rotation or training program is a major concern for trainees, trainees will figure out how they will be assessed and evaluated at the end of rotation or training program, and they will focus their efforts accordingly.^[18],[39] This would be an effective motivation for them to achieve the desired outcomes if the assessment system was aligned appropriately with the learning objectives. Constructive feedback is a powerful tool that will help trainees to identify opportunities for improvement.

Finally, most respondents perceived trainees' evaluation and feedback on their training centers and supervisors as a factor that will improve their performance. This may further be used by program' governors as an indicator to identify strengths and weaknesses and address them accordingly.^[18]

Study strength and weakness

This study identified the current status of orthopedic training in Saudi Arabia from the trainees' perspective. It will set the stage for future improvements in the curriculum, instruction, and assessment methods in Saudi orthopedic residency programs. We are confident that the findings of this study will stimulate discussions and buildup a solid ground for future research in this field. The results of this study will reinforce the strengths of training programs and help to promote quality and effectiveness of orthopedic residency education, thus improving the orthopedic profession in Saudi Arabia.

We acknowledge that our data represent trainees' perspective only. An extension of this study is needed to explore the needs from the perspectives of the educators or training centers. In addition, our study has limited generalizability as it represents the Saudi orthopedic training program. However, many other national or international developing programs share some of the concerns raised in this study and could benefit from our results and recommendations. A perfect program does not exist, and program evaluation is not about proving the success or failure of a program. The success of the program depends on being open to feedback and adjusting the program accordingly.^[40] Regular evaluation of the program by the trainees gives you this continuing feedback.

Conclusions and Recommendations

We identified certain areas of strengths in the orthopedic training programs that need to be enhanced and other opportunities for improvement that need to be addressed by program planners. Our findings may be used as important adjuncts in program evaluation for future planning to improve training process and ultimately the training outcome. More research is needed to gain more understanding in specific training areas such as surgical skills training, teaching method, and assessment. In addition, similar studies can be used to address the training needs of orthopedic educators and program directors' perspective aiming to optimize training and improve patient care.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

Authors' contributions

KHA conceptualized the idea, designing the questionnaire, conduct the research, data collection and organization, and drafted the first manuscript. SB designing the questionnaire, editing manuscripts, and critical review during the writing of the manuscript. AAA Data analysis and critical review during the writing of the manuscript. WT critical review during the writing of the manuscript. SR provided critical supervision during the writing of the manuscript, critical review of the final draft. All authors participated in the writing of and made essential contributions to this paper and critically reviewed and approved the final manuscript and are responsible for the content and similarity index.