Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
Activity Report
Author’s Reply
Book Review
Case Report
Commentary
Current Issue
Editorial
Erratum
Guest Editor Profile
Letter to Editor
Letter to the Editor
Letters to Editor
Original Article
Protocol
Radiology Quiz
Review Article
Surgical Technique
Systematic Review Article
Technical Note
Technical Notes
Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Filter by Categories
Activity Report
Author’s Reply
Book Review
Case Report
Commentary
Current Issue
Editorial
Erratum
Guest Editor Profile
Letter to Editor
Letter to the Editor
Letters to Editor
Original Article
Protocol
Radiology Quiz
Review Article
Surgical Technique
Systematic Review Article
Technical Note
Technical Notes
View/Download PDF

Translate this page into:

Original Article
ARTICLE IN PRESS
doi:
10.25259/JMSR_118_2022

The orthopedic surgery preparatory exam course: Measuring residents’ satisfaction and perceptions through video ethnomethodology and survey analysis

Department of Orthopedic Surgery, King Abdulaziz Medical City, Jeddah, Saudi Arabia
Department of Medical education, College of Public Health and Health Informatics, King Saud bin Abdulaziz University for Health Sciences and King Abdullah International Medical Research Center, Jeddah, Saudi Arabia
Department of Medical Education, King Saud bin Abdulaziz University for Health Sciences, Jeddah, Saudi Arabia
Corresponding author: Mohammed M. Almutairi, Naif Road, Jeddah, Makkah, Saudi Arabia. mohmmad99955@gmail.com
Licence
This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-Share Alike 4.0 License, which allows others to remix, transform, and build upon the work non-commercially, as long as the author is credited and the new creations are licensed under the identical terms.

How to cite this article: Batouk O, Justinia TE, Almutairi MM, Omair A. The orthopedic surgery preparatory exam course: Measuring residents’ satisfaction and perceptions through video ethnomethodology and survey analysis. J Musculoskelet Surg Res, doi: 10.25259/JMSR_118_2022

Abstract

Objectives:

Orthopedic training in Saudi Arabia has changed how residents are assessed for their board examination. The new format uses scenario cases instead of real patients during the assessment. However, these changes were not received well by the residents. The Orthopedic Surgery Preparatory Exam Course is a 3-day training course designed specifically to address these concerns. This study investigated the effectiveness, perceptions, and satisfaction levels of the residents who took this course, emphasizing the video recording approach.

Methods:

A quantitative survey was designed by the research team and validated by experts. Edits were made after their feedback. Purposive sampling was used, and all 12 attendees of the course participated. All participants were 5th-year residents enrolled in the orthopedics residency training program in Saudi Arabia. A qualitative ethnomethodology analysis of video recordings for pre- and post-mock exams was also conducted.

Results:

The analysis demonstrated that participants were less anxious in the post-mock exam and showed improvement in their confidence and communication skills based on analysis of the assessed criteria such as eye contact, vocal projection, gestures, posture, and movement. The participants generally perceived the course useful in preparing for their board exam.

Conclusion:

The participants generally perceived the course to be useful in helping them to prepare for their board exam by familiarizing them with the exam environment. The participants reported the strength of the course as being able to provide a realistic simulated experience. On the other hand, the participants viewed the weakness of the course in its duration and generally requested more days to be added to the course. They were generally satisfied with the course helping them prepare for the board exam. It is recommended to incorporate the course as part of the residency training program curriculum.

Keywords

Board exam
Ethnomethodology
Perception
Preparation
Orthopedic course
Orthopedic surgery preparatory exam course exam
Residents’ satisfaction

INTRODUCTION

Saudi orthopedic training in Saudi Arabia has changed the way board-eligible residents are assessed and evaluated. At present, the delivery of the board exam has shifted to using scenario cases instead of real patients during the assessment. There were some indications that this change was not received well by the residents. From our work with residents, we heard repeated verbal reports on their experience with the new method of using scenario examples in the exam. They reported their experience of the scenario cases as somewhat odd and generally preferred using the previously used real cases method, which they were trained on in medical school time. The residents also (verbally) expressed their unfamiliarity with the new setup and expressed a need to improve the delivery of scenario cases and a need to be oriented with this new approach, preferably through an unbiased structured course. Unfortunately, there are very few courses that are conducted to address orthopedic board exam preparation concerns specifically. One example of a course that is designed to address these issues is the Canadian Orthopedic Forum in Calgary, Canada. Another example is the course studied in the paper; the Orthopedic Surgery Preparatory Exam Course (OSPEC), which is currently conducted in Saudi Arabia.

It is known that assessment drives learning to a great extent.[1] To improve teaching and learning, one option is the use of exams. The preparatory exam courses have proven to significantly improve exam scores by addressing exam concerns.[2] A study by de Virgilis et al.[3] has shown that reading assignments on a weekly basis conjoined with weekly preparatory examinations, improved overall American Board of Surgery In-Training Examination (ABSITE) scores. Godellas and Huang[4] concluded that various factors participate to residents’ successful performance. These include the amount of study, previous performance in similar exams, probationary status, amount of sleep, and conference attendance. These factors improve the examination and knowledge application performance, both during residency and throughout their career. The OSPEC is considered a “previous performance” experience for students attending the course.

In a comprehensive review, Klein et al. concluded that the performance on the Orthopedic In-Training Examination (OITE) was proven to be highly projective of achieving well on the American Board of Orthopedic Surgery Part 1 Examination (ABOS1). The study examined the residents’ performance on OITE, and the results were similar to the ABOS1 that the residents had undertaken.[5] The OSPEC reinforces the acquired knowledge while pinpointing the deficiencies found in the examinee. The course examined both knowledge and attitudes. It was a 3-day course during which knowledge content was delivered and assessed by a mock exam, and other skills were assessed as well. Derossis et al. concluded that the amount of dedicated study and conference attendance correlated significantly with resident performance and, thus, success rate.[6]

Ethnomethodology research is the study of the everyday practice people engage in for the procreation of social order.[7] Ethnomethodology’s research focuses on documenting the methodic practices through which society’s members produce their world as sensible. Unlike other forms of quantitative and qualitative research paradigms, ethnomethodology does not provide an exact “method.” There is no clear-cut set of procedures, although this approach involves numerous critical analytical processes, which have been used in other empirical studies.[7] One confusing issue for newcomers to ethnomethodology is that it lacks a formally stated theory and a formal methodology.[8] These issues never prevented ethnomethodologists from doing ethnomethodological studies and bringing important literature of “findings.”[7] David Scott has noted the “off-stage role of theory,” in ethnomethodological writings, “the concern that there is nowhere in the ethnomethodological corpus a systematic theoretical statement that would serve as a touchstone for ethnomethodological inquiries.”[8,9]

The current study aimed to investigate the perceptions of residents and their satisfaction with the OSPEC course and to analyze the strengths and weaknesses of this course.

MATERIALS AND METHODS

The OSPEC was delivered through presentations, focus groups, and video-recorded pre- and post-course mock exams with professional feedback. The residents who attended this course went through many educational session cycles. This included the simulated (mock) examination room for each participant. Throughout the session cycles, each participant’s focus was directed toward case-relevant knowledge and skills related to the examination, including the exam environment, tips, and the communication skills required while conducting mock exam cycles. Each resident had one mock exam on the 1st day of the course (pre), then expert feedback on every subject’s performance. On the 3rd day of the course, each resident had another mock exam (post) and received expert feedback (from communication skills experts and other mock examiners).

All course activities were video recorded as part of the course process. Recording the course events was an important part of the educational process for the course attendees. During the OSPEC course, every effort was made to simulate a real exam experience by providing an environment that is as close to the real exam environment as possible and a professional atmosphere to the course. The study was conducted in Jeddah, Saudi Arabia, after preparing a setup similar to actual board examination rooms.

The subjects selected in this study were board-eligible residents (i.e., the subjects have finished their training in the residency program). They were 5th-year residents (R5) from all the regions of Saudi Arabia. This course was done in Jeddah over 3 full days, starting at 0800–1700 h. All of the residents who participated in this OSPEC course later sat the Saudi board exam after they attended the OSPEC course.

The subjects were aware of the study before it was conducted, and they were given an information sheet [Appendix 1] about the purpose of the study and why they were selected for it. In addition, the subjects were informed of their option to withdraw from, or not participate in the study. After the course, they were consented to and invited to fill up the web-based questionnaire [Appendix 2]. This was sent to the subjects’ e-mails after the date of the actual board exam.

Sampling

Due to the specific purpose of this research, sample selection was made through purposive sampling. Therefore, this study included only R5 residents enrolled in the orthopedics residency training program in Saudi Arabia (Saudi Council for Health Specialties) and were “Board-eligible.” The chosen method was a mixed methods approach, utilizing both quantitative and qualitative methods.

Quantitative methods

Questionnaires

A survey was distributed among all subjects after the course was over and after their actual exam. The web-based questions were about whether the OSPEC course matched their expectations. Do they think this course would help them to conduct their final exam?

Do they feel that the OSPEC will help them to achieve their goal?

Out of all the course attendees, 12 residents qualified for the criteria, and all twelve participated in the study. All subjects were preparing to sit for their real final exam 3 months after the course date. It was not possible to interview any other participants for this study because it was done for a specific course that was conducted at a particular time, and these were the only available participants. Therefore, the findings of the qualitative portion of this study are intended to complement the findings of the quantitative portion.

The participants received a survey distributed in English after the course was over and after taking their final exam. This electronic survey questionnaire [Appendix 3] was sent to their e-mail addresses that they provided on their signed informed consent. The response rate was 100%. The sample is considered relatively low for a quantitative study. However, all subjects participated and represented 100% of the population since this study targeted a very specific and limited group of residents, and it was impossible to obtain a larger sample.

Qualitative methods

Ethnomethodological analysis of video recordings

The study was conducted in a hotel auditorium in Jeddah. Measures were taken to ensure that the course and study setup was appropriate. The setup included creating a special environment for the subjects to mimic their final board examination rooms. In addition, all activities of the course, including mock exams, were video recorded by professionals with high-tech recording audio and video equipment.

A qualitative method was used, which is widely accepted in medical education.[10,11] Qualitative analysis of the recorded videos was conducted using an ethnomethodology approach.

The mock exams

All 12 residents had two mock exams during the course, which were video recorded. After each exam, the candidate received feedback from the experts about his performance and the feedback was recorded too. Each mock exam lasted for 15 min for each subject. After the mock exams, the expert feedback was given to each subject and lasted for 10 min (25 min for each mock exam and expert feedback).

At the end of the 3-day course: Recordings were collected for the following:

Video recordings day 1

  • Mock examinations (15 min for 12 candidates = 180 min)

  • Experts’ feedback (10 min for 12 candidates =120 min)

  • Total recorded videos from Day 1= 300 min.

Video recordings day 3

  • Mock examinations (15 min for 12 candidates = 180 min)

  • Experts’ feedback (10 min for 12 candidates = 120 min)

  • Total recorded videos from Day 3 = 300 min

This study’s total time of qualitative video recordings was 10 h, which is a rich sample.

Comparison checklists

A checklist [Appendix 3] was designed by an unbiased communication skills expert (consultant physician) who is not a member of the research team. The purpose of the checklist is to aid the ethnomethodological analysis of the video recordings to compare each participant’s recordings on two different occasions (the 1st and 3rd day). The purpose was to examine the effectiveness of the course by checking for any improvements between the 2 days. The focus here was to examine and pay attention to subjects’ behavior throughout the examination period and non-verbal communication. The focus was not on the accuracy of their scientific answers.

This was achieved by observing and analyzing the videotape recordings and taking notes of comments while the occurrences were going on (live feedback).

The checklist contains points about (nonverbal communication skills like eye-to-eye contact, nervousness, and body language, and verbal communication skills like tone, fluency, and approach to the case).

The feedback was given by the course experts or faculty, including the communication skills expert, as part of the qualitative portion of the study. They evaluated the candidates’ performance on two different occasions (pre- and post-mock exams) and gave verbal feedback and advice on each occasion.

Validity

Questionnaires

Suitably validated questionnaires were not found for the purpose of the study. The questionnaire was designed by the research team and validated by experts who have Ph.D. in Medical Education, and edits were made after their feedback. The study questionnaire was tested by experts, and edits were made after feedback. After the revisions, the questionnaire was piloted with a group of five orthopedic surgeons. No further changes were required after the pilot.

Recordings

Expert audio-visual technicians accurately recorded all course events using very advanced recording equipment. Regular, timely checks to ensure the quality of recordings were done throughout the course events.

Video analysis

A checklist was designed specifically for the ethnomethodology analysis of the video recordings. The purpose was to examine the differences between before and after the recorded mock exams.

Evaluation

The checklists were analyzed by the principal investigator and were validated by a co-investigator who has a Ph.D. in Health Informatics and is an expert in qualitative research methods.

RESULTS

Quantitative

Questionnaires were sent to twelve participants after the oral exam. The survey results by the twelve respondents (100% response rate) were tabulated in graphic format. The followings are the 12 respondent’s answers for every question received [Table 1].

Table 1:: Participant’s answers.
Question Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Rating average Response count
The OSPEC course was useful and helped me prepare for my exam 66.7% (8) 25% (3) 8.3% (1) 0.0% 0.0% 1.42 12
The OSPEC course mock exam scenario cases were constructed to resemble real-life cases 83.3% (10) 8.3% (1) 8.3% (1) 0.0% 0.0% 1.25 12
The OSPEC course helped me to quickly recall different clinical or surgical techniques to give the most appropriate treatment for different clinical scenarios 50% (6) 33.3% (4) 16.7% (2) 0.0% 0.0% 1.67 12
Recording the mock exam through video surveillance helped me to reflect more deeply on my performance 66.7% (8) 16.7% (2) 16.7% (2) 0.0% 0.0% 2.17 12
The mock exam experience made me feel like I was on the hot seat and felt real pressure while experts were examining me 66.7% (8) 16.7% (2) 16.7% (2) 0.0% 0.0% 1.50 12
I benefited from the tips and presentations by the course director 58.3% (7) 33.4% (4) 8.3% (1) 0.0% 0.0% 1.50 12
I felt an improvement in my performance when I did the second-round mock exam compared to the first-round mock exam 75% (9) 16.7% (2) 8.3% (1) 0.0% 0.0% 1.30 12
The round-table discussions gave me in-depth scientific knowledge of a wide variety of scenario cases 33.4% (4) 41.7% (5) 25% (3) 0.0% 0.0% 1.92 12
The facility (venue) provided for the course was comfortable and suitable for this course (comfort, seating, space, AC, accessibility, and technical/audio-visual equipment) 66.7% (8) 25% (3) 8.3% (1) 0.0% 0.0% 1.42 12
Having attended the board exam, I believe that the OSPEC helped improve my communication skills and conduct during the exam 58.3% (7) 33.4% (4) 8.3% (1) 0.0% 0.0% 1.50 12

OSPEC: Orthopedic surgery preparatory exam course

Qualitative

The analysis of the videotapes and recordings of four attendees was performed, and it was decided to analyze only four because theoretical saturation was reached, and it is a normal practice in qualitative methods to sample until theoretical saturation is achieved.[12] The early observation of the few numbers of the participants’ behaviors in their preand post-tests with the collection of the expert and faculty panel opinions reflect the good and fair perception of the participants towards the OSPEC activities. The evidence showed that participants had a generally positive experience with the course. The analysis demonstrated that participants were less anxious in the post-mock exam and showed improvement in their confidence and communication skills based on analysis of the assessed criteria such as eye contact, vocal projection, gestures, posture, and movement [Table 2].

Table 2:: Participant’s performance.
Participant 1 Participant 2 Participant 3 Participant 4
Mock 1 Mock 2 Mock 1 Mock 2 Mock 1 Mock 2 Mock 1 Mock 2
Content
Maximum: 4 3 (75%) 3 (75%) 3 (75%) 3 (75%) 3 (75%) 3 (75%) 3 (75%) 3 (75%)
Arrangement of Ideas
Maximum: 9 7 (77.8%) 9 (100%) 7 (77.8%) 9 (100%) 8 (88.9%) 9 (100%) 8 (88.9%) 9 (100%)
Voice projection
Maximum: 4 2 (50%) 4 (100%) 2 (50%) 3 (75%) 3 (75%) 4 (100%) 3 (75%) 3 (75%)
Vocal delivery
Maximum: 4 3 (75%) 4 (100%) 2 (50%) 3 (75%) 4 (100%) 4 (100%) 3 (75%) 3 (75%)
Eye contact
Maximum: 4 3 (75%) 4 (100%) 3 (75%) 3 (75%) 3 (75%) 4 (100%) 3 (75%) 3 (75%)
Gestures
Maximum: 4 2 (50%) 4 (100%) 2 (50%) 3 (75%) 2 (50%) 3 (75%) 2 (50%) 3 (75%)
Pace
Maximum: 4 2 (50%) 4 (100%) 2 (50%) 3 (75%) 2 (50%) 3 (75%) 2 (50%) 3 (75%)
Clarity of expression
Maximum: 4 2 (50%) 3 (75%) 3 (75%) 3 (75%) 3 (75%) 3 (75%) 2 (50%) 3 (75%)
Listening
Maximum: 4 2 (50%) 4 (100%) 3 (75%) 3 (75%) 2 (50%) 3 (75%) 2 (50%) 4 (100%)

DISCUSSION

The study investigated the orthopedic residents’ perception and satisfaction with the OSPEC course as a structured study plan prior to the exam. Durak and his associates have found that instructive case-based exams followed by case discussions seemed to be a high-potential and motivating teaching tool for 6th-year students in the clinical problem-solving domain.[1] Within the OSPEC, mock exams are conducted, followed by expert discussion and feedback.

Miyamoto et al.[13] showed that several factors, including structured study habits and the use of specific study materials, contribute to residents’ successful OITE performance. They concluded that the adaptation of these findings by current orthopedic residents might positively impact OITE performance. Furthermore, Barnes concluded in her study that the oral examination represents an additional tool that may contribute to developing a student’s connection between knowledge, effective communication, and pharmacy practice.[14]

Booth TC et al. concluded[15] that “Efforts to help reduce anxiety levels in the oral component would improve perceived fairness.” One of the main aims of the OSPEC is to reduce the candidates’ anxiety levels, which was addressed in the study.

As Pletch et al. highlighted the importance of having a structured study plan, as residents might have different studying attributes, which might hinder their preparation for the ABSITE.[16] These findings are in accordance with a study published in 2013, which found that residents who emphasized self-assessment examination before OITE as a preparation tool had high scores, unlike their peers.[14] De Virgilio and Stabile concluded that assigning residents weekly reading assignments and assessments improved the mean ABSITE test scores compared to the previous years.[3] Using practical techniques in medical education like digital transformation, virtual reality, augmented reality, mixed reality, and extended reality, and three-dimensional medical images would maximize the effectiveness of patient care in the field of surgery while maintaining the educational needs of residents.[17-19]

The OSPEC seemed to be of benefit, as verbal reports from orthopedics residents were repeatedly received each time after the OSPEC course was conducted. They generally portrayed good feelings and positive perceptions about the course. They voiced overall satisfaction with the course content, its procedures and how it positively affected their performance during their real examination afterward. However, these perceptions have not been documented through previous empirical research. Such feedback and our experience with the OSPEC course over 3 years gives some indication that such a course can positively impact resident training and their confidence; and warrants further research.

Most participants shared their gratitude for integrating the course as part of their training in orthopedics. The results showed that the OSPEC course was more of a constructed plan in preparing for the board exam. The evidence also showed that the critique repeatedly received following the course was of great help in utilizing and enhancing the knowledge and skills they acquired while addressing the deficiencies they might have exhibited during the course.

Limitations

A validated questionnaire was not found, and the researcher had to design the questionnaire specifically for this study. The questionnaire was distributed to the participants after the course and the actual exam, which was about 10 weeks. The gap in time would have affected their memory collection of the events. Another limitation is that the evaluation was not performed by an independent evaluator.

CONCLUSION

The participants generally perceived the course to be useful in helping them to prepare for their board exam by familiarizing them with the exam environment. They were generally satisfied with the course. The participants reported the strength of the course as being able to provide a realistic simulated experience. On the other hand, the participants viewed the weakness of the course in its duration and generally requested more days to be added to the course.

Recommendation

Based on the findings and feedback from participants, it is recommended to incorporate the course as part of the residency training program curriculum of orthopedics. It would also be useful to introduce similar courses to other residency training programs. Early exposure of junior residents to a board exam environment during their residency training is also advisable to familiarize them with the exam early on, as participant’s demonstrated resilience and confidence in their exams following the course.

AUTHORS’ CONTRIBUTIONS

OB and TJ conceived and designed the study. OB, TJ, and AO conducted research, provided research materials, collected and organized data, and wrote the initial draft. MA and TJ wrote the final draft of the article and provided logistic support. All authors have critically reviewed and approved the final draft and are responsible for the manuscript’s content and similarity index.

DECLARATION OF PARTICIPANT CONSENT

The authors certify that they have obtained all appropriate participant consent forms. In the form, the participants have given their consent for their images and other clinical information to be reported in the journal. The participants understand that their names and initials will not be published, and due efforts will be made to conceal their identities, but anonymity cannot be guaranteed.

CONFLICTS OF INTEREST

There are no conflicting relationships or activities.

FINANCIAL SUPPORT AND SPONSORSHIP

This study did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References

  1. , , , . Use of case-based exams as an instructional teaching tool to teach clinical reasoning. Med Teach. 2007;29:e170-4.
    [CrossRef] [PubMed] [Google Scholar]
  2. , , . Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22:962-8.
    [CrossRef] [PubMed] [Google Scholar]
  3. , . Weekly reading assignments and examinations result in sustained improvement in American board of surgery in-training examination (ABSITE) scores. Am Surg. 2005;71:830-2.
    [CrossRef] [PubMed] [Google Scholar]
  4. , . Factors affecting performance on the American board of surgery in-training examination. Am J Surg. 2001;181:294-6.
    [CrossRef] [PubMed] [Google Scholar]
  5. , , , , . Passing the boards: Can USMLE and orthopaedic in-training examination scores predict passage of the ABOS Part-I examination? J Bone Joint Surg Am. 2004;86:1092-5.
    [CrossRef] [Google Scholar]
  6. , , , , . Study habits of surgery residents and performance on American board of surgery in-training examinations. Am J Surg. 2004;188:230-6.
    [CrossRef] [PubMed] [Google Scholar]
  7. , , , , , . General surgery resident use of electronic resources: 15 minutes a day. J Am Coll Surg. 2020;230:442-8.
    [CrossRef] [PubMed] [Google Scholar]
  8. , , . Pharmacy students' perceptions of their preparedness to provide pharmaceutical care. Am J Pharm Educ. 2010;74:8.
    [CrossRef] [PubMed] [Google Scholar]
  9. , , . Qualitative research methods in medical education. Anesthesiology. 2019;131:14-22.
    [CrossRef] [PubMed] [Google Scholar]
  10. , . Introducing medical educators to qualitative study design: Twelve tips from inception to completion. Med Teach. 2016;38:456-63.
    [CrossRef] [PubMed] [Google Scholar]
  11. , , , . Qualitative research essentials for medical education. Singapore Med J. 2018;59:662-7.
    [CrossRef] [PubMed] [Google Scholar]
  12. Qualitative Research: Theory, method and practice David silverman qualitative research: Theory, method and practice sage 390 21.99 0761949348 0761949348 [Formula: See text] Nurse Res. 2005;12:92-3.
    [CrossRef] [PubMed] [Google Scholar]
  13. , , , . Orthopedic surgery residents' study habits and performance on the orthopedic in-training examination. Am J Orthop (Belle Mead NJ). 2007;36:E185-8.
    [Google Scholar]
  14. . Ethnomethodology as science. Soc Stud Sci. 1985;15:751-62.
    [CrossRef] [Google Scholar]
  15. , , , , . The fellowship of the royal college of radiologists (FRCR) examination: A review of the evidence. Clin Radiol. 2018;73:992-8.
    [CrossRef] [PubMed] [Google Scholar]
  16. , , , , . Study strategies for general surgery residents preparing for the American board of surgery in-training examination: What to keep, discard, and adopt. J Surg Educ. 2021;78:43-9.
    [CrossRef] [PubMed] [Google Scholar]
  17. , , , , , , et al. Digital transformation will change medical education and rehabilitation in spine surgery. Medicina (Kaunas). 2022;58:508.
    [CrossRef] [PubMed] [Google Scholar]
  18. , , , , , , et al. XR (Extended reality: Virtual reality, augmented reality, mixed reality) technology in spine medicine: Status quo and quo vadis. J Clin Med. 2022;11:470.
    [CrossRef] [PubMed] [Google Scholar]
  19. , , , , , . Augmented and virtual reality in surgery-the digital surgical environment: Applications, limitations and legal pitfalls. Ann Transl Med. 2016;4:454.
    [CrossRef] [PubMed] [Google Scholar]

Appendix 1

PARTICIPANT INFORMATION SHEET

Study title

The Orthopedic Surgery Preparatory Exam Course OSPEC): Measuring effectiveness, residents’ satisfaction and perceptions through video and survey analysis.

Invitation

You are invited to take part in a research study. Please take time to read the following information. You are free to inquire about anything that is not clear to you, and you are welcome to ask for more information about the study. Please take time to consider and decide whether you wish to take part or not.

The purpose of the study

The study is a requirement for a master’s degree in Medical Education. The aim of this study is to investigate the residents’ perceptions and their satisfaction with the OSPEC course and to analyze the strengths and weaknesses of this course.

Participant selection

You have been selected and asked to participate in this study because you are involved in some capacity with the OSPEC course.

It is entirely up to you to decide whether you wish to take part or not. If you do, you will be given this information sheet to keep and be asked to sign and return a consent form. You are still free to withdraw from participation at any time without giving a reason.

Focus Group

Group discussions will be held during the course. The comments will be audio recorded and then used by the researchers in a focus group context in the study.

By signing the consent form, you agree to use any of your video or audio recordings for the study.

Observation

Some examinee activities during the OSPEC course will be video recorded (like the mock exams). These videos will be analyzed and used in the study only if you agree to participate.

Some of the feedback given to the examinees and the comments on their mock examinations will be audio recorded. Professional advice will be given to the examinees. This will also be audio recorded. This feedback will be used and analyzed in the study if you agree to participate.

By signing the consent form, you agree to use any of your video or audio recordings for the study.

Questionnaire

After the course is over, you will receive a questionnaire in your e-mail (please print down your e-mail address in the consent form). Please fill it out and send it back to my e-mail address.

Appendix 2

QUESTIONNAIRE

The Orthopedic Surgery Preparatory Exam Course (OSPEC): Measuring effectiveness, residents’ satisfaction and perceptions through video ethnomethodology and survey analysis

Section I: Choose one option for each item:

1. The OSPEC course was useful and helped me to prepare for my Board Exam.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

2. The OSPEC course mock exam scenario cases were constructed to resemble real-life cases.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

3. The OSPEC course helped me to quickly recall different clinical or surgical techniques to give the most appropriate treatment for different clinical scenarios.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

4. Recording the mock exam via video surveillance helped me to reflect more deeply on my performance.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

5. The mock exam experience made me feel like I was in the “hot seat” and felt real pressure while I was examined by the experts

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

6. I benefitted from the tips and presentations by the course director and the communication skills expert.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

7. I felt an improvement in my performance when I did the second-round mock exam compared to the first-round mock exam.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

8. The Round-table discussions gave me in-depth scientific knowledge of a wide variety of scenario cases.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

9. The facility (venue) provided for the course was comfortable and suitable for the purpose of this course (comfort, seating, space, AC, accessibility, and technical/audio-visual equipment).

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

10. The OSPEC course improved my communication skills and my professional conduct during the exam

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

11. Having attended my Board Exam, I believe that the OSPEC course helped to improve my communication skills and conduct during the exam.

[ ] strongly agree

[ ] agree

[ ] neither agree nor disagree

[ ] disagree

Section II: Answer each question to the best of your ability:

12. What are the strengths of the OSPEC course?

13. What are the weaknesses of the OSPEC course?

14. Do you have any suggestions for improving the OSPEC course?

Appendix 3

Criteria 4 3 2 1 0 Comments
Topic Development Content Demonstrates mastery of topic Demonstrates adequate understanding of topic Demonstrates some understanding of topic Demonstrates lack of understanding
Arrangement of Ideas Strong introductory statement Adequate introductory statement Weak introductory statement Lack of introductory statement
Strong logical progression Adequate logical progression Weak logical progression Lack of logical progression
Strong concluding statement Adequate concluding statement Weak concluding statement Lack of concluding statement
Oral Communication Voice Projection Clearly and consistently audible Mostly audible Sometimes audible Inaudible 0
Vocal Delivery Energetically communicates enthusiasm Some energy and enthusiasm Little energy and enthusiasm Monotone
Eye Contact Consistently makes eye contact with all members of audience Makes consistent eye contact with limited group within audience Makes some eye contact with audience Makes little or no eye contact with audience
Gestures Consistently reinforces verbal message Often reinforces verbal message Sometimes reinforces verbal message Detracts from verbal message
Pace Consistently effective Mostly effective At times too fast or too slow Consistently too fast or too slow
Clarity of Expression Minimal gap- fillers (“um,” “uh,” “like,” etc.) Few gap-fillers Some gap-fillers Gap-fillers interfere with expression
Listening Demonstrates proactive listening: takes relevant notes, answers questions, asks questions that demonstrate mastery of topic Demonstrates active listening: takes notes, answers questions with a prompt, demonstrates engagement with topic Demonstrates attentive listening: takes cursory notes Unable to respond to questions, demonstrates weak understanding of topic
Show Sections