Proficiency-based progression simulation training for more than an interesting educational experience

Background

Until very recently, doctors and surgeons had been trained using an apprenticeship model developed in the late 19^th century by Dr. William Stewart Halsted at the Johns Hopkins Hospital, Baltimore, USA.^[1] Surgeons have conventionally acquired their craft operating on real patients, while working for long hours on hospital wards and in the operating room. Up until late in the 20^th century, most surgical operations were carried out through an open incision, with larger incisions usually implying more advanced and complex surgery. The amount of pain experienced by the patient and time spent recovering in the hospital were closely correlated to the size of the incision made by the surgeon. Advances in computers and microchip technology that ushered in a digital age have also revolutionized the operating room. This new digital age enabled the same image processing capability that underpins the camera in mobile phones to be used at the end of a thin 30-cm long fiber-optic telescope to look inside patients, through a small surgical incision. This new minimally invasive approach to surgery was then used to perform increasingly more advanced surgical procedures, as well as robotic surgery.^[2] As a consequence, patients who had major surgery had less scarring, pain, and time in the hospital. However, it quickly became clear that even very experienced surgeons had difficulties acquiring the new and very different skills necessary to perform keyhole and robotic surgery.^[3] Surgeons now had to perform a procedure while looking at a TV monitor that produced images inferior to that perceived by the naked eye and lacked many of the subtle visual cues for judging the depth of field.^[4] There were also considerable difficulties in coordinating surgical instruments that passed through tiny incisions, which pivoted against the body wall, therefore giving the impression of counterintuitive instrument movements on the monitor.^[5] Tactile feedback from the tissues being operated on was reduced, or, in the case of surgical robots, absent. All of these human-factor difficulties made the already difficult job of performing surgery safely now orders of magnitude more challenging.^[6]

Simulation-Based Training

These training difficulties required the surgical and scientific community to reflect on why this was so and highlighted the need to develop new ways of training surgeons for these procedures. The revolution in computer technology that had led to the problems faced by surgeons would subsequently be the very same technology that would offer a very powerful training solution. Aviation had used computer-generated virtual reality (VR) simulations to train pilots for decades. However, unlike airplanes and airports with standardized features, real patients are all different. Furthermore, the aviation industry had over decades worked out precise protocols for dealing with different airplanes, airport terrains, and flight scenarios. Surgery in comparison was very much a craft with individual surgeons applying their own art to procedure performance.^[2] To utilize simulations for training, surgeons had first to develop surgical procedure templates, including, for example, the individual steps of the procedure and the choice of instruments. They also had to identify optimal procedural performance and deviations from optimal procedural performance so that engineers and computer scientists could build the simulation and accurately characterize the operation to ensure that procedure performance was quantifiable. Thus, surgical procedures could be learned and rehearsed on a VR simulation before operating on a patient for the first time.^[7],[8],[9]

Proficiency-Based Progression Simulation Training

Training on a simulation does not on its own predict optimum skill acquisition. Proficiency-based progression (PBP)^[6] training on simulation does. A PBP training program requires a detailed, comprehensive, and validated metric-based characterization of the skills to be learned.^[8] The metrics derived from this characterization are then used to guide the construction of a curriculum based on what the trainee needs to know and be able to do. Most importantly, the validated metrics are also used to establish a quantitatively defined benchmark which trainees must demonstrate before training progression. Benchmarks are based on the objectively assessed performance of experienced clinicians. Trainees are fully cognizant of the metrics, which are also used to implement a deliberate practice^[10] approach to education and training. Trainers and trainees are also fully aware that training is complete only on demonstration of the proficiency benchmark. When applied with scientific rigor, a PBP approach to medical education and training is effective, objective, transparent, and fair to the trainee and their training organization.

Proficiency-Based Progression Requires “Deliberate” Not “Repeated” Practice Training

The availability of VR computer and other simulations means that surgeons can now learn how to perform a specific procedure using the exact same instruments and devices, in the exact same way, on simulation models or on virtual patients (some of which are based on real cases). In the past, they learned these skills (and made mistakes) on real patients, but in a simulation or skills laboratory, they can learn the skills on part-task trainers or virtual patients. This enables them to perform the parts of complete procedures repeatedly and learn what not to do as well as what to do. This type of learning with performance feedback is called deliberate practice and is fundamental to a PBP approach to training. Deliberate practice constitutes a very powerful approach to training that contrasts with the traditional apprenticeship model where performance feedback and learning were much more hit-and-miss. PBP using deliberate practice provides performance feedback to the trainee, which is objective, transparent, and fair. In 2011, the Department of Health (DoH) in the United Kingdom (UK) proposed that all health-care procedures should be learned this way and a procedure should not be performed on a real patient the first time it is performed. (https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/146989/dh_131061.pdf.pdf).

Training Must be More Than an Interesting Educational Experience

PBP using this more deliberate approach to the acquisition of skills for the operating room relies on systematic, simulation-based education and training.^[8] It means that surgeons (and other health care workers) can be optimally prepared for the operating room with their performance benchmarked against other surgeons before operating in vivo on a real patient. Research has now shown that surgeons trained using this approach perform significantly better and make fewer errors than traditionally trained surgeons,^{[11],[12],[13],[14],[15]} and approximately 40% of what is learned on a VR simulation transfers to real-world tasks.^[16] The DoH in the UK has advocated that, wherever possible, learning to take care of patients should be enhanced with technology-enhanced learning (TEL) through using digital and simulation technology.

Training with VR simulation and TEL allows and ensures learning to a quantitatively defined performance level with greater homogeneity in trainee skillsets.^[8] Evidence from prospective, randomized studies demonstrates that this “outcome-based” approach to education and training produces trainees with skillsets that are 40%–70% better than trainees trained using a conventional/traditional approach to training.^{[11],[12],[13],[15]} These studies also demonstrate that trainees who receive the exact same curriculum but without the quantitatively defined performance benchmark (i.e., simulation training) perform only marginally better than those receiving conventional training [Figure - 1].^[14] Furthermore, similar results have been observed for an outcome-based communication skill training program (e.g., handover). These results clearly demonstrate that simulation-based training is effective for technical as well as communication skills training, but the simulation training must be more than an interesting educational experience. These results also demonstrate that requiring trainees (no matter how senior or experienced) to “train” and use a simulation or skills laboratory does not guarantee a quality-assured and verified performance level at the completion of training. Using the exact same resources with a PBP curriculum and the requirement to demonstrate quantitatively defined skills benchmarks does.

Figure 1: Trainees demonstrating the quantitatively defined proficiency benchmark for an arthroscopic Bankart procedure after training

This approach to training may be conceptually and intellectually appealing, but it represents a paradigm shift in how surgeons and doctors are educated and trained.^{[6],[9],[17],[18],[19]}