Advertisement

Validated simulation models in pediatric surgery: A review

Open AccessPublished:June 27, 2022DOI:https://doi.org/10.1016/j.jpedsurg.2022.06.015

      Highlights

      • This is a comprehensive oversight on the availability and validation of simulation models in the field of pediatric surgery.
      • The number of validated simulation models for pediatric surgery procedures is growing, however, few are available.
      • This indicates a need for more efforts to develop adequate simulation models and make these available for widespread use.

      Abstract

      Introduction: This review evaluates the validation and availability of simulation models in the field of pediatric surgery that can be used for training purposes.

      Methods

      MEDLINE and EMBASE were searched for studies describing a simulation models in pediatric surgery. Articles were included if face, content and/or construct validity was described. Additionally, the costs and availability were assessed. Validation scores for each model were depicted as percentage (0–100), based on the reported data, to compare the outcomes. A score of >70% was considered adequate.

      Results

      Forty-three studies were identified, describing the validation process of 38 simulation models. Face validity was evaluated in 33 articles, content in 36 and construct in 19. Twenty-two models received adequate validation scores (>70%). The majority (27/38, 70%) was strictly inanimate. Five models were available for purchase and eleven models were replicable based on the article.

      Conclusion

      The number of validated inanimate simulation models for pediatric surgery procedures is growing, however, few are replicable or available for widespread training purposes.
      Level of evidence: Level II.

      Keywords

      1. Introduction

      Until present, training for specific pediatric surgery procedures remains challenging. This is firstly owing to the high technical complexity of procedures with a small working space [
      • Yokoyama S.
      • Mizunuma K.
      • Kurashima Y.
      • et al.
      Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review.
      ] and secondly to limited case exposure as a result of the rarity of some of the pediatric surgical conditions [
      • Patel E.A.
      • Aydın A.
      • Desai A.
      • Dasgupta P.
      • Ahmed K.
      Current status of simulation-based training in pediatric surgery: a systematic review.
      ]. The latter leads to little exposure in clinical setting, which is exaggerated by the increasing emphasis on efficiency and maximizing productivity in the operating theater [
      • Beasley S.W.
      The challenges facing training in pediatric surgery worldwide.
      ]. During training, operative time is increased, especially in the early stages of a trainee's learning curve [
      • Gostlow H.
      • Marlow N.
      • Babidge W.
      • Maddern G.
      Systematic review of voluntary participation in simulation-based laparoscopic skills training: motivators and barriers for surgical trainee attendance.
      ]. Trainees subsequently decrease the number of patients that can be operated in one day and hence increase costs. Pressure to increase theater throughput can result in compromising the training opportunities of surgical trainees [
      • Beasley S.W.
      The challenges facing training in pediatric surgery worldwide.
      ]. Moreover, there is the ethical debate regarding less experienced surgeons operating on live patients less than supervision [
      • Gostlow H.
      • Marlow N.
      • Babidge W.
      • Maddern G.
      Systematic review of voluntary participation in simulation-based laparoscopic skills training: motivators and barriers for surgical trainee attendance.
      ]. Traditionally, training is mostly based on the apprenticeship model and mentoring. This model assumes that trainees gain knowledge and skills simply by exposing them to procedures and they learn to perform surgical procedures by operating on patients less than strict supervision, which in time becomes less strict until the surgeon is fully capable of performing the procedure without supervision [
      • Patel E.A.
      • Aydın A.
      • Desai A.
      • Dasgupta P.
      • Ahmed K.
      Current status of simulation-based training in pediatric surgery: a systematic review.
      ]. However, if supervision is not strictly monitored, it may result in an increased risk of treatment failure exposing the patient to a higher risk of complications. Particularly in this current era where maintaining competency, patient outcome, and safety is being increasingly scrutinized, it is worthwhile to critically look at other ways of training besides the traditional apprenticeship model of surgical training [
      • Oquendo Y.A.
      • Riddle E.W.
      • Hiller D.
      • Blinman T.A.
      • Kuchenbecker K.J.
      Automatically rating trainee skill at a pediatric laparoscopic suturing task.
      ].
      An alternative is simulation based training, using inanimate or animate simulation models. Simulation models have demonstrated their merit for skill acquisition [
      • Dawe S.R.
      • Windsor J.A.
      • Broeders J.A.
      • et al.
      A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy.
      ]. However, with the increasing number of simulation models in pediatric surgery it is important to evaluate their validity. This review assesses the validation of these simulation models and describes the current availability and costs.

      2. Methods

      2.1 Information sources and search

      Eligible studies were identified through an online search of MEDLINE and EMBASE. The search including the following index terms: “pediatric or pediatric” combined with “surg” and “simulation”. Additional terms were “model” and “simulation training”. The search strategy was designed in cooperation with librarians to minimize sampling bias. The full search strategy is described in Supplementary Fig. 1. No restrictions regarding language, year or publication type were imposed.

      2.2 Study eligibility criteria

      Titles and abstracts were screened according to the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines. All retrieved studies were independently assessed by two reviewers (MJ and SB) through Rayyan QCRI according to the eligibility criteria [
      • Ouzzani M.
      • Hammady H.
      • Fedorowicz Z.
      • Elmagarmid A.
      Rayyan — a web and mobile app for systematic reviews.
      ]. Articles were first screened on title and abstracts. Remaining results were examined on full text. Conflicts were resolved by discussion. Studies were eligible for inclusion if the authors described any form of validation of a simulation model for general pediatric surgery (as defined by the European training requirements [
      UEMS
      Training requirements for the specialty of pediatric surgery, European standards of postgraduate medical specialist training.
      ] including pediatric urology, pediatric traumatology but excluding pediatric neurosurgery [
      • Schmedding A.
      • Rolle U.
      • Czauderna P.
      Pediatric surgical training.
      ]). Exclusion criteria were: studies only describing simulation models, simulation models for other specialties, simulation models for nonsurgical procedures, conference abstracts, letters to the editor, nonenglish articles and studies not published in full text. In case of duplicate publications from one institute, the most recent and complete dataset was considered for inclusion. Models were classified based on the procedure to be practiced.

      2.3 Data extraction

      The following variables were extracted: first author, country, year of publication, number of participants, type of validation of the simulation model, score for the validation, costs of construction and/or use of the model, replicability of the simulation model, and availability of the model. In order to be considered a replicable model, written or video instructions, detailed enough to produce the simulation model at-home, needed to be provided in the article. Furthermore, only materials that were easily available to the researchers (e.g. common materials that could be found in a hospital or bought at a dime store) were to be used, without the need for expensive equipment.
      Scores for validation were calculated based on the reported data and depicted as percentage (0–100) of the scale used in the article. A score of >70% on the used scale used was considered adequate for validation and >90% was considered excellent. Regarding the number of participants, eight experts and eight novices or trainees was regarded the minimum requirement for adequate participant numbers [
      • Yusoff M.S.B.
      ABC of content validation and content validity index calculation.
      ].
      Types of validation were based on the definitions by McDougall and Van Nortwick et al. [
      • McDougall E.M.
      Validation of surgical simulators.
      ,
      • Van Nortwick S.S.
      • Lendvay T.S.
      • Jensen A.R.
      • et al.
      Methodologies for establishing validity in surgical simulation studies.
      . Assessment of the realism of the simulation models (by experts and nonexperts) was regarded face validity. The judgement of the appropriateness of the simulator as a teaching modality (assessed by experts) was regarded content validity. Construct validity was defined as the ability of the simulation model to distinguish between the experienced and inexperienced pediatric surgeon, which meant that the simulation model had the capability of an objective assessment tool [
      • McDougall E.M.
      Validation of surgical simulators.
      ,
      • Van Nortwick S.S.
      • Lendvay T.S.
      • Jensen A.R.
      • et al.
      Methodologies for establishing validity in surgical simulation studies.
      ,
      • Carter F.J.
      • Schijven M.P.
      • Aggarwal R.
      • Grantcharov T.
      • Francis N.K.
      • Hanna G.B.
      • et al.
      Consensus guidelines for validation of virtual reality surgical simulators.
      ].

      3. Results

      3.1 Study selection

      The combined search resulted in 4917 articles, consisting of 4499 unique citations and 418 duplicates. After screening 43 articles were found eligible for inclusion and data extraction. The PRISMA flowchart for study selection is shown in Fig. 1.

      3.2 Study characteristics

      An overview of included studies can be seen in Supplementary Table 1. A total of 38 simulation models were described. The majority of the studies (91%) were published after 2013, with most publications in 2014, 2015, and 2016 (six each year). The majority of articles described a process for face (33 articles), content (36 articles) or construct (19 articles) validation. A total of 22 (58%) simulation models received adequate validation scores for at least one form of validation (20 face validation and 21 content validation).
      The majority (27, 70%) of simulation models were strictly inanimate, five were living animal models, one simulation model used a chicken cadaver and five consisted of an inanimate casing with fetal bovine tissue for organs. No articles regarding augmented or virtual reality were identified.
      Of all studies, fourteen calculated or estimated the costs of construction of the simulation model, ranging from €0.20 to €1800. A detailed description or video material for replication was provided for eleven models [
      • Thompson J.L.
      • Grisham L.M.
      • Scott J.
      • Mogan C.
      • et al.
      Construction of a reusable, high-fidelity model to enhance extracorporeal membrane oxygenation training through simulation.
      ,
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ,
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ,
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ,
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ,
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Millán C.
      • Rey M.
      • Lopez M.
      LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
      ,
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ] and only five models were available for purchase [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ,
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ,
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      .

      3.3 Pediatric surgical simulation models

      3.3.1 ECMO cannulation

      Overall, two validation studies describing two different simulation models were identified, both describing an inanimate simulation model [
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ,
      • Thompson J.L.
      • Grisham L.M.
      • Scott J.
      • Mogan C.
      • et al.
      Construction of a reusable, high-fidelity model to enhance extracorporeal membrane oxygenation training through simulation.
      . The model described by Botden et al. is a low budget model which can be used for ECMO cannulation. It consisted of a 3D printed reusable base with small water-balloons to simulate the vessels. It was evaluated by a target group and experts and showed face and content validity with scores of 76% and 78% respectively [
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ]. The article by Thompson et al. described a simulation model for initiating ECMO treatment, including the cannulation. It consisted of a high-fidelity mannequin model with latex tubing for vessels to simulate veno-arterial cannulation of an unstable neonate. Face and content validity statements were scored on a 5-point Likert scale, however, outcome scores of this process were not provided. Videos for replication are provided and costs of construction are estimated at $25 [
      • Thompson J.L.
      • Grisham L.M.
      • Scott J.
      • Mogan C.
      • et al.
      Construction of a reusable, high-fidelity model to enhance extracorporeal membrane oxygenation training through simulation.
      ]. The model described by Botden et al. is commercially available for €75 [] (Table 1).
      Table 1Simulation models for ECMO cannulation. Values are stated as numbers or percentage.
      Simulation models for ECMO cannulation
      ModelAuthorType of modelOpen/ MISParticipants (experienced)FaceContentValidation?ReproducableCosts describedAvailable (price)
      ECMO cannulationBotden et al.InanimateOpen21 (14)76%78%Yes (Face, content)NoNoYes, (€75)
      ECMO cannulationThompson et al.InanimateOpen17 (3)unknownunknownNoYes (video)$25No

      3.3.2 Congenital diaphragmatic hernia

      A total of seven articles described six simulation models for MIS repair of congenital diaphragmatic hernia (CDH) [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      ,
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Pérez-Merino E.M.
      • Usón-Casaús J.M.
      • Zaragoza-Bayle C.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Palme R.
      • Sánchez-Margallo F.M.
      Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      . No simulation models for open repair of a congenital diaphragmatic hernia were found. Of the six models, five were inanimate models [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      and one was a rabbit model (New Zealand white rabbit, 3.0–3.5 kg) [
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Pérez-Merino E.M.
      • Usón-Casaús J.M.
      • Zaragoza-Bayle C.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Palme R.
      • Sánchez-Margallo F.M.
      Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      . For the rabbit model, described by Perez-Merino et al. and Uson-Casaus et al., an incision was made in the Bochdalek triangle of the rabbits to introduce an experimental diaphragmatic hernia, which was thoracoscopically repaired after 72 h. For this model face validity was established (scores 78% and 86% respectively) as well as content validity (80% and 92% respectively) [
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      . Uson-Casaus et al. also describe a process of construct validity based on total time, VAS performance scores and quality of the sutures, however, the latter two are subjective scores which are not suitable for demonstrating construct [
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ].
      Barsness et al. established face (82%) and content (86%) validity of an inanimate neonatal ribcage model, which can be used without an additional box trainer [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      ]. The model consists of the left side of a neonatal thoracic cavity, which is printed in acrylonitrile-butadiene-styrene plastic and covered with a synthetic silicon rubber skin. Bökkerink et al. have developed an inanimate CDH model that can be used in any conventional box trainer. This model consists of a round plastic cup covered with a nonlatex surgical glove with the fingers cut off. It showed both face (70%) and content (72%) validity [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ]. Ljuhar et al. describe a validation process for an inanimate simulator which can be used for both CDH repair and inguinal hernia repair [
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ]. The model consists of three pieces of ply wood with an opening on the side. A piece of neoprene with a cut-out defect was places more than the opening to simulate a congenital diaphragmatic hernia. Content validation (80%) as well as construct validation (based on a combined score) was described, however, of the 107 participants none were pediatric surgeons. Therefore, content validation for pediatric surgery was not established [
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ] (Table 2).
      Table 2Simulation models for procedures in the thoracic cavity or chest.
      Simulation models for the thoracic cavity
      Thoracic cavity/chest
      ModelAuthorType of modelOpen/MISParticipants (experienced)FaceContentValidationReproducableCosts describedAvailable (price)
      Chest tube placementAl-Qadhi et al.InanimateOpen24 (10)60%NoNoNoNo
      Pediatric chest modelHarada et al.Inanimate MIS box trainerMIS30 (14)-NoNoNo*No
      Pediatric chest modelTakazawa et al.Inanimate MIS box trainerMIS53 (8)-NoNoNo*No
      Pleural empyemaMarecos et al.Animal (rabbit)MIS30 (30)95%-Yes (Face)Yes (description)No*No
      Lobectomy (neonatal ribcage)Barsness et al.inanimate casing animal tissueMIS33 (11)90%88%Yes (Face, Content)NoNo*No
      CDH simulation models
      CDHBarsness et al.InanimateMIS40 (9)82%86%Yes (Face, Content)No$218*No
      CDHObata et al.InanimateMIS29 (10)68%NoNoNo*No
      CDHUson-Casaus et al.Animal (Rabbit)MIS25 (5)78%80%NoYes (description)No*No
      CDHPerez-Merino et al.Animal (Rabbit)MIS6 (6)86%92%NoNoNo*No
      CDHLjuhar et al.InanimateMIS107 (0)-80%NoYesNo*No
      CDHReino-Pires et al.InanimateMIS19 (6)78%68%NoYes€11*No
      CDHBökkerink et al.InanimateMIS60 (18)70%72%NoYes (video, instructions)€1.65*Yes (improved version €10)
      EA simulation model
      EA with TEFMaricic et al.InanimateMIS39 (7)86%88%NoNoNo*No
      EA with TEFBarsness et al.inanimate casing animal tissueMIS11 (11)90%90%Yes (Face, Content)NoNo*No
      EA with TEFBarsness et al.inanimate casing animal tissueMIS20 (8)94%92%Yes (Face, Content)No$290*No
      EA with TEFBarsness et al.inanimateMIS44 (14)82%80%Yes (Face, Content)No$202 (re-usable 20 times)*No
      EA with TEFDeie et al.Inanimate modelMIS40 (6)78%88%NoNoNo*No
      EABökkerink et al.InanimateMIS60 (18)74%74%Yes (Face, Content)Yes (video, instructions)€0.20*No
      * simulation model for use in a box simulator and/or needs a laparoscopic camera or instruments for use possibly resulting in additional costs.
      Obata et al. described construct validation based on total time for their inanimate model [
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ]. The model consists of a left and right thoracic cavity divided by a mediastinum sheet and a detachable diaphragm unit. It is covered with a soft skin sheet. This model was established to replicate the full procedure of thoracoscopic repair of a CDH. While they also described a process for face validation, it only received a score of 68% and did not establish the validation criteria in this review. Neither did Reino-Pires et al., who reported a score of 68% for content validation for another inanimate simulation model, consisting of ordinary materials purchased in a dime store (food container, a neoprene band for the diaphragm and a body wash sponge simulating a collapsed lung) [
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ,
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      .
      Uson-Casaus et al. provided a detailed description and Bökkerink et al. provided video instructions for replication of the simulation models [
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      .
      Calculated or estimated costs are provided by Barsness et al. ($218), Reino-Pires et al. (€11) and Bökkerink et al. (€1.65) [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      . The inanimate simulation model by Bökkerink et al. is the only model that is commercially available (€10 for an improved version) [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ] (Table 2).

      3.3.3 Chest/thoracic cavity

      Overall, four simulation models for procedures in the thoracic cavity or chest (other than CDH) are described in five articles [
      • Harada K.
      • Takazawa S.
      • Tsukuda Y.
      • Ishimaru T.
      • Sugita N.
      • Iwanaka T.
      • Mitsuishi M.
      Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model.
      ,
      • Takazawa S.
      • Ishimaru T.
      • Harada K.
      • Deie K.
      • Fujishiro J.
      • Sugita N.
      • Mitsuishi M.
      • Iwanaka T.
      Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer.
      ,
      • Al-Qadhi S.A.
      • Pirie J.R.
      • Constas N.
      • Corrin M.S.
      • Ali M.
      An innovative pediatric chest tube insertion task trainer simulation: a technical report and pilot study.
      ,
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
      .
      Marecos et al. establish excellent face validation (95%) for a living rabbit model with pleura empyema (New Zealand rabbit, weighing 3.0–4.0 kg) [
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ]. Furthermore, they provide a detailed description of preparation of the model in the methods. A simulation model for MIS lobectomy in an inanimate neonatal ribcage with fetal bovine tissue for organs is described by Barsness et al. [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
      ]. They established excellent face validity (90%) as well as content validity (88%) [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
      ].
      The pediatric chest inanimate model described by Harada et al. and Takazawa et al. consisted of a pneumoperitoneum model with a detachable rubber sheet for the esophageal crura unit and styrene material for the organs (stomach, liver, spleen) which was covered with synthetic skin. The model both established construct validity for intracorporeal suturing and knot tying [
      • Harada K.
      • Takazawa S.
      • Tsukuda Y.
      • Ishimaru T.
      • Sugita N.
      • Iwanaka T.
      • Mitsuishi M.
      Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model.
      ,
      • Takazawa S.
      • Ishimaru T.
      • Harada K.
      • Deie K.
      • Fujishiro J.
      • Sugita N.
      • Mitsuishi M.
      • Iwanaka T.
      Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer.
      . However, no face or content validity data were provided on this model.
      Al-Qadhi et al. describe a model for chest tube placement consisting of a plaster shell with a multilayer silicon insert representing the thoracic cavity, filled with gauzes in a silicon layer for muscle fibers and foam for subcutaneous fat. This model receives a score of 60% for content validation [
      • Al-Qadhi S.A.
      • Pirie J.R.
      • Constas N.
      • Corrin M.S.
      • Ali M.
      An innovative pediatric chest tube insertion task trainer simulation: a technical report and pilot study.
      ]. No simulation models are commercially available and none of the articles provided any cost estimates (Table 2).

      3.3.4 Esophageal atresia

      Four models for MIS esophageal atresia (EA) repair are described in six articles [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ,
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      . All models are inanimate with the exception of the EA with TEF model described by Barsness et al., which consisted of an inanimate casing with fetal bovine tissue for the organs [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      .
      Maricic et al. describe a model consisting of domestic materials such as plastic tubes to simulate ribs and latex balloons for the esophagus which is inserted in a rubber thoracic cavity [
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ]. Deie et al. describe a rapid-prototyped neonatal chest model with an artificial esophagus model [
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      ].
      Face and content validity is described for all models, however, Maricic et al. and Deie et al. described the opinion of less than eight experts (seven and six respectively) [
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ,
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      . Barsness et al. described face (90% and 94%) and content (90% and 92%) validation for the inanimate casing with fetal bovine tissue [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      . For the strictly inanimate simulation model developed by Barsness et al. this was 82% for face and 80% for content validation [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ]. Bökkerink et al. described an inanimate simulation model consisting of two water balloons on a suturing pad, which can be used in a conventional box trainer, of which face (74%) and content (74%) validation were established.
      A process for construct validation was described by Maricic et al. (based on total time, errors and incomplete anastomosis), Barsness et al. (based on OSATS) and Deie et al. (29-point checklist, error score, number of manipulations and task completion time) [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ,
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      .
      The EA with TEF model described by Barsness et al. was estimated to cost $290 for the construction of the inanimate casing with fetal bovine tissue for organs and $202 dollar for the inanimate model, the latter being reusable up to twenty times [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ]. Bökkerink et al. estimate the cost of construction of the EA model at €0.20 per model and provided a detailed description and video instructions for replication of the model at home, to use in any MIS box trainer [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ]. No models were commercially available (Table 2).

      3.3.5 Fundoplication

      One inanimate model and one animal model for fundoplication are described in a total of three articles [
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ,
      • Ieiri S.
      • Ishii H.
      • Souzaki R.
      • Uemura M.
      • Tomikawa M.
      • Matsuoka N.
      • Takanishi A.
      • Hashizume M.
      • Taguchi T.
      Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication.
      ,
      • Jimbo T.
      • Ieiri S.
      • Obata S.
      • Uemura M.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Masumoto K.
      • Hashizume M.
      • Taguchi T.
      A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system.
      . All models are for MIS procedure, no models for open procedures were identified. Ieiri et al. describe a model consisting of a suturing pad which was modified to a suture ligature model of the crura of the diaphragm [
      • Ieiri S.
      • Ishii H.
      • Souzaki R.
      • Uemura M.
      • Tomikawa M.
      • Matsuoka N.
      • Takanishi A.
      • Hashizume M.
      • Taguchi T.
      Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication.
      ]. Jimbo et al. describe the development of an infant body based on computed tomography data with an esophageal crura unit made as a detachable sheet [
      • Jimbo T.
      • Ieiri S.
      • Obata S.
      • Uemura M.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Masumoto K.
      • Hashizume M.
      • Taguchi T.
      A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system.
      ]. The animal model described by Esposito et al. was multifunctional with the option to practice inguinal hernia repair, varicocelectomy, nephrectomy and fundoplication [
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ]. However, it was only tested by ten trainees and no experts, a process of establishing content validation was conducted only by comparing it to a pig model (95% in favor of the rabbit model). Ieiri et al. and Jimbo et al. only reported construct validation for an inanimate simulation model (based on total time, force on the tissue, stitches spacing for the former and total time, suturing balance, path length and velocity for the latter). They did not describe face or content validity, therefore no rating could be given on these values [
      • Ieiri S.
      • Ishii H.
      • Souzaki R.
      • Uemura M.
      • Tomikawa M.
      • Matsuoka N.
      • Takanishi A.
      • Hashizume M.
      • Taguchi T.
      Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication.
      ,
      • Jimbo T.
      • Ieiri S.
      • Obata S.
      • Uemura M.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Masumoto K.
      • Hashizume M.
      • Taguchi T.
      A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system.
      . Only Esposito et al. provided instructions for replication [
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ] and neither model is currently commercially available (Table 3).
      Table 3Simulation models for procedures in the upper abdomen.
      Simulation models for procedures in the upper abdomen
      Fundoplication simulation models
      ModelAuthorType of modelOpen/ MISParticipants (experienced)FaceContentValidationReproducableCosts describedAvailable (price)
      FundoplicationIeiri et al.InanimateMIS20 (10)-NoNoNo*No
      FundoplicationJimbo et al.InanimateMIS49 (15)-NoNoNo*No
      FundoplicationEsposito et al.Animal (Rabbit, Pig)MIS10 trainees-95%NoYes (instructions)No*No
      Pyloromyotomy
      PyloromyotomyBallouhey et al.InanimateMIS80 (15)78%78%Yes (Face, content)Yes (instruction)No*No
      PyloromyotomyWilliams et al.InanimateMIS27 (9)83%83%Yes (Face, content)No$30*No
      PyloromyotomySkertich et al.InanimateMIS28 (0)96%82%NoNo€290*No
      PyloromyotomyPlymale et al.Inanimate,MIS55 (29)80%85%Yes (Face, Content)NoNo*No
      Duodenal atresia simulation models
      Duodenal atresiaOrdorica-Flores et al.Animal (rabbit)MIS (use in simulator)13 (13)96%84%Yes (Face, Content)No$32*No
      Duodenal atresiaBarness et al.inanimate casing with animal tissueMIS18 (6)88%95%NoNoNo*No
      * simulation model for use in a box simulator and/or needs a laparoscopic camera or instruments for use possibly resulting in additional costs.

      3.3.6 Pyloromyotomy

      For practicing pyloromyotomy four inanimate MIS simulation models were identified [
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ,
      • Williams A.
      • McWilliam M.
      • Ahlin J.
      • Davidson J.
      • Quantz M.A.
      • Bütter A.
      A simulated training model for laparoscopic pyloromyotomy: is 3D printing the way of the future?.
      ,
      • Plymale M.
      • Ruzic A.
      • Hoskins J.
      • French J.
      • Skinner S.C.
      • Yuhas M.
      • Davenport D.
      • Iocono J.A.
      A middle fidelity model is effective in teaching and retaining skill set needed to perform a laparoscopic pyloromyotomy.
      ,
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      . Ballouhey et al. described a low cost replicable model using basic materials such as a balloon filled with silicone paste. The model was placed in a pediatric laparoscopic surgery simulator for use [
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ]. They established face validity (78%) and content validity (78%). They described construct validity as well, however, this was based on subjective scores by expert observers and not the simulation model itself (OSATS, pyloromyotomy OSATS, mucosal perforation and incomplete pyloromyotomy) [
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ]. A detailed description of the model was provided in the supplementary data of the article for replication of the model. Williams et al. established face validity (83%), content validity (83%) for their 3D printed model for use in a box trainer. They attempted construct validity based on mean procedural time, however, the latter was not discriminative [
      • Williams A.
      • McWilliam M.
      • Ahlin J.
      • Davidson J.
      • Quantz M.A.
      • Bütter A.
      A simulated training model for laparoscopic pyloromyotomy: is 3D printing the way of the future?.
      ]. Plymale et al. described face (80%) and content validation (85%) for a middle fidelity MIS simulation model [
      • Plymale M.
      • Ruzic A.
      • Hoskins J.
      • French J.
      • Skinner S.C.
      • Yuhas M.
      • Davenport D.
      • Iocono J.A.
      A middle fidelity model is effective in teaching and retaining skill set needed to perform a laparoscopic pyloromyotomy.
      ]. Skertich et al. described the development of a model for gastroschisis, perforated NEC and pyloromyotomy. For pyloric stenosis silicone a raw sausage with a core of an inflated balloon was used, which was placed in a box trainer. They describe excellent face (96%) and content validity (82%), however, only trainees and no expert pediatric surgeons were included [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ]. They do provide cost estimates for construction of the model, which are €290 per simulation model [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ] (Table 3).

      3.3.7 Duodenal atresia

      For duodenal atresia two (partly) animate simulation models were identified [
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator.
      . Ordorica-Flores et al. describe the validation process of a rabbit model (weighing 3.0–4.5 kg) with excellent face validity (96% score) and content validity (84% score) [
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ]. The costs are estimated at $32 per rabbit. Barsness et al. describe a model consisting of an inanimate casing with fetal bovine tissue for the organs [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator.
      ]. They describe face validity (88%) as well as excellent content validity (95%), however, this is based on the opinion of only six experts. Simulation models are not commercially available (Table 3).

      3.3.8 Bile ducts

      Two simulation models are described in three articles [
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Schwab B.
      • Rooney D.M.
      • Hungness E.S.
      • Barsness K.A.
      Preliminary evaluation of a laparoscopic common bile duct simulator for pediatric surgical education.
      ,
      • Burdall O.C.
      • Makin E.
      • Davenport M.
      • Ade-Ajayi N.
      3D printing to simulate laparoscopic choledochal surgery.
      . Schwab et al. and Santos et al. described an inanimate simulation model for common bile duct surgery consisting of a liver, gallbladder, extrahepatic biliary system and duodenum, all created out of synthetic materials [
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Schwab B.
      • Rooney D.M.
      • Hungness E.S.
      • Barsness K.A.
      Preliminary evaluation of a laparoscopic common bile duct simulator for pediatric surgical education.
      . Schwab et al. described face (80%) and content validity (90%), however this was only based on the opinion of trainees. Santos et al. described construct validity, however this was based on OSATS scores given by expert observers and not the model itself.
      Burdall et al. described the validation process of an inanimate model for laparoscopic choledochal surgery [
      • Burdall O.C.
      • Makin E.
      • Davenport M.
      • Ade-Ajayi N.
      3D printing to simulate laparoscopic choledochal surgery.
      ]. However, the model only scored 56% on their scale for face and 68% for content validity, additionally, no experts were included [
      • Burdall O.C.
      • Makin E.
      • Davenport M.
      • Ade-Ajayi N.
      3D printing to simulate laparoscopic choledochal surgery.
      ]. Santos et al. provided a detailed description of the model and estimated the costs of construction at $465 [
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ]. No description for replication or cost estimates were provided of the model by Burdall et al. [
      • Burdall O.C.
      • Makin E.
      • Davenport M.
      • Ade-Ajayi N.
      3D printing to simulate laparoscopic choledochal surgery.
      ] (Table 4).
      Table 4Abdominal simulation models.
      Abdominal simulation models
      Bile ducts simulation models
      ModelAuthorType of modelOpen/MISParticipants (experienced)FaceContentValidationReproducableCosts describedAvailable (price)
      Common Bile ductSchwab et al.InanimateMIS30 (0)80%90%NoNoNo*No
      Common Bile ductSantos et al.InanimateMIS21 (5)-NoYes, instructions$465*No
      laparoscopic choledochal surgeryBurdall et al.InanimateMIS10 (0)56%68%NoNoNo*No
      Gastroschisis (silo placement)
      gastroschisisSkertich et al.InanimateOpen28 (0)98%82%NoNo€450No
      GastroschisisBacarese-Hamilton et al.InanimateOpen18 (14)78%82%Yes (Face, Content)NoNoNo
      NEC (drain placement)
      Perforated NECSkertich et al.InanimateOpen28 (0)100%82%NoNo€1000No
      * simulation model for use in a box simulator and/or needs a laparoscopic camera or instruments for use possibly resulting in additional costs.

      3.3.9 Gastroschisis

      Two simulation models for silo placement in gastroschisis were identified [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ,
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      . Skertich et al. describe a simulation model for silo placement for gastroschisis, which can also be used for percutaneous drain placement for perforated NEC and laparoscopic pyloromyotomy [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ]. This simulation model scored excellent for face (98%) and content validity (82%), however, no experts were included in this validation process and therefore validation was not established. Bacarese-Hamilton et al. described the validation process of another inanimate model for silo placement, using an umbilical cannulation simulation mannequin, which demonstrated face (78%) and content validity (82%) [
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      ]. Skertich et al. estimated the costs of their model at €450 [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ]. Both models were not commercially available (Table 4).

      3.3.10 Perforated NEC

      As mentioned previously, the simulation model by Skertich et al. could be used for drain placement for perforated necrotizing enterocolitis (NEC) as well. Although they reported excellent scores for face (100%) and content (82%) validity, no experts were included in this process, therefore validation was not established. Costs for construction were estimated at €1000 and the model was not commercially available [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ].

      3.3.11 Urology

      Millán et al. described the validation of an inanimate simulation model for uretral reimplantation [
      • Millán C.
      • Rey M.
      • Lopez M.
      LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
      ]. The model consisted of a water balloon and nasogastric tube covered with a silicone box, for use with MIS instruments. They established face (88%) and content validation (90%).
      Two inanimate models for pyeloplasty were identified [
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ,
      • Cheung C.L.
      • Looi T.
      • Lendvay T.S.
      • Drake J.M.
      • Farhat W.A.
      Use of 3-dimensional printing technology and silicone modeling in surgical simulation: development and face validation in pediatric laparoscopic pyeloplasty.
      . Cheung et al. described a process of face validation for a 3D printed silicone MIS model which can be used in a box trainer, however, the model scored only 68% on their used scale and was only evaluated by three experts [
      • Cheung C.L.
      • Looi T.
      • Lendvay T.S.
      • Drake J.M.
      • Farhat W.A.
      Use of 3-dimensional printing technology and silicone modeling in surgical simulation: development and face validation in pediatric laparoscopic pyeloplasty.
      ]. The simulation model for open pyeloplasty described by Rod et al., consisting of two water balloons, achieved a score of 82% for face validity as well, however, for content validation it only achieved 70% [
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ]. Cheung et al. and Rod et al. described the costs for construction of the simulation model ($100 and less than $1 respectively), the latter also provided a detailed description for replication [
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ,
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      . Construction costs of the model by Millán et al. were estimated at $25 per model and written instructions for replication of the model were provided [
      • Millán C.
      • Rey M.
      • Lopez M.
      LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
      ]. None of these simulation models were commercially available (Table 5).
      Table 5Urogenital, anorectal and inguinal hernia simulation models.
      Urogenital, anorectal and inguinal hernia simulation models
      Uretral reimplantation
      ModelAuthorType of modelOpen/MISParticipants (experienced)FaceContentValidationReproducableCosts describedAvailable (price)
      Uretral reimplantationMillán et al.InanimateMIS34 (12)88%90%Yes (Face, Content)Yes$25*No
      Pyeloplasty simulation models
      Pyeloplasty modelCheung et al.InanimateMIS25 (3)68%NoNo$100 *No
      Pyeloplasty modelRod et al.InanimateOpen118 (44)82%70%Yes (Face, Content)Yes$1No
      Anorectal malformation
      Anorectal malformation with perineal fistulaVan Ling et al.InanimateOpen44 (24)80%84%Yes (Face, Content)No€70 -€100Yes (€75)
      Inguinal hernia
      Inguinal herniaLjuhar et al.InanimateMIS107 (0)-80%NoYesNo*No
      * simulation model for use in a box simulator and/or needs a laparoscopic camera or instruments for use possibly resulting in additional costs.

      3.3.12 Anorectal malformations

      Van Ling et al. described the development and validation process of a simulation model of an anorectal malformation with perineal fistula [
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ]. This model consisted of a sponge for the perineal body and a double layered balloon for the rectal fistula. It showed face (80%) and content (84%) validity, the construction costs were provided (€70-€100) and it is commercially available online for €75,- [
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ] (Table 5).

      3.3.13 Inguinal hernia

      Ljuhar et al. describe a simulation model for the MIS repair of an inguinal hernia [
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ]. This model can be placed in a box trainer for training. They describe a process of content validity (80%) but did not include pediatric surgeons in the process, therefore content validation for use in training for pediatric surgery was not established. For construct validation they used a novel scoring system which did discriminate between novices and experts, but was based on subjective measures. A detailed description of the construction is provided in the article [
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ]. The model is not available for purchase (Table 5).

      3.3.14 MIS simulators

      Overall, four MIS box trainers and one animal model for MIS training are described in seven articles [
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ,
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      ,
      • Azzie G.
      • Gerstle J.T.
      • Nasr A.
      • Lasko D.
      • Green J.
      • Henao O.
      • Farcas M.
      • Okrainec A.
      Development and validation of a pediatric laparoscopic surgery simulator.
      ,
      • Nasr A.
      • Carrillo B.
      • Gerstle J.T.
      • Azzie G.
      Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills.
      ,
      • Trudeau M.O.
      • Carrillo B.
      • Nasr A.
      • Gerstle J.T.
      • Azzie G.
      Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
      ,
      • Torres A.
      • Inzunza M.
      • Jarry C.
      • Serrano F.
      • Varas J.
      • Zavala A.
      Development and validation of a new laparoscopic endotrainer for neonatal surgery and reduced spaces.
      . The Pediatric Laparoscopic Simulator (PLS) is the most frequently described (four articles) MIS box trainer. Content validity was only established by Retrosi et al. (86%) [
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ]. Construct validity was established by Azzie et al. (peg transfer, extracorporeal suturing and intracorporeal suturing), by Nasr et al. (intracorporeal suturing), Trudeau et al. (advanced suturing) and Retrosi et al. (object transfer, precision cutting and intracorporeal suturing) [
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ,
      • Azzie G.
      • Gerstle J.T.
      • Nasr A.
      • Lasko D.
      • Green J.
      • Henao O.
      • Farcas M.
      • Okrainec A.
      Development and validation of a pediatric laparoscopic surgery simulator.
      ,
      • Nasr A.
      • Carrillo B.
      • Gerstle J.T.
      • Azzie G.
      Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills.
      ,
      • Trudeau M.O.
      • Carrillo B.
      • Nasr A.
      • Gerstle J.T.
      • Azzie G.
      Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
      . The latter compare the PLS to the EoSim, establishing excellent content (90%) and construct validity for the EoSim as well.
      Bökkerink et al. compared the EoSim to the LaparoscopyBoxx and establish face (77% and 84% respectively) and content validity (79% and 90% respectively) for both simulators, with a favor for the LaparoscopyBoxx [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      ]. Torres et al. described a neonatal box trainer and established face (82%) and construct validity for nine basic exercises and an intracorporeal suturing task (based on total time needed for the exercise) [
      • Torres A.
      • Inzunza M.
      • Jarry C.
      • Serrano F.
      • Varas J.
      • Zavala A.
      Development and validation of a new laparoscopic endotrainer for neonatal surgery and reduced spaces.
      ]. For content validity, however, the simulator scored a mere 64%.
      Zimmerman et al. described the use of chicken cadavers for neonatal laparoscopic surgery training. Adhesiolysis, cholecystectomy, intestinal resection and intestinal anastomosis were performed on the model and face (77%) and content (84%) validity were established [
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ]. Cost estimates are provided for the EoSim (€1800) and Laparoscopyboxx (€315), including 3 mm instruments. Both box trainers can be used either with MIS camera or with a tablet resulting in less additional costs (Table 6).
      Table 6MIS trainer simulation models.
      MIS trainer
      ModelAuthorType of modelOpen/MISParticipants (experienced)FaceContentValidationReproducableCosts describedAvailable (price)
      Neonatal laparoscopic surgeryZimmerman et al.Chicken cadaversMIS27 (9)77%84%Yes (Face, Content)YesNo*No
      Neonatal Box trainerTorres et al.InanimateMIS49 (unknown)82%64%NoNoNo*No
      PLSAzzie et al.InanimateMIS84 (45)-NoNoNo*No
      PLSNasr et al.InanimateMIS75 (37)-NoNoNo*No
      PLSRetrosi et al.InanimateMIS28 (8)86%Yes (Content)NoNo*No
      PLSTrudeau et al.InanimateMIS60 (39)-NoNoNo*No
      EoSimRetrosi et al.InanimateMIS28 (8)90%Yes (Content)NoNoYes (€1800)
      EoSimBökkerink et al.InanimateMIS32 (17)77%79%Yes (Face, Content)NoNoYes (€1800)
      LaparoscopyBoxxBökkerink et al.InanimateMIS44 (24)84%90%Yes (Face, Content)NoNoYes (€315)
      * simulation model needs a MIS camera or MIS instruments for use possibly resulting in additional costs.

      4. Discussion

      This review provides an overview of the simulation models that are currently validated for use in the training for pediatric surgical procedures. By applying criteria for adequate validation of the pediatric simulation models, an indication can be given about areas where simulation models in pediatric surgery may be improved. Of all identified simulation models with a validation process (n = 38), only twenty-two received adequate validation scores, leaving ample room for improvement.

      4.1 Validation of simulation models

      Establishing the validity of simulation models is critical before examining the effectiveness of simulation-based training [
      • Van Nortwick S.S.
      • Lendvay T.S.
      • Jensen A.R.
      • Wright A.S.
      • Horvath K.D.
      • Kim S.
      Methodologies for establishing validity in surgical simulation studies.
      ]. However, before validity can be assessed, consistent terminology should be used. In the included studies there was a lack of this consistency in the terminology, and terms were used in a different context to that described by McDougall and Van Nortwick et al. [
      • McDougall E.M.
      Validation of surgical simulators.
      ,
      • Van Nortwick S.S.
      • Lendvay T.S.
      • Jensen A.R.
      • et al.
      Methodologies for establishing validity in surgical simulation studies.
      . Furthermore, some studies failed to label the assessed validity, even though it had been demonstrated in the study. Other studies used different criteria, such as the Standards for Educational and Psychological Testing [
      • Sireci S.
      • Faulkner-Bond M.
      Validity evidence based on test content.
      ]. Although a broad range of different scales was used, most studies reported the use of Likert-scale but failed to present data in the intended way [
      • Boone H.N.
      • Boone D.A.
      Analyzing Likert data.
      ,
      • Joshi A.
      • et al.
      Likert scale: explored and explained.
      . In order to compare simulation models and gain insight in whether a simulation model achieved adequate validation, we expressed face and content validation as a percentage on the scale used by the authors.
      In addition to receiving adequate scores, adequate numbers of participants (including experts) are needed for proper validation [
      • Yusoff M.S.B.
      ABC of content validation and content validity index calculation.
      ,
      • Patel E.A.
      • Aydın A.
      • Desai A.
      • Dasgupta P.
      • Ahmed K.
      Current status of simulation-based training in pediatric surgery: a systematic review.
      . Some studies failed to include experts all together, others stated that experts were asked for their opinion, however, these experts were not pediatric surgeons. Content validity is by definition not possible without experts, resulting in a lack of knowledge of the appropriateness of the model. Moreover, there was a noticeable lack of power analysis to determine the number of required subjects, which suggest that studies relied on convenience samples of subjects rather than predetermined required numbers.
      Simulation models that received adequate validation scores with the correct number of participants and experts were the ECMO cannulation model by Botden et al. [
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ], pleura empyema model by Marecos et al. (only face validity) [
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ], the lobectomy model and the CDH model by Barsness et al. [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
      and two EA with TEF models by Barsness et al. [
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ], three pyloromyotomy models (by Ballouhey et al. [
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ], Williams et al. [
      • Williams A.
      • McWilliam M.
      • Ahlin J.
      • Davidson J.
      • Quantz M.A.
      • Bütter A.
      A simulated training model for laparoscopic pyloromyotomy: is 3D printing the way of the future?.
      ] and Plymale et al. [
      • Plymale M.
      • Ruzic A.
      • Hoskins J.
      • French J.
      • Skinner S.C.
      • Yuhas M.
      • Davenport D.
      • Iocono J.A.
      A middle fidelity model is effective in teaching and retaining skill set needed to perform a laparoscopic pyloromyotomy.
      ]), an animal model for duodenal atresia by Ordorica-Flores et al. [
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ], a gastroschisis model by Bacarese-Hamilton et al. [
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      ], a model for ureteral reimplantation by Millán et al. [
      • Millán C.
      • Rey M.
      • Lopez M.
      LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
      ], a pyeloplasty model by Rod et al. [
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ], a simulation model for ARM by Van Ling et al. [
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ], the neonatal laparoscopic surgery model by Zimmerman et al. [
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ], the PLS and the EoSim evaluated by Retrosi et al. (both only content validity) [
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ], the EoSim and the LaparoscopyBoxx evaluated by Bökkerink et al. [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      ].

      4.2 Evaluation and limitations of simulation models

      The majority of the evaluated simulation models were inanimate. Models entirely made out of animal specimens were described for only four procedures: pleural empyema [
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ], CDH [
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Pérez-Merino E.M.
      • Usón-Casaús J.M.
      • Zaragoza-Bayle C.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Palme R.
      • Sánchez-Margallo F.M.
      Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      , MIS procedures [
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ] and duodenal atresia [
      • Skertich N.J.
      • Schimpke S.W.
      • Lee T.
      • Wiegmann A.L.
      • Pillai S.
      • Rossini C.
      • Madonna M.B.
      • Shah A.N.
      Pediatric surgery simulation-based training for the general surgery resident.
      ]. For all but one of these procedures (pleural empyema) an inanimate alternative was identified [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      ,
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      ,
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ,
      • Azzie G.
      • Gerstle J.T.
      • Nasr A.
      • Lasko D.
      • Green J.
      • Henao O.
      • Farcas M.
      • Okrainec A.
      Development and validation of a pediatric laparoscopic surgery simulator.
      ,
      • Nasr A.
      • Carrillo B.
      • Gerstle J.T.
      • Azzie G.
      Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills.
      ,
      • Trudeau M.O.
      • Carrillo B.
      • Nasr A.
      • Gerstle J.T.
      • Azzie G.
      Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
      ,
      • Torres A.
      • Inzunza M.
      • Jarry C.
      • Serrano F.
      • Varas J.
      • Zavala A.
      Development and validation of a new laparoscopic endotrainer for neonatal surgery and reduced spaces.
      .
      There are the ethical and cost considerations when using animal models for simulation-based training [
      • Barsness K.
      Simulation-based education and performance assessments for pediatric surgeons.
      ]. One article measured dyspnea and pain levels of their CDH animal models, which were significantly higher compared to control animals [
      • Pérez-Merino E.M.
      • Usón-Casaús J.M.
      • Zaragoza-Bayle C.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Palme R.
      • Sánchez-Margallo F.M.
      Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ]. This underlines the notion that using animal models is a delicate ethical issue and that alternatives for animal models should always be investigated and used wherever and whenever appropriate [
      • Patel E.A.
      • Aydın A.
      • Desai A.
      • Dasgupta P.
      • Ahmed K.
      Current status of simulation-based training in pediatric surgery: a systematic review.
      ,
      • Sharma D.
      • Agrawal V.
      • Bajaj J.
      • Agarwal P.
      Low-cost simulation systems for surgical training: a narrative review.
      ,
      • Mortell A.
      • Montedonico S.
      • Puri P.
      Animal models in pediatric surgery.
      ,
      • Barsness K.A.
      Trends in technical and team simulations: challenging the status quo of surgical training.
      . This review shows that inanimate alternatives are available for most animal models. These inanimate simulation models have the advantage that they can be used more easily during courses in different settings, without a wet-lab facility, and at-home for continued training after a course. Most of the models are (partly) reusable [
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ,
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ,
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Azzie G.
      • Gerstle J.T.
      • Nasr A.
      • Lasko D.
      • Green J.
      • Henao O.
      • Farcas M.
      • Okrainec A.
      Development and validation of a pediatric laparoscopic surgery simulator.
      ,
      • Nasr A.
      • Carrillo B.
      • Gerstle J.T.
      • Azzie G.
      Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills.
      ,
      • Trudeau M.O.
      • Carrillo B.
      • Nasr A.
      • Gerstle J.T.
      • Azzie G.
      Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
      ,
      • Torres A.
      • Inzunza M.
      • Jarry C.
      • Serrano F.
      • Varas J.
      • Zavala A.
      Development and validation of a new laparoscopic endotrainer for neonatal surgery and reduced spaces.
      or lower in costs than their animal counterpart [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Cheung C.L.
      • Looi T.
      • Lendvay T.S.
      • Drake J.M.
      • Farhat W.A.
      Use of 3-dimensional printing technology and silicone modeling in surgical simulation: development and face validation in pediatric laparoscopic pyeloplasty.
      ,
      • Bergmeister K.D.
      • Aman M.
      • Kramer A.
      • et al.
      Simulating surgical skills in animals: systematic review, costs acceptance analyses.
      .
      Inanimate simulation models, especially low-fidelity models, have the proclaimed disadvantage of having a lower degree of realism. However, research has shown that there is no advantage in learning success achieved by a higher degree of realism of the simulator. It is therefore questionable whether the additional costs and expenses of high-fidelity simulators are justified when comparable knowledge and skill outcomes are achieved with low-budget simulators [
      • Massoth C.
      • Röder H.
      • Ohlenburg H.
      • et al.
      High-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students.
      ].
      It was noticeable that the majority of the evaluated simulation models (79%) focused on MIS surgery instead of open surgery. The development and implementation of simulation-based training was triggered by the evidence of increased complications during the early adaptation of MIS, related to the unique challenges of this surgical approach [
      • Massoth C.
      • Röder H.
      • Ohlenburg H.
      • et al.
      High-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students.
      ]. This resulted in simulation models mainly focusing on minimally invasive surgery instead of on open surgery [
      • Stefanidis D.
      • Sevdalis N.
      • Paige J.
      • Zevin B.
      • Aggarwal R.
      • Grantcharov T.
      • Jones D.
      MS‖ for the association for surgical education simulation committee simulation in surgery.
      ]. For the latter only eight models were identified, although this is still the most common approach in pediatric surgery [
      Tam PKHLaparoscopic surgery in childrenArchives of Disease.
      ].
      Only five simulation models were commercially available: a model CDH repair by Bökkerink et al. [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ], the model for ECMO cannulation by Botden et al. [
      • Botden S.M.B.I.
      • Bökkerink G.M.
      • Leijte E.
      • et al.
      Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
      ], the ARM model by Van Ling et al. [
      • van Ling J.A.
      • Bökkerink G.M.J.
      • de Blaauw I.
      • Botden S.M.B.I.
      Development of a posterior sagittal anorectal surgical teaching model.
      ], the EoSim evaluated by Retrosi et al. [
      • Retrosi G.
      • Cundy T.
      • Haddad M.
      • Clarke S.
      Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
      ] and the LaparoscopyBoxx by Bökkerink et al. [
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Verhoeven B.H.
      • de Blaauw I.
      • Botden S.M.B.I.
      Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
      ]. For only eleven simulation models the authors provided written description or videos detailed enough to replicate the simulation model at-home: an ECMO cannulation model by Thompson et al., a model for CDH repair by Reino-Pires et al., a model for EA repair and a model for CDH repair by Bökkerink et al., a simulation model for inguinal and diaphragmatic defects by Ljuhar et al., an animal model for pleura empyema by Marecos et al., a fundoplication model by Esposito et al., a pyloromyotomy model by Ballouhey et al., a model for pyeloplasty by Rod et al., a model for bile duct exploration by Santos et al., a model for ureteral reimplantation by Millán et al. and the neonatal laparoscopic surgery model by Thompson et al. [
      • Thompson J.L.
      • Grisham L.M.
      • Scott J.
      • Mogan C.
      • et al.
      Construction of a reusable, high-fidelity model to enhance extracorporeal membrane oxygenation training through simulation.
      ,
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Bökkerink G.M.J.
      • Joosten M.
      • Leijte E.
      • Lindeboom M.Y.A.
      • Blaauw I.
      • Botden S.M.B.I.
      Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ,
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ,
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ,
      • Rod J.
      • Marret J.B.
      • Kohaut J.
      • Aigrain Y.
      • Jais J.P.
      • de Vries P.
      • Lortat-Jacob S.
      • Breaud J.
      • Blanc T.
      Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
      ,
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Millán C.
      • Rey M.
      • Lopez M.
      LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
      ,
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ]. Moreover, most MIS simulation models were developed for use in a box simulator or needed a laparoscopic camera for use [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Marecos M.C.
      • Torres R.A.
      • Bailez M.M.
      • Vagni R.L.
      • Klappenbach R.F.
      Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
      ,
      • Ballouhey Q.
      • Micle L.
      • Grosos C.
      • Robert Y.
      • Binet A.
      • Arnaud A.
      • Abbo O.
      • Lardy H.
      • Longis B.
      • Bréaud J.
      • Fourcade L.
      A simulation model to support laparoscopic pyloromyotomy teaching.
      ,
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Zimmermann P.
      • Wiseman A.X.
      • Sanchez O.
      • et al.
      The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
      ,
      • Obata S.
      • Ieiri S.
      • Uemura M.
      • Jimbo T.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Hashizume M.
      • Taguchi T.
      An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
      ,
      • Harada K.
      • Takazawa S.
      • Tsukuda Y.
      • Ishimaru T.
      • Sugita N.
      • Iwanaka T.
      • Mitsuishi M.
      Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model.
      ,
      • Takazawa S.
      • Ishimaru T.
      • Harada K.
      • Deie K.
      • Fujishiro J.
      • Sugita N.
      • Mitsuishi M.
      • Iwanaka T.
      Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • Chin A.C.
      Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
      ,
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ,
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      ,
      • Ieiri S.
      • Ishii H.
      • Souzaki R.
      • Uemura M.
      • Tomikawa M.
      • Matsuoka N.
      • Takanishi A.
      • Hashizume M.
      • Taguchi T.
      Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication.
      ,
      • Jimbo T.
      • Ieiri S.
      • Obata S.
      • Uemura M.
      • Souzaki R.
      • Matsuoka N.
      • Katayama T.
      • Masumoto K.
      • Hashizume M.
      • Taguchi T.
      A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system.
      ,
      • Williams A.
      • McWilliam M.
      • Ahlin J.
      • Davidson J.
      • Quantz M.A.
      • Bütter A.
      A simulated training model for laparoscopic pyloromyotomy: is 3D printing the way of the future?.
      ,
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator.
      ,
      • Schwab B.
      • Rooney D.M.
      • Hungness E.S.
      • Barsness K.A.
      Preliminary evaluation of a laparoscopic common bile duct simulator for pediatric surgical education.
      ,
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      ,
      • Cheung C.L.
      • Looi T.
      • Lendvay T.S.
      • Drake J.M.
      • Farhat W.A.
      Use of 3-dimensional printing technology and silicone modeling in surgical simulation: development and face validation in pediatric laparoscopic pyeloplasty.
      , possibly making it more difficult to use at-home or resulting in additional costs.

      4.3 Incorporation of simulation-based training and future perspectives

      In order to make optimal use of simulation based training, face, content and construct validity of the simulation models used should be established. Only when simulation models are validated in an adequate fashion the optimal effect of simulation-based training can be expected.
      Preferably, simulation models are either commercially available or easily constructed and are optimized for use at-home. Inanimate simulation models that do not require expensive simulation towers or laparoscopic cameras are therefore preferred. Ideally simulation models are utilised to minimize potential risks to patients by having trainees learn part of a new procedure away from the clinical setting. Efforts should be made to implement existing and future validated procedure-specific simulation models for pediatric surgical procedures into the training curricula. By doing so, pediatric surgical trainees are given the opportunity to acquire the skill unique to pediatric surgery. Furthermore, procedures that are rare and infrequent in the clinical setting may be trained to retain the needed skills.
      Future research may focus on evaluation of construct validity and assessment methods, preferably both objective and automatically generated feedback. Furthermore, future research should focus on developing more simulation models for open surgical procedures to make them widespread available for use.

      5. Limitations

      The quality of the included studies varied. Overall, there was a lack of this consistency in the terminology used regarding validation and almost all studies lacked a power analysis. A major flaw in several studies was the lack of including experts in the validation process [
      • Reino-Pires P.
      • Lopez M.
      Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
      ,
      • Ljuhar D.
      • Alexander S.
      • Martin S.
      • et al.
      The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
      ,
      • Esposito C.
      • Escolino M.
      • Draghici I.
      • Cerulo M.
      • Farina A.
      • De Pascale T.
      • Cozzolino S.
      • Settimi A.
      Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
      ,
      • Santos B.F.
      • Reif T.J.
      • Soper N.J.
      • Nagle A.P.
      • Rooney D.M.
      • Hungness E.S.
      Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
      ,
      • Usón-Casaús J.
      • Pérez-Merino E.M.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Sánchez-Margallo F.M.
      Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Pérez-Merino E.M.
      • Usón-Casaús J.M.
      • Zaragoza-Bayle C.
      • Rivera-Barreno R.
      • Rodríguez-Alarcón C.A.
      • Palme R.
      • Sánchez-Margallo F.M.
      Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
      ,
      • Maricic M.A.
      • Bailez M.M.
      • Rodriguez S.P.
      Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
      ,
      • Deie K.
      • Ishimaru T.
      • Takazawa S.
      • Harada K.
      • Sugita N.
      • Mitsuishi M.
      • Fujishiro J.
      • Iwanaka T.
      Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
      ,
      • Plymale M.
      • Ruzic A.
      • Hoskins J.
      • French J.
      • Skinner S.C.
      • Yuhas M.
      • Davenport D.
      • Iocono J.A.
      A middle fidelity model is effective in teaching and retaining skill set needed to perform a laparoscopic pyloromyotomy.
      ,
      • Ordorica-Flores R.
      • Orpinel-Armendariz E.
      • Rodríguez-Reyna R.
      • Pérez-Escamirosa F.
      • Castro-Luna R.
      • Minor-Martínez A.
      • Nieto-Zermeño J.
      Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
      ,
      • Barsness K.A.
      • Rooney D.M.
      • Davis L.M.
      • O'Brien E.
      Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator.
      ,
      • Bacarese-Hamilton J.
      • Pena V.
      • Haddad M.
      • Clarke S.
      Simulation in the early management of gastroschisis.
      ,
      • Schwab B.
      • Rooney D.M.
      • Hungness E.S.
      • Barsness K.A.
      Preliminary evaluation of a laparoscopic common bile duct simulator for pediatric surgical education.
      ,
      • Trudeau M.O.
      • Carrillo B.
      • Nasr A.
      • Gerstle J.T.
      • Azzie G.
      Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
      , whereas experts are needed to establish validation.
      Pediatric surgery is a generic term used to delineate a variety of subspecialties based on the age of the patients. This review focusses on what general pediatric surgery entails in Europe according to the European Training Requirements, however, this may vary locally. Therefore, this review may potentially omit relevant models which were not evaluated according to the for mentioned definition.
      For the assessment of specific pediatric surgical skills it is important that the assessment method used has shown construct validity. However, this review focusses on the simulation models rather than on assessment methods. Therefore, it focusses on whether these simulation models resemble the clinical procedure (face validity) and have proper teaching capabilities (content validity). The next step will be to evaluate proper assessment methods and construct validity.

      6. Conclusion

      This review included both inanimate and animal models, which showed that there are currently adequate inanimate alternatives for animal models. The number of (inanimate) simulation models for specific pediatric surgery procedures is growing, although the validation process varied heavily between simulation models and only twenty-two met adequate face and/or content validity scores. Additionally, only five were available for purchase and eleven were replicable, using instructions from the articles, which indicates a need for more efforts to develop adequate simulation models and make these available for widespread use.

      Declaration of Competing Interest

      None declared.

      References

        • Yokoyama S.
        • Mizunuma K.
        • Kurashima Y.
        • et al.
        Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review.
        Pediatr Surg Int. 2019; 35: 1085-1094https://doi.org/10.1007/s00383-019-04539-5
        • Patel E.A.
        • Aydın A.
        • Desai A.
        • Dasgupta P.
        • Ahmed K.
        Current status of simulation-based training in pediatric surgery: a systematic review.
        J Pediatr Surg. 2019; 54: 1884-1893
        • Beasley S.W.
        The challenges facing training in pediatric surgery worldwide.
        Front Pediatr. 2013; 1 (Published 2013 Sep 11): 24https://doi.org/10.3389/fped.2013.00024
        • Gostlow H.
        • Marlow N.
        • Babidge W.
        • Maddern G.
        Systematic review of voluntary participation in simulation-based laparoscopic skills training: motivators and barriers for surgical trainee attendance.
        J Surg Educ. 2017; 74 (Mar-Apr): 306-318https://doi.org/10.1016/j.jsurg.2016.10.007
        • Oquendo Y.A.
        • Riddle E.W.
        • Hiller D.
        • Blinman T.A.
        • Kuchenbecker K.J.
        Automatically rating trainee skill at a pediatric laparoscopic suturing task.
        Surg Endosc. 2018; 32 (Apr): 1840-1857https://doi.org/10.1007/s00464-017-5873-6
        • Dawe S.R.
        • Windsor J.A.
        • Broeders J.A.
        • et al.
        A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy.
        Ann Surg. 2014; 259: 236-248
        • Ouzzani M.
        • Hammady H.
        • Fedorowicz Z.
        • Elmagarmid A.
        Rayyan — a web and mobile app for systematic reviews.
        Syst Rev. 2016; 5: 210https://doi.org/10.1186/s13643-016-0384-4
        • UEMS
        Training requirements for the specialty of pediatric surgery, European standards of postgraduate medical specialist training.
        2014 (Available at) ([accessed June 15th])
        • Schmedding A.
        • Rolle U.
        • Czauderna P.
        Pediatric surgical training.
        Eur J Pediatr Surg. 2017; 27: 245-250https://doi.org/10.1055/s-0037-1603102
        • Yusoff M.S.B.
        ABC of content validation and content validity index calculation.
        Educ Med J. 2019; 11: 49-54https://doi.org/10.21315/eimj2019.11.2.6
        • McDougall E.M.
        Validation of surgical simulators.
        J Endourol. 2007; 21: 244-247
        • Van Nortwick S.S.
        • Lendvay T.S.
        • Jensen A.R.
        • et al.
        Methodologies for establishing validity in surgical simulation studies.
        Surgery. 2010; 147: 622-630
        • Carter F.J.
        • Schijven M.P.
        • Aggarwal R.
        • Grantcharov T.
        • Francis N.K.
        • Hanna G.B.
        • et al.
        Consensus guidelines for validation of virtual reality surgical simulators.
        Surg Endosc. 2005; 19: 1523-1532
        • Thompson J.L.
        • Grisham L.M.
        • Scott J.
        • Mogan C.
        • et al.
        Construction of a reusable, high-fidelity model to enhance extracorporeal membrane oxygenation training through simulation.
        Adv Neonatal Care. 2014; 14 (PMID24675629): 103-109https://doi.org/10.1097/ANC.0000000000000054
        • Reino-Pires P.
        • Lopez M.
        Validation of a low-cost do-it-yourself model for neonatal thoracoscopic congenital diaphragmatic hernia repair.
        J Surg Educ. 2018; 75 (NovEpub 2018 Apr 20. PMID: 29685785): 1658-1663https://doi.org/10.1016/j.jsurg.2018.04.005
        • Bökkerink G.M.J.
        • Joosten M.
        • Leijte E.
        • Lindeboom M.Y.A.
        • Blaauw I.
        • Botden S.M.B.I.
        Validation of low-cost models for minimal invasive surgery training of congenital diaphragmatic hernia and esophageal atresia.
        J Pediatr Surg. 2021; 56: 465-470https://doi.org/10.1016/j.jpedsurg.2020.05.045
        • Ljuhar D.
        • Alexander S.
        • Martin S.
        • et al.
        The laparoscopic inguinal and diaphragmatic defect (LIDD) model: a validation study of a novel box trainer model.
        Surg Endosc. 2018; 32: 4813-4819https://doi.org/10.1007/s00464-018-6232-y
        • Marecos M.C.
        • Torres R.A.
        • Bailez M.M.
        • Vagni R.L.
        • Klappenbach R.F.
        Pediatric thoracoscopic training in an experimental pleural empyema rabbit model.
        J Laparoendosc Adv Surg Tech A. 2006; 16 (Aug): 397-399https://doi.org/10.1089/lap.2006.16.397
        • Esposito C.
        • Escolino M.
        • Draghici I.
        • Cerulo M.
        • Farina A.
        • De Pascale T.
        • Cozzolino S.
        • Settimi A.
        Training models in pediatric minimally invasive surgery: rabbit model versus porcine model: a comparative study.
        J Laparoendosc Adv Surg Tech. 2016; (Jan): 79-84https://doi.org/10.1089/lap.2015.0229
        • Ballouhey Q.
        • Micle L.
        • Grosos C.
        • Robert Y.
        • Binet A.
        • Arnaud A.
        • Abbo O.
        • Lardy H.
        • Longis B.
        • Bréaud J.
        • Fourcade L.
        A simulation model to support laparoscopic pyloromyotomy teaching.
        J Laparoendosc Adv Surg Tech A. 2018; 28 (Jun): 760-765https://doi.org/10.1089/lap.2017.0263
        • Rod J.
        • Marret J.B.
        • Kohaut J.
        • Aigrain Y.
        • Jais J.P.
        • de Vries P.
        • Lortat-Jacob S.
        • Breaud J.
        • Blanc T.
        Low-cost training simulator for open dismembered pyeloplasty: development and face validation.
        J Surg Educ. 2018; 75 (Jan-FebEpub 2017 Aug 1): 188-194https://doi.org/10.1016/j.jsurg.2017.06.010
        • Santos B.F.
        • Reif T.J.
        • Soper N.J.
        • Nagle A.P.
        • Rooney D.M.
        • Hungness E.S.
        Development and evaluation of a laparoscopic common bile duct exploration simulator and procedural rating scale.
        Surg Endosc. 2012; 26 (Sep): 2403-2415https://doi.org/10.1007/s00464-012-2213-8
        • Millán C.
        • Rey M.
        • Lopez M.
        LAParoscopic simulator for pediatric ureteral reimplantation (LAP-SPUR) following the Lich-Gregoir technique.
        J Pediatr Urol. 2018; 14 (Apr): 137-143https://doi.org/10.1016/j.jpurol.2017.11.020
        • Zimmermann P.
        • Wiseman A.X.
        • Sanchez O.
        • et al.
        The avian model: a novel and cost-effective animal tissue model for training in neonatal laparoscopic surgery.
        J Ped Endosc Surg. 2019; 1: 99-105https://doi.org/10.1007/s42804-019-00027-8
        • Botden S.M.B.I.
        • Bökkerink G.M.
        • Leijte E.
        • et al.
        Training the component steps of an extra-corporeal membrane oxygenation (ECMO) cannulation outside the clinical setting.
        J Artif Organs. 2020; 23: 328-334https://doi.org/10.1007/s10047-020-01176-x
        • van Ling J.A.
        • Bökkerink G.M.J.
        • de Blaauw I.
        • Botden S.M.B.I.
        Development of a posterior sagittal anorectal surgical teaching model.
        BMC Pediatr. 2021; 21 (Jan 27): 57https://doi.org/10.1186/s12887-021-02514-5
        • Retrosi G.
        • Cundy T.
        • Haddad M.
        • Clarke S.
        Motion analysis-based skills training and assessment in pediatric laparoscopy: construct, concurrent, and content validity for the eoSim simulator.
        J Laparoendosc Adv Surg Tech A. 2015; 25 (Nov): 944-950https://doi.org/10.1089/lap.2015.0069
        • Bökkerink G.M.J.
        • Joosten M.
        • Leijte E.
        • Verhoeven B.H.
        • de Blaauw I.
        • Botden S.M.B.I.
        Take-home laparoscopy simulators in pediatric surgery: is more expensive better?.
        J Laparoendosc Adv Surg Tech A. 2021; 31: 117-123https://doi.org/10.1089/lap.2020.0533
      1. https://www.pediatrickboxx.com [accessed 10th of June 2021

        • Obata S.
        • Ieiri S.
        • Uemura M.
        • Jimbo T.
        • Souzaki R.
        • Matsuoka N.
        • Katayama T.
        • Hashizume M.
        • Taguchi T.
        An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair.
        J Laparoendosc Adv Surg Tech A. 2015; 25 (SepEpub 2015 May 27. PMID: 26375773): 775-781https://doi.org/10.1089/lap.2014.0259
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        The development and evaluation of a novel thoracoscopic diaphragmatic hernia repair simulator.
        J Laparoendosc Adv Surg Tech A. 2013; 23 (AugEpub 2013 Jun 22. PMID: 23789735): 714-718https://doi.org/10.1089/lap.2013.0196
        • Usón-Casaús J.
        • Pérez-Merino E.M.
        • Rivera-Barreno R.
        • Rodríguez-Alarcón C.A.
        • Sánchez-Margallo F.M.
        Evaluation of a Bochdalek diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
        J Laparoendosc Adv Surg Tech A. 2014; 24 (Apr): 280-285https://doi.org/10.1089/lap.2013.0358
        • Pérez-Merino E.M.
        • Usón-Casaús J.M.
        • Zaragoza-Bayle C.
        • Rivera-Barreno R.
        • Rodríguez-Alarcón C.A.
        • Palme R.
        • Sánchez-Margallo F.M.
        Development of an optimal diaphragmatic hernia rabbit model for pediatric thoracoscopic training.
        Exp Anim. 2014; 63 (PMID: 24521868; PMCID: PMC4160932): 93-98https://doi.org/10.1538/expanim.63.93
        • Harada K.
        • Takazawa S.
        • Tsukuda Y.
        • Ishimaru T.
        • Sugita N.
        • Iwanaka T.
        • Mitsuishi M.
        Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model.
        Minim Invasive Ther Allied Technol. 2015; 24: 226-232https://doi.org/10.3109/13645706.2014.996161
        • Takazawa S.
        • Ishimaru T.
        • Harada K.
        • Deie K.
        • Fujishiro J.
        • Sugita N.
        • Mitsuishi M.
        • Iwanaka T.
        Pediatric thoracoscopic surgical simulation using a rapid-prototyped chest model and motion sensors can better identify skilled surgeons than a conventional box trainer.
        J Laparoendosc Adv Surg Tech A. 2016; 26 (SepEpub 2016 Jul 19): 740-747https://doi.org/10.1089/lap.2016.0131
        • Al-Qadhi S.A.
        • Pirie J.R.
        • Constas N.
        • Corrin M.S.
        • Ali M.
        An innovative pediatric chest tube insertion task trainer simulation: a technical report and pilot study.
        Simul Healthc. 2014; 9 (Oct): 319-324https://doi.org/10.1097/SIH.0000000000000033
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        • O'Brien E.
        Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator.
        J Laparoendosc Adv Surg Tech A. 2015; 25 (MayEpub 2014 Dec 23): 429-434https://doi.org/10.1089/lap.2014.0364
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        Collaboration in simulation: the development and initial validation of a novel thoracoscopic neonatal simulator.
        J Pediatr Surg. 2013; 48 (Jun): 1232-1238https://doi.org/10.1016/j.jpedsurg.2013.03.015
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        • Chin A.C.
        Validation of measures from a thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
        J Pediatr Surg. 2014; 49 (Jandiscussion 32-3): 29-32https://doi.org/10.1016/j.jpedsurg.2013.09.069
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        • O'Brien E.
        Evaluation of three sources of validity evidence for a synthetic thoracoscopic esophageal atresia/tracheoesophageal fistula repair simulator.
        J Laparoendosc Adv Surg Tech A. 2015; 25 (Jul): 599-604https://doi.org/10.1089/lap.2014.0370
        • Maricic M.A.
        • Bailez M.M.
        • Rodriguez S.P.
        Validation of an inanimate low cost model for training minimal invasive surgery (MIS) of esophageal atresia with tracheoesophageal fistula (AE/TEF) repair.
        J Pediatr Surg. 2016; 51 (Sep): 1429-1435https://doi.org/10.1016/j.jpedsurg.2016.04.018
        • Deie K.
        • Ishimaru T.
        • Takazawa S.
        • Harada K.
        • Sugita N.
        • Mitsuishi M.
        • Fujishiro J.
        • Iwanaka T.
        Preliminary study of video-based pediatric endoscopic surgical skill assessment using a neonatal esophageal atresia/tracheoesophageal fistula model.
        J Laparoendosc Adv Surg Tech A. 2017; 27 (JanEpub 2016 Nov 18): 76-81https://doi.org/10.1089/lap.2016.0214
        • Ieiri S.
        • Ishii H.
        • Souzaki R.
        • Uemura M.
        • Tomikawa M.
        • Matsuoka N.
        • Takanishi A.
        • Hashizume M.
        • Taguchi T.
        Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: suture ligature model of the crura of the diaphragm in infant fundoplication.
        Pediatr Surg Int. 2013; 29 (May): 501-504https://doi.org/10.1007/s00383-013-3276-x
        • Jimbo T.
        • Ieiri S.
        • Obata S.
        • Uemura M.
        • Souzaki R.
        • Matsuoka N.
        • Katayama T.
        • Masumoto K.
        • Hashizume M.
        • Taguchi T.
        A new innovative laparoscopic fundoplication training simulator with a surgical skill validation system.
        Surg Endosc. 2017; 31 (Apr): 1688-1696https://doi.org/10.1007/s00464-016-5159-4
        • Williams A.
        • McWilliam M.
        • Ahlin J.
        • Davidson J.
        • Quantz M.A.
        • Bütter A.
        A simulated training model for laparoscopic pyloromyotomy: is 3D printing the way of the future?.
        J Pediatr Surg. 2018; 53 (MayEpub 2018 Feb 8): 937-941https://doi.org/10.1016/j.jpedsurg.2018.02.016
        • Plymale M.
        • Ruzic A.
        • Hoskins J.
        • French J.
        • Skinner S.C.
        • Yuhas M.
        • Davenport D.
        • Iocono J.A.
        A middle fidelity model is effective in teaching and retaining skill set needed to perform a laparoscopic pyloromyotomy.
        J Laparoendosc Adv Surg Tech A. 2010; 20 (Jul-Aug): 569-573https://doi.org/10.1089/lap.2009.0406
        • Skertich N.J.
        • Schimpke S.W.
        • Lee T.
        • Wiegmann A.L.
        • Pillai S.
        • Rossini C.
        • Madonna M.B.
        • Shah A.N.
        Pediatric surgery simulation-based training for the general surgery resident.
        J Surg Res. 2021; 258 (Feb): 339-344https://doi.org/10.1016/j.jss.2020.05.038
        • Ordorica-Flores R.
        • Orpinel-Armendariz E.
        • Rodríguez-Reyna R.
        • Pérez-Escamirosa F.
        • Castro-Luna R.
        • Minor-Martínez A.
        • Nieto-Zermeño J.
        Development and preliminary validation of a rabbit model of duodenal atresia for training in pediatric surgical skills.
        Surg Innov. 2019; 26 (Dec): 738-743https://doi.org/10.1177/1553350619881068
        • Barsness K.A.
        • Rooney D.M.
        • Davis L.M.
        • O'Brien E.
        Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator.
        J Laparoendosc Adv Surg Tech A. 2015; 25 (Mar): 256-260https://doi.org/10.1089/lap.2014.0358
        • Schwab B.
        • Rooney D.M.
        • Hungness E.S.
        • Barsness K.A.
        Preliminary evaluation of a laparoscopic common bile duct simulator for pediatric surgical education.
        J Laparoendosc Adv Surg Tech A. 2016; 26 (Oct): 831-835https://doi.org/10.1089/lap.2016.0248
        • Burdall O.C.
        • Makin E.
        • Davenport M.
        • Ade-Ajayi N.
        3D printing to simulate laparoscopic choledochal surgery.
        J Pediatr Surg. 2016; 51 (May): 828-831https://doi.org/10.1016/j.jpedsurg.2016.02.093
        • Bacarese-Hamilton J.
        • Pena V.
        • Haddad M.
        • Clarke S.
        Simulation in the early management of gastroschisis.
        Simul Healthc. 2013; 8 (Dec): 376-381https://doi.org/10.1097/SIH.0b013e31829b3ff9
        • Cheung C.L.
        • Looi T.
        • Lendvay T.S.
        • Drake J.M.
        • Farhat W.A.
        Use of 3-dimensional printing technology and silicone modeling in surgical simulation: development and face validation in pediatric laparoscopic pyeloplasty.
        J Surg Educ. 2014; 71 (Sep-Oct): 762-767https://doi.org/10.1016/j.jsurg.2014.03.001
        • Azzie G.
        • Gerstle J.T.
        • Nasr A.
        • Lasko D.
        • Green J.
        • Henao O.
        • Farcas M.
        • Okrainec A.
        Development and validation of a pediatric laparoscopic surgery simulator.
        J Pediatr Surg. 2011; 46 (May): 897-903https://doi.org/10.1016/j.jpedsurg.2011.02.026
        • Nasr A.
        • Carrillo B.
        • Gerstle J.T.
        • Azzie G.
        Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills.
        J Pediatr Surg. 2014; 49 (May): 791-794https://doi.org/10.1016/j.jpedsurg.2014.02.063
        • Trudeau M.O.
        • Carrillo B.
        • Nasr A.
        • Gerstle J.T.
        • Azzie G.
        Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator.
        J Laparoendosc Adv Surg Tech A. 2017; 27 (Apr): 441-446https://doi.org/10.1089/lap.2016.0516
        • Torres A.
        • Inzunza M.
        • Jarry C.
        • Serrano F.
        • Varas J.
        • Zavala A.
        Development and validation of a new laparoscopic endotrainer for neonatal surgery and reduced spaces.
        Arq Bras Cir Dig. 2021; 33 (Jan 25): e1559https://doi.org/10.1590/0102-672020200004e1559
        • Van Nortwick S.S.
        • Lendvay T.S.
        • Jensen A.R.
        • Wright A.S.
        • Horvath K.D.
        • Kim S.
        Methodologies for establishing validity in surgical simulation studies.
        Surgery. 2010; 147 (MayEpub 2009 Dec 16. PMID: 20015529): 622-630https://doi.org/10.1016/j.surg.2009.10.068
        • Sireci S.
        • Faulkner-Bond M.
        Validity evidence based on test content.
        Psicothema. 2014; 26: 100-107
        • Boone H.N.
        • Boone D.A.
        Analyzing Likert data.
        J Ext. 2012; 50
        • Joshi A.
        • et al.
        Likert scale: explored and explained.
        Curr J Appl Sci Technol. 2015; : 396-403
        • Patel E.A.
        • Aydın A.
        • Desai A.
        • Dasgupta P.
        • Ahmed K.
        Current status of simulation-based training in pediatric surgery: a systematic review.
        J Pediatr Surg. 2019; 54 (SepEpub 2018 Dec 8. PMID: 30573294): 1884-1893https://doi.org/10.1016/j.jpedsurg.2018.11.019
        • Barsness K.
        Simulation-based education and performance assessments for pediatric surgeons.
        Eur J Pediatr Surg. 2014; 24 (AugEpub 2014 Aug 21. PMID: 25144355): 303-307https://doi.org/10.1055/s-0034-1386650
        • Sharma D.
        • Agrawal V.
        • Bajaj J.
        • Agarwal P.
        Low-cost simulation systems for surgical training: a narrative review.
        J Surg Simul. 2020; 7: 33-52https://doi.org/10.1102/2051-7726.2020.0005
        • Mortell A.
        • Montedonico S.
        • Puri P.
        Animal models in pediatric surgery.
        Pediatr Surg Int. 2006; 22: 111-128
        • Barsness K.A.
        Trends in technical and team simulations: challenging the status quo of surgical training.
        Semin Pediatr Surg. 2015; 24: 130-133
        • Bergmeister K.D.
        • Aman M.
        • Kramer A.
        • et al.
        Simulating surgical skills in animals: systematic review, costs acceptance analyses.
        Front Vet Sci. 2020; 7 (Published 2020 Sep 30)570852https://doi.org/10.3389/fvets.2020.570852
        • Massoth C.
        • Röder H.
        • Ohlenburg H.
        • et al.
        High-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students.
        BMC Med Educ. 2019; 19: 29https://doi.org/10.1186/s12909-019-1464-7
        • Stefanidis D.
        • Sevdalis N.
        • Paige J.
        • Zevin B.
        • Aggarwal R.
        • Grantcharov T.
        • Jones D.
        MS‖ for the association for surgical education simulation committee simulation in surgery.
        Ann Surg. 2015; (MayVolume 261 - Issue 5 - p): 846-853https://doi.org/10.1097/SLA.0000000000000826
      2. Tam PKHLaparoscopic surgery in childrenArchives of Disease.
        Child. 2000; 82: 240-243https://doi.org/10.1136/adc.82.3.240