Abstract

Objective

This study aims to evaluate and improve the cadaver dissection training program given by the Turkish Otorhinolaryngology and Head and Neck Surgery Association’s resident school. The program addresses goals, outcomes, content, implementation sequence, educational strategies, and the steps for evaluation and regulation.

Methods

The Kern educational program development model was used by the 2023-2025 ENT Resident School Management for cadaver dissection training: both qualitative and quantitative research methods were employed. A needs analysis was conducted with structured focus interviews with the 12 students accepted into the course. Dissection steps, educational resources, and dissection videos were shared with residents before the training. A personalized cadaver dissection training plan, aligned with the Curriculum Development and Standardization System, was developed and shared with the instructors. Post-training, the Kirkpatrick program evaluation model was used to conduct first and second-level program evaluations.

Results

The first-level evaluation revealed high satisfaction with the training program, and the second-level evaluation indicated a statistically significant increase between pre-test and post-test scores (p=0.015). There was a significant positive correlation between the total mastoidectomy success scale scores and the difference scores (pre- and post-test) (p=0.019; r=0.663), while no significant correlation was found for the endoscopic sinus surgery success scale scores (p=0.996).

Conclusion

The Resident School, developed by the members of the Turkish Otorhinolaryngology and Head and Neck Surgery Association, is a high-participation, skill-intensive training program. The program, conducted with great dedication, received high participant satisfaction and assessed knowledge and skill learning levels.

Introduction

Program development in medical education involves the systematic process of designing, developing, testing, implementing, evaluating, and refining a program. Evaluating the effectiveness and success of programs implemented in postgraduate education is crucial for program improvement (1). Program evaluation involves collecting, analyzing, and interpreting data to assign value to a specific educational activity (2). It determines the program’s purpose, model, process, management, and principles and identifies factors that may affect the evaluation process and outcome. Stakeholders must carefully review these factors to make informed decisions about the program (3, 4).

The Resident School, established under the roof of the Turkish Society of Otorhinolaryngology and Head and Neck Surgery [Turkish Ear, Nose, Throat (ENT) and HNS], has been contributing to the education of ENT residents for many years. As with any educational program, it is necessary to evaluate whether the assistant school training meets its objectives, and to conduct evaluation studies to develop the program according to the needs changing over time (5). This study aims to develop, implement, and evaluate a cadaver dissection training program for the Resident School.

Methods

The Resident Shool cadaver dissection training program was developed by the school administration (also the authors) following the Kern curriculum development steps and the program was evaluated. The data of the study were evaluated retrospectively. This retrospective study was conducted following the approval of the Kocaeli University Ethics Committee (decision no: KÜ GOKAEK-2024/13.16, date: 22.08.2024).

Program Development

The cadaver dissection training program for the ENT Resident School was developed using the Kern program development steps (6).

1. Problem Identification: Cadaver dissection training is a crucial step for achieving interventional competencies in specialty education. The Turkish ENT and HNS Association has been supporting ENT resident education through cadaver dissection courses for many years. However, the rapid increase in the number of residents in recent years poses a significant threat to the standardization of skill training. Therefore, it was necessary to revisitthe training provided in the Resident School, define its outcomes, and evaluate whether the desired goals were achieved after the training.

2. Needs Assessment: A needs assessment was planned after the Resident School was announced and applications were received. Since the participants come from different training centers and have different seniority, students’ needs and expectations for school education may also differ. A focus group discussion was conducted with the residents prior to the training during the interview, the researcher first introduced himself, shared information about the purpose of the interview and how long the interview would last, and verbal approval was obtained for recording. Using the Curriculum Development and Standardization System (CDSS) as a basis, qualitative data were obtained through online structured focus group interviews to identify each resident’s educational deficiencies (Table 1).

3. Goals and Learning Objectives: The goals and learning objectives of the Resident School were defined following the 2019 version 2.4 TUKMOS interventional competency levels, considering the seniority levels of the participants.

4. Educational Strategies: Before the cadaver training, the Resident School management shared step-by-step surgical dissection guides, related books, online educational materials, and master dissection video recordings with the accepted students.

5. Program Implementation: The cadaver dissection was conducted at the TORLAK Surgical Anatomy and Distance Education Center on dates scheduled by the school management and the Turkish ENT and HNS Association. The managerial aspect of the program involved internal Resident School Management and external support (ENT School Instructors) at every stage. Expert instructors provided one-on-one dissection training to students during the dissection sessions.

6. Program Evaluation: The first- and second-level evaluations of the cadaver dissection training program were conducted using the Kirkpatrick program evaluation model (7). At the end of the training, all participants were asked to anonymously complete an online satisfaction survey. At the end of the training program, all participants were asked to evaluate the program in every aspect according to the propositions A multiple-choice test was administered before and after the training. The results of pre-test and post-test comparisons from online and face-to-face training sessions were used for the second-level evaluation. Additionally, the endoscopic sinus surgery (ESS) and mastoidectomy success scales were administered to assess skill training (8, 9).

Statistical Analysis

Qualitative data were collected during focus group interviews, and video recordings were transcribed. Researchers coded the data, combined statements with similar meanings, and created themes. These themes were verified through participant feedback. Data analysis was performed using IBM SPSS Statistics for Windows version 25.0. Qualitative variables were presented as frequency and percentage. Normal distribution of quantitative variables was tested using the Shapiro-Wilk test. For variables with normal distribution, arithmetic mean and standard deviation values were presented; for non-normally distributed variables, median, minimum, and maximum values were presented. The paired sample t-test was used to compare pre-test and post-test results. Pearson correlation was used to compare difference scores with total scores on the mastoidectomy and ESS success scales. A Type I error rate of 0.05 was considered.

Results

Participant Focus Group Results

Twelve students were invited to the focus group interviews, conducted in two sessions. Demographic characteristics of the students are summarized in Table 2. Qualitative data examples summarizing the students’ expectations from the training are provided.

Examples of qualitative data summarizing students’ expectations from education in needs assessment analysis:

➤ “It would be good to see complications in the cadaver and intervene.”

➤ “I am coming to the end of my residency; I attended the course to see different approaches schools of thought.”

➤ “It will be good for understanding 3D anatomy.”

➤ “Facelift will be very popular in the future; I think it will be great to receive this training on cadavers at the resident level.”

➤ “I have not performed experienced mastoidectomy yet, I will do it on a cadaver first.”

➤ “There were subjects that I missed a lot due to Covid and the earthquake, I plan to complete them in this course.”

➤ “It will be useful for clinics where the addition of an endoscope is not performed in cadaveric ear surgery.”

Kirkpatrick Model Program Evaluation Results

First Level (Reaction): The participants’ responses indicating that they were generally satisfied with the training provided are shown in Table 3.

According to the students, the aspects of education that need to be improved are:

➤ “If more time was allocated for fascial plastic surgery, we could have performed do more procedures.”

➤ “When performing temporal bone dissection, there may be a transfer to the tower or a second microscope eye. Computed tomography images of the cadavers could have been made available.”

➤ “I just think that allocating a weekend to each subspecialty branch and teaching theoretical lessons in the presentation room beforehand will increase the benefit of the course for us.”

➤ “Course date could have been notified earlier.”

➤ “As ENT physicians, we see that in recent years there has been a trend towards rhinoplasty in ENT practice and good results of this trend along with education. However, unfortunately, I see that the focus on head and neck surgery and reconstruction is decreasing, and sometimes we have to cooperate with different departments or be dependent on different departments. I know that the number of clinics performing head and neck reconstruction (regional and free flaps), which is also included in our Specialty Board in Medicine core training program, is few compared to abroad. In this respect, I believe that organizing training (zoom lectures, meetings, live surgeries) and encouraging reconstruction will improve our healthcare service delivery as ENT physicians.”

➤ “The course program was very detailed. It was very instructive for us. Having two residents per cadaver was very valuable in completing all the steps. Surgical sets, microscopes and endoscopes were perfectly equipped. In order for the program to be more complete, the course duration could have been longer, or it could have been done in three separate departments: Functional endoscopic sinus surgery, otology and facial plastic. If the course program and content had been announced earlier, it would have been more beneficial for us in terms of studying and preparing. But even in its current state, it was incredibly educational and productive for residents.”

Instructor Feedback

Sixteen instructors took part in the training, with 12 providing feedback post-training. Demographic characteristics of the instructors are shown in Table 2. Satisfaction survey results are presented in Table 4.

Qualitative feedback results from trainers:

➤ I really liked that the residents were trained, ready and motivated, and that they had determined their own expectations from the dissection.

➤ Everything was very systematic, there were no problems.

➤ Resident expectations were determined correctly, and regular and planned dissection could be performed.

➤ The course was very well-organized, and it is obvious that care was taken: in order to include more people, the number of hands-on trainee participants can be increased, and additional trainees can be recruited for free/a symbolic fee to watch the dissection without touching the cadaver, and to participate in all other trainings: theoretical, video, etc. (2 per cadaver) (e.g., +2 trainees).

➤ Adding missing surgical instruments (Scissors, curettes and rongeur tips should be renewed).

Second Level (Learning): Significant improvement was observed between pre-test and post-test scores (pre-test 5.75±1.91, post-test 6.83±1.9, t=-2,862, p=0.015).

Skill Evaluation Results: The mastoidectomy and endoscopic sinus surgery (ESS) success scales were used to assess skill levels (Tables 5 and 6). There was a significant positive correlation between the total mastoidectomy success scale scores and the difference scores (pre-test and post-test). No significant correlation was found for the ESS success scale scores (p=0.996) (Table 7).

Discussion

In this study, we discuss the development steps of the Turkish ENT and HNS Association, Resident School cadaver dissection training program and its evaluation results. After the training, the program evaluation using the Kirkpatric model concluded that satisfaction was high and that it was a training that contributed to the resident’s education in terms of knowledge and skills.

A training program should be planned following a targeted purpose and be evaluated for whether it has achieved its goal (1, 2). When planning a training program, the needs of the participants should be determined first. The needs assessment analysis we conducted in our study showed that although the residents were similar in terms of seniority, their skill competency levels were different from each other. Such that, while some residents had performed all the steps of mastoidectomy and expected advanced surgery from this training, there were others who expected “I have not experienced mastoidectomy yet, I will do it on a cadaver first.” We determined each resident’s skill level and informed the trainers in advance and tried to implement the individualized education model.

Evaluation of a program provides very important feedback for the development and sustainability of the program. According to the Kirkpatrick program evaluation model, in the first stage, data can be collected to evaluate the program based on the participants’ perceptions, that is, their satisfaction or dissatisfaction with the educational training program (7). In our study, we applied a satisfaction survey that questioned every aspect of the program to understand whether the residents and trainers found the training program useful, and when we evaluated their answers to the propositions, results showed that they were satisfied with the program. Moreover, their comments at the end of the program showed that participants found the training very useful and that they wanted to participate in such a program again-indicating that the program was successful for the first level evaluation. In qualitative data, deficiencies in the training program (such as duration of the training program, program content, surgical instruments) were recorded as important data sources for the development and improvement of future programs.

A second level evaluation is carried out shortly after the end of the program to investigate the changes in knowledge, skills and attitudes. Objective evaluation methods, performance tests, and attitude scales can be used at this stage (7). It is recommended to conduct measurement and evaluation in both cognitive and skill areas at every stage of the program following the goals (1, 2). The Turkish ENT and HNS Association has been organizing pre- and post-graduation training programs under the name of ENT schools for more than ten years, but these have no systematic program evaluation. Regarding this subject, only Ecevit et al. (10) evaluated the two-year rhinology training program in terms of staff satisfaction and knowledge. In their study, the authors drew attention to the school’s learning objectives, appropriate educational models and measurement and evaluation deficiencies. In our study, in addition to cognitive evaluation, we evaluated the skill levels of the residents using surgical skill scales, unlike the existing literature. Cognitively, statistical success was achieved in the post-test analysis conducted before the start of the training and after the training. However, as stated by the students in the qualitative evaluation form, it was considered restrictive due to the delay in determining the student admission list and the short duration of the surgical dissection training. Therefore, we recommend paying attention to the duration of the program. Secondly, although a standard educational resource was offered to all students, it was up to the students to prepare for them or not. As stated by the students in the qualitative data results, extending the training over a long period of time and providing additional theoretical courses in the future will contribute to standardization.

Objective measures in skills-based training courses allow trainees and trainers to evaluate performance and monitor progress. For this purpose, we tried to measure skill training with success scales. Francis et al. (9) developed the Global Evaluation Scale and the mastoidectomy evaluation scale to evaluate ear surgery in a real environment. Kara (11) reviewed the validity and reliability of various tools, including the mastoidectomy evaluation scale, used in evaluating the skill training of residents. In our study, we evaluated the competency of the residents in cadaver dissection steps by using these validated scales. At the end of the dissection training, one trainer evaluated the two students with whom they performed the dissection. Therefore, the fact that six different trainers evaluated 12 residents was considered the most important limitation of the study in terms of objectivity and standardization. Again, the fact that there were residents with different levels of surgical skills (seniority, facilities of the clinic where they received training, interest of resident, etc.) also led to different scores. In general, as expected, while the residents were successful in the first steps of mastoidectomy, they showed the lowest score (2.6) in the opening of the digastric region (recognition of the digastric muscle, recognition of the stylomastoid foramen). Although demonstration of proficiency achievements has not yet become a requirement for graduation in otolaryngology specialty, we think that this stage of defining surgical goals will become mandatory in the coming years. Therefore, scales that describe and objectively evaluate surgical steps will be needed. In their article published in 2024, Jayaraman Patnaik et al. (12) reported a scale study evaluating mastoidectomy in cadaver dissection for residents. They conducted the study with 16 residents at a tertiary care teaching hospital and showed the internal consistency of the Likert scale assessment. Mowry et al. (13) evaluated residents with the scale they developed for temporal bone dissection. Residents participated in weekly dissections for nine months every year. Individual student scores for each dissection skill were monitored over time. The authors reported that this criterion they developed was easy to use and that scoring was consistent among evaluators. They also emphasized that this criterion was successful in distinguishing between those who know and those who do not. Further to the above-described, another important limitation of our study is that we did not monitor the development process of the residents in terms of skills or evaluate their immediate proficiency.

The residency training program should provide residents with the skills and competence to perform surgery safely. However, education opportunities in this field are not standard and equal for every education center. For this reason, residents can experience difficulties in improving their surgical skills in this field. Cadaver dissection courses offer the most realistic skill training opportunity to meet this need. Reports on objective measurement tools used for assessment and evaluation of skills are limited in the literature. Laeeq et al. (14) evaluated 17 residents in the Johns Hopkins ENT residency program while performing ESS in the operating room. In their study, the authors evaluated three steps, including maxillary antrostomy+anterior ethmoidectomy, posterior ethmoidectomy+sphenoidostomy, and frontal sinusotomy. Between 2009 and 2011, eight evaluators completed a total of 73 evaluations for 17 residents (seniority 2-5). As a result of the evaluation, they stated that the residents showed the lowest score in the ESS steps in the frontal sinusotomy. In our study, residents received higher scores, and the lowest score was for frontal sinusotomy, which is consistent with the literature.

Cadaver dissection training requires knowledge of surgical anatomy as well as skill in the use of surgical instruments and surgical technique. In our program evaluation, we found that the residents achieved significant success in the post-test results after training. When we evaluated the skill levels of the residents who improved themselves in terms of knowledge, we found that there was a high level of positive correlation between the results of the mastoidectomy success chart and the difference scores (pre- and post-test difference) (p=0.019; r=0.663), meaning that the students who achieved significant success in the post-test. They also demonstrated a high level of skill, and we observed that they received high scores. In general, ESS success scale scores were seen to be higher than mastoidectomy success scale scores. The reason for this may be that students perform this surgery more during their continuing education in their clinics or that the trainers working at ESS make more optimistic evaluations in scoring.

Another most important limitation of our study is that skill evaluations could not be performed because skill training regarding head and neck, and facial plastic is not implemented as a standard. Yet another limitation is that the results of the training program are according to the Kirkpatrick program evaluation model 3 (impact) and 4^th (results) step could not be evaluated^..

Conclusion

This report presents the first cadaver dissection training program, which was developed for ENT residents under the roof of the Turkish ENT and HNS Association and was evaluated using the Kirkpatrick model. Statistically significant results at each evaluation level indicated the program’s success. We believe that there is a need for long-term, standardized training program development and evaluation studies for the Resident School, where skills are evaluated with objective measurement and evaluation methods and the reflections of the training program on the field can be closely monitored in the long term.