ABSTRACT

Aims:

The pandemic has increased the importance of remote teaching resources in medical education and highlighted the importance of out-of-class learning in a hybrid setting. Achieving such a model seems to be challenging, especially for institutions with limited experience and resources. This study aims to demonstrate the educational benefit of such hybrid curricula by using simple modules, reproducible in various settings even with limited resources.

Methods:

All fifth-year medical students at our institution (n=43) were included in this study. Their original random grouping to take internships at different times at our institution was utilized. The first group to undergo their psychiatry internship was registered as a control group (group 1, n=20) and the latter as an intervention group (group 2, n=23). First, an initial need assessment was administered to both groups before their internships to guide the design of the intervention. According to the needs identified, we came up with a plan consisting of clips and simple animations corresponding to three disorders in psychiatry. We integrated this module into the curriculum of the intervention group and checked its efficacy using the pre/post-survey method. We later compared the two groups for knowledge retention, self-evaluated sufficiency, and satisfaction with multiple-choice questions and a late-post survey. Statistical significance within the intervention group had been determined by dependent samples t-test whereas it was determined by independent samples t-test between the two groups, following normality analysis by the Shapiro-Wilk test.

Results:

In the initial need assessment, Likert scores (1-5) of both groups showed agreement with low concentration [mean =4.3 (±0.85) and 4.1 (±0.85)] and stated disagreement about “patient variety,” [mean =2.5 (±1.28) and 1.8 (±0.94)] and “management” [mean =3.0 (±1.15) and 2.9 (±1.01)]. After the visual display, Likert scores of the intervention group improved significantly in certain items reflecting self-sufficiency, and the post-survey had more correct answers (+21.5%, p=0.017). When the two groups were compared, the intervention group answered more questions correctly (+12.6%, p=0.058). They also stated benefits in memory, exam-preparedness, and sufficiency.

Conclusion:

Simplistic, affordable, and easily prepared visual supplementation can offer an improvement in quality and increase student satisfaction with online teaching.

INTRODUCTION

The coronavirus disease-2019 pandemic has made it compulsory to transition into remote teaching techniques in a rapid trial and error method (1-3). Medical schools tried to incorporate different methods within their online curricula such as online simulations with simulated patients or supervised virtual consultations (4-7). In fields like psychiatry where student-patient interaction has become limited, the use of such add-ons has been reported to be fruitful in terms of both satisfaction and knowledge retention (8-10).

However, in countries such as Türkiye, most institutions could not transition fast enough and had to either postpone their lectures indefinitely or move on with two-dimensional slide-based online sessions (11). In those countries where there is underreporting and general dependence on face-to-face conventional teaching, it can be hard to estimate the real repercussions of this situation on students (11). There are limited reports around the world about the negative effects the students face in resource-limited settings such as the lack of “social presence,” limited patient encounters, and decreased sense of clinical competency and motivation among the students (6, 12, 13). This can be worrisome as it can create negative prejudice towards innovative teaching methods and technology in medical education (11-13). Under the influence of such a negative connotation, graduates may not be equipped well with the requirements of modern medical practice (14).

Almost three years after the onset of the pandemic, face-to-face medical education ensues today, but remote teaching methods also continue to proliferate. It has therefore become important for institutions to translate the experience they gained during the pandemic into practice now to catch up with the state-of-the-art level of teaching. This can be especially hard for those institutions that could not incorporate such methods into their curricula (11-13). That is why we conducted this study on remote teaching to determine student demands, self-reported insufficiencies and respond with an intervention plan that is applicable and efficient. Specifically, we believe that the diversification of the education material by using simplistic visual supplementation can yield to improvements -large or small- in online medical education even in resource-limited settings. We would like to demonstrate that such innovations need not be heavily dependent on resources and experience but can still influence medical learning even after the pandemic.

MATERIAL AND METHODS

Procedure

We received formal approval from the Koç University Ethics Committee on 25 February 2021, before conducting this study (decision no: 2021.098.IRB3.059). Our study involved all the medical students in their fifth year -totaling 43- who were to begin their psychiatry rotations at different times. This approach to include most students -if not all- in one term has been demonstrated in the literature for similar interventions (7, 8, 10). Since fifth-year medical students are already randomly divided into two groups in our institution, we decided to study each group separately and designated the first group as a control group and the second group as an intervention group (Figure 1). Enrollment within the analysis was voluntary and formal consent was given by all student participants.

Figure 1

During the planning phase, we first devised an initial needs assessment. It targeted both the psychiatric curriculum completed by group 1 and the general state of online education. The survey consisted of three domains: online education compared to prior face-to-face settings, self-evaluated sufficiency, and types of additional external resources. Each domain had a list of statements measured by the Likert 5 scale (15). The responses and the types of additional resources guided the design of the intervention (refer to Table 1 in the results section for the questions involved). Following that, the initial topics to target with our intervention were selected from the child and adolescent psychiatry curriculum. We prioritized three core topics: Attention deficit hyperactivity disorder (ADHD), conduct disorder (CD), and autism spectrum disorder (ASD). These topics were later dissected into their learning objectives as approach, diagnosis, and management.

Table 1

Two months after completing their psychiatry rotation, the first group was administered a MiniQuiz to determine memory without the intervention. It included three multiple-choice questions (MCQs) for each topic (ADHD, CD, ASD) focused on formerly mentioned objectives; approach, diagnosis, and management. Questions were prepared by us using the most recent diagnostic criteria (16).

Before the beginning of the psychiatry rotation of group 2, intervention design was carried out based on the responses within the initial needs assessment and auxiliary material already being used. We narrowed down our options of easy and applicable materials: clips from TV shows/movies containing psychiatric elements, and software-generated custom animations. For the first visual element, a list of TV shows/movies/clips was put together (Supplement A) by the psychiatric faculty members. We aimed to select the most representative footage to display symptom patterns of CD, ASD, and ADHD. Our selection included Modern Family (2010) Season 1 Episode 18 for ADHD, Temple Grandin (2010) for ASD, and the 400 Blows (from French Original “Les Quatre Cents Coups”) (1959) for CD. In the end, three 5-minute clips were prepared to be displayed for group 2. For the second visual element, the online Toonly^TM (Bryxen Inc. Dublin, OH, USA) software is selected to generate animations of exemplary patient-doctor interviews. Using the software, a script is played out using 2D characters to generate a roughly 5-minute animation (Supplement B). It included the first encounter with the patient, the initial history, and the first steps of management. The scripts were written by the research team, using academic case reviews and up-to-date DSM5 criteria (17, 18). The final products were shown to students via separate links on the days of corresponding lectures.

In order to check the response of group 2 students, we used the pre-and-post-survey method with multiple choice questions and Likert 5 statements (Figure 1). It had three multiple choice questions referring to each of the abovementioned sections of learning objectives, prepared by the psychiatry department. Their answers did not consist of anything exclusive to the online lectures already available. Upon completion of the psychiatry rotation by group 2 students, a late-post survey was given two months later (Figure 1). The late post-survey tried to measure any change in the needs of students by using the same layout of statements in the “initial needs assessment.” The same Mini-Quiz applied to group 1 was then applied to group 2, post-intervention.

Figure 1

Statistical Analysis

All the participant data is assembled in a single SPSS^TM (IBM Inc. Armonk NY, USA) sheet. We based our analysis on mean Likert scores given to each survey item and analyzed the distribution (n<30 in each group) using the Shapiro-Wilk test. The distribution of the measurements yielded results in parallel to normal distribution with p>0.05. Taking this into consideration, we used independent samples t-test to compare the answers given to the initial need assessment by the two groups or the statements scored separately by group 2 about each type of visual modality. We additionally used paired samples t-test to determine the significance of change within group 2. The change from the pre-survey to the post-survey and from the initial need assessment to the late-post survey are determined this way. The answers given to the MCQs are compared in terms of correctness by the chi-square test. We analyzed the power of our intervention within group 2 for the statements of the needs assessment with significant change in a post-hoc fashion with an online tool (19).

RESULTS

All 43 of the students volunteered to participate in the study with 20 students in group 1 and 23 in group 2. However, four students in group 2 failed to be present during either one or all the days of visual intervention and therefore were excluded from the pre-survey/post-survey analysis. Since the visual supplementation was later separately shared with them, they were included in later steps (late-post survey and mini-quiz). The study population consisted of 21 males and 22 females with a mean age of 22.7 years. In parallel to this study, all students completed and passed their psychiatry rotation on the expected dates. The initial need assessment conducted at the beginning of the study revealed similar trends (Table 1) in both groups. After the intervention, in part A, agreement by mean Likert scores of the first three statements decreased, unlike the last two statements. In part B, agreement with all statements climbed up.

Table 1

Significantly positive changes were observed in statements about “concentration” (statement A2), “case informativeness” (statement A5), “history taking” (statement B), “approach” (statement B), and “management” (statement B) with post-hoc powers 65%, 98%, 90.3%, 72.5%, and 74.1%, respectively.

The visual intervention introduced changes to both students’ perspectives and also their grasp of knowledge (Table 2, Sections A and B) as highlighted by the improvements in the post-survey results of group 2 students.

Table 2

The number of correct answers given to multiple choice questions also rose (+20.5%, p=0.017) within the pre- and post-survey results of group 2 (Table 2, Section B). Two questions (Q1 of CD and Q1 of ASD) were answered correctly by everyone even in the pre-survey.

Table 2

Regarding the comparison between the intervention group and the control group, even though there was a difference in the correct answers (group 2: 87% vs. group 1: 74.4%) given to MCQs in the mini-quizzes, it was not significant and therefore further analysis is not shown (+12.6%, p=0.058). The feedback reflecting at the visual material on the late-post survey returned mainly positive and comparable between the two modalities (Table 2, Sections C and D).

Table 2

DISCUSSION

Our study highlighted certain needs within a hybrid medical education setting, unlike a certain number of reports that stated increased student satisfaction (9, 20). However, the term “online education” on its own is a broad concept, and applications change even from one medical school to another in the same region. In schools where the online content was limited to didactic recordings of lectures and low-quality synchronous meetings, reports signify the decrease in student motivation and the lack of dimensionality within the current medical education (4, 5). This might have yielded a negative connotation for such innovative methods in settings where experience with such tools remained limited (11-13).

As identified by our study, students agreed with the “efficiency” and “realism” of the prior educational setting and stated higher “concentration” before (Table 1). They also reported insecurities about “approaching patients,” running “diagnostics,” and “management.” When we take a look at the list of complementary resources used, visual platforms like Osmosis^TM (Osmosis Inc. Baltimore, MD, USA), Youtube^TM (Youtube Inc. San Bruno, CA, USA), and Sketchy^TM (Sketchy Medical Inc. Los Angeles, CA, USA) collectively seem to be highly preferred by both groups. The role of such materials and popular culture in medical education is shown by multiple studies such as organizing movie nights, teleconferences with patients, and showing patient footage (8-10, 21, 22). We, therefore, shaped our intervention design accordingly and preferred to use animations and clips because it was easy to find, disseminate, and operate with positive reports in the literature (9, 23). The first improvement we observed was reported by group 2 students in their post-survey. They assigned higher scores to “sufficiency of the material,” “memorability,” “enjoyability,” and “ease of concentration” (Table 1). Other statements such as “recognition of symptoms,” “taking history,” and “management” -although statistically insignificant- also were scored higher after the intervention (Tables 1 and 2). Being a more objective measure, the overall correct answers given to multiple choice questions also improved significantly (Table 1) after the intervention. Since there was no lecturing present in the supplementation, the visual input we provided could have helped the students to use the information they have previously learned more efficiently. Therefore, beyond being simply a tool for enjoyment and motivation, our study displays that visual materials can have a role in turning theory into practice. When students were asked to compare the two modalities we used in our intervention, statistically significant differences (Table 2) in responses were given. Students deemed the animations to be more helpful in “understanding” the subject and “efficiency.”This can be due to the fact that the animations contained an ideal encounter and therefore had more teaching value than the clips. For the clips, the students significantly gave higher scores to the statements  about “motivation” and “enjoyability.”

Table 1

Table 2

In terms of the answers given to multiple choice questions in the mini-quizzes to assess long-term memory, it may seem hard to conclude since certain questions are answered correctly by all and statistical significance was limited. However, the effect of visual content on memory has also been analyzed before, especially for movies (23). Therefore, even though the benefit of the information retained is unclear, our study displays that simple visual supplementation can provide students with further motivation to look back on previous topics when preparing for exams and when they encounter psychiatric patients in other rotations.

We had several limitations within this study. The first one was about the number of students who were included in this study due to the small annual capacity of our institution. This likely has affected the statistical significance of our results and weakened the testing of our hypothesis. This scarcity of responders has also affected our capability to perform a healthy power analysis. Even though some of our results in the significant statements have relatively high power, these values are calculated with post-hoc calculation which is known to yield relatively unreliable results (19). We are aware that our findings would become more convincing if supported by the addition of the upcoming fifth-year medical students each year. Furthermore, we would like to overcome this limitation by extending the supplementation to other rotations as well to include more students. Other institutions can also benefit from the content we have created (Supplements A, B) and replicate the study. This way, we can perform sampling out of a larger student body and have a healthier outcome about the population of medical students. It can also boost the generalizability of our results and persuade other regions to take part in similar initiatives. Another issue we have faced was regarding the quality of the MCQs we prepared. The discrimination between the two groups and within the second group was challenging since some questions were either too easy or too well-known by the students. This has obscured the effect on memory and knowledge retention. Better quality assessment tools with a larger design team can overcome this problem.

CONCLUSION

We present a form of visual supplementation as a simple, cheap, readily available, and efficient method of diversifying the content of a hybrid medical education setting. As demonstrated here, disproportionate to the small scale of the intervention, a modest but positive outcome in multiple domains of teaching can be achieved with such a module. Institutions and teaching faculty can therefore be encouraged to engage in similar initiatives which make use of the resources at hand regardless of how limited. This way, even if a fully online educational setting is abandoned after the pandemic, the new and innovative methods introduced by the hybrid classrooms can be utilized to maintain a high standard of teaching in medical schools. We believe this can alter the negative connotation behind “online education” in regions with less experience and access by rendering it a complementary tool to support traditional learning in medicine.

Ethics Committee Approval: This study was approved by the Committee on Human Research of Koç University (decision no: 2021.098.IRB3.059).

Informed Consent: Informed consent was obtained from all of the subjects.

Conflict of Interest: The authors declared no conflict of interest.

Author Contributions: Concept: Z.B.K., S.C.S., Ş.K., A.Ö.M., A.E.Y., Design: Z.B.K., T.M., Data Collection or Processing: Z.B.K., S.C.S., F.Y., O.E., T.M., Analysis or Interpretation: Z.B.K., S.C.S., T.M., Literature Search: Z.B.K., S.C.S., F.Y., O.E., Ş.K., A.Ö.M., Writing: Z.B.K., S.C.S., F.Y., O.E., A.Ö.M., A.E.Y.

Financial Disclosure: The authors declared that this study received no financial support.