You are here

Article Text

Article menu
XML
Medical education and training
Original research
Educational needs in the COVID-19 pandemic: a Delphi study among doctors and nurses in Wuhan, China
  1. Xun Hou1,2,
  2. Wenjie Hu1,3,
  3. Lene Russell4,5,
  4. Ming Kuang3,
  5. Lars Konge4,6,
  6. Leizl Joy Nayahangan4
  1. 1Clinical Competence Training Center, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
  2. 2Gastrointestinal Surgery Center, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
  3. 3Department of Hepatic Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
  4. 4Copenhagen Academy for Medical Education and Simulation (CAMES), Centre for HR and Education, The Capital Region of Denmark, Copenhagen, Denmark
  5. 5Department of Intensive Care, Rigshospitalet, Copenhagen, Denmark
  6. 6Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
  1. Correspondence to Dr Wenjie Hu; huwjie{at}mail.sysu.edu.cn

Abstract

Objective To identify theoretical and technical aspects regarding treatment, prevention of spread and protection of staff to inform the development of a comprehensive training curriculum on COVID-19 management.

Design Cross-sectional study.

Setting Nine hospitals caring for patients with COVID-19 in Wuhan, China.

Participants 134 Chinese healthcare professionals (74 doctors and 60 nurses) who were deployed to Wuhan, China during the COVID-19 epidemic were included. A two-round Delphi process was initiated between March and May 2020. In the first round, the participants identified knowledge, technical and behavioural (ie, non-technical) skills that are needed to treat patients, prevent spread of the virus and protect healthcare workers. In round 2, the participants rated each item according to its importance to be included in a training curriculum on COVID-19. Consensus for inclusion in the final list was set at 80%.

Primary outcome measures Knowledge, technical and behavioural (ie, non-technical) skills that could form the basis of a training curriculum for COVID-19 management.

Results In the first round 1398 items were suggested by the doctors and reduced to 67 items after content analysis (treatment of patients: n=47; infection prevention and control: n=20). The nurses suggested 1193 items that were reduced to 70 items (treatment of patients: n=49; infection prevention and control: n=21). In round 2, the response rates were 82% in doctors and 93% in nurses. Fifty-eight items of knowledge, technical and behavioural skills were agreed on by the doctors to include in the final list. For the nurses, 58 items were agreed on.

Conclusions This needs assessment process resulted in a comprehensive list of knowledge, technical and behavioural skills for COVID-19 management. Educators can use these to guide decisions regarding content of training curricula not only for COVID-19 management but also in preparation for future viral pandemic outbreaks.

  • medical education & training
  • infection control
  • epidemiology
  • preventive medicine
  • respiratory infections

Data availability statement

The Chinese version of the questionnaires, along with the deidentified data, will be made available upon reasonable request (contact email: huwjie@mail.sysu.edu.cn).

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2020-045940

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This study employed a modified Delphi process to assess educational needs in fighting COVID-19, a fundamental step that is currently missing in various curricula on COVID-19 management.

    • All participants were front-line health workers caring for patients with COVID-19 from Wuhan, China.

    • Experiences, skills and resources in China might be different from other countries.

    Introduction

    COVID-19, caused by SARS-CoV-2, has rapidly become a global pandemic.1 COVID-19 has affected 235 countries, territories and areas (as of 15 October 2020), infected more than 38 million people and claimed more than 1 million lives so far, as per the WHO.2

    The rapid escalation of infected individuals has put overwhelming pressure on healthcare systems worldwide. When the outbreak started in China, the sudden increase of infections demanded an emergent call for additional healthcare workers from other provinces. In total, 32 395 healthcare personnel from different provinces were dispatched to Wuhan and the neighbouring cities in Hubei.3 At present, tens of thousands of healthcare workers across the world are mobilised to work with patients with COVID-19 and many face challenges such as long shifts, shortages of medical resources including personal protective equipment (PPE), and the high risk of contracting a potentially life-threatening disease. While the focus continues to be on patient treatment and care, healthcare workers may also be tasked with identifying, triaging and isolating patients, and at the same time protecting themselves, their colleagues and their families from contracting the infection. This responsibility has raised undeniable concern from international organisations and the public, with increasing calls not only to ensure adequate medical resources but also continued training and education for healthcare workers on preparedness and response.4

    During the initial wave of the pandemic, training programmes were quickly rolled out to fast-train healthcare workers, mostly based on existing curricula designed to educate general specialist knowledge, skills and attitudes regarding all sorts of diseases. But now, as the world may see a second wave of infections, with new COVID-19 cases being reported by countries that have eased restrictions, scientists are advising different government bodies to prepare to ensure healthcare capacity is established with adequate resources, facilities and workforce to minimise infection and mortality risks.5 This also presents an urgent call for educators to reflect on key lessons from the initial wave and identify training programmes that should have been implemented. This allows the development of a highly relevant training curriculum that is aligned to current needs in order to adequately prepare healthcare workers for the next wave.6

    In this study, we aimed to identify theoretical and technical aspects regarding treatment, prevention of spread and protection of staff to inform the development of a comprehensive training curriculum on COVID-19 management.

    Methods

    Study design

    This cross-sectional study employed a two-round modified Delphi process from March to May 2020 to gather information from healthcare workers who were actively involved in caring for infected patients during the initial epidemic in Wuhan, China (figure 1). The Delphi method is a systematic, iterative process that uses survey questionnaires to collect and refine expert opinion on an important topic.7

    Figure 1
    Figure 1

    Overview of the Delphi process.

    Steering group

    An international steering group was formed to facilitate and manage the Delphi process. The group consisted of a Chinese gastrointestinal surgeon deployed to work in Wuhan, a Chinese hepatic surgeon and the director of a clinical competence training centre who was in charge of the training of managing COVID-19, a Danish professor of medical education, a senior medical education researcher in Denmark with extensive experience in conducting needs assessments, and a Danish specialist in intensive care medicine including care for patients with COVID-19; the latter three are all from the Copenhagen Academy for Medical Education and Simulation (CAMES). Questionnaires exploring the theoretical and practical skills to include in a course on COVID-19 management were developed by the steering group and translated to Chinese. The two Chinese surgeons coordinated with the different hospital settings in Wuhan to identify participants and distribute the questionnaires. The answers from the different rounds were translated to English and analysed by the CAMES-based researchers in Copenhagen.

    Participants

    The participants were doctors and nurses from different cities who were deployed into clinical work in response to the sudden surge of COVID-19 cases in Wuhan, China. They were identified through the list of deployed medical personnel from the Health Commission of Guangdong Province. Initially, medical team leaders in nine different hospitals in Wuhan were contacted and informed of the aim of the study. They were then asked to send the survey link to all the healthcare workers within their team with the aim of including a representative sample large enough to reach saturation. Invitations were sent through email or the mobile messaging app, WeChat (Tencent, Shenzhen, China), detailing the objectives of the study, the importance of participation and the link to the survey. The survey questionnaires were in Chinese and were developed and sent using the online survey platform Wen Juan Xing (Ranxing Information Technology, Changsha, China).

    Delphi round 1

    The first round of the survey was a brainstorming phase (online supplemental file 1). The participants were initially asked to provide demographic information to help establish the characteristics of the population, including job title, years of experience, number of days working in Wuhan and what their roles were depending on the hospital setting. After which, they were asked to answer three specific questions:

    1. What do you need to know and be able to do to optimally treat patients who are infected with the virus?

    2. What do you need to know and be able to do to prevent spread of the virus?

    3. What do you need to know and be able to do to protect yourself as a healthcare worker?

    The participants were instructed to identify both knowledge and technical and behavioural skills (ie, non-technical skills).

    The answers were divided according to profession (doctors or nurses). The steering committee reviewed the answers, removed duplicates and organised the items into different categories to send to round 2.

    Delphi round 2

    Individual questionnaires were prepared for doctors (online supplemental file 2) and for nurses (online supplemental file 3). In round 2, the participants were initially asked to indicate their specialties, followed by rating of the different items from round 1. They were asked to rate each item according to importance of including it in a course on COVID-19 management, where 1=not at all important, 2=slightly important, 3=important and 4=very important. The participants were guided by two questions:

    1. How important is it to include each item in a course for doctors/nurses in the treatment of patients with COVID-19?

    2. How important is it to include each item in a course for doctors/nurses in the prevention of COVID-19 and protection of staff?

    Data analysis

    Answers from the doctors and the nurses were analysed separately. Participant characteristics were recorded as categorical and characterised by descriptive statistics accordingly. In round 1, the steering group performed content analysis by removing duplicates and counting frequency of occurrence. After which, similar items or procedures were grouped together (ie, can be implemented in one training programme) and were allocated into different categories that were defined during content analysis. Items that were suggested by only one participant were either grouped with a similar item or excluded. In round 2, ranges, frequencies and percentages were calculated. Consensus level for inclusion in the final list was set at 80%, that is, four out of five participants rating the item as important or very important to include in a COVID-19 course. The mean score for each remaining item was calculated.

    Statistical analysis was performed using SPSS V.25.0 software package.

    Patient and public involvement

    This study was based on questionnaires sent to medical workers; therefore, neither patients nor the public were involved in the design, or conduct, or reporting or dissemination plans of this research.

    Results

    In total, 134 individuals agreed to participate and completed the surveys (n=74 doctors and n=60 nurses). Table 1 presents the demographic characteristics of the participants, including specialties, place of assignment and number of days working in Wuhan.

    View this table:
    Table 1

    Demographic characteristics of doctors and nurses who participated in the study

    Round 1

    In round 1 for doctors, there were 1398 different aspects that were suggested and reduced to 67 items after content analysis. These were divided into treatment of patients (n=47) and infection prevention and control (n=20).

    In the same round, the nurses suggested 1193 different aspects. These were reduced to 70 items, divided into treatment of patients (n=49) and infection prevention and control (n=21).

    Round 2

    In round 2, the response rates for the two groups were high, with 82% of the doctors and 93% of the nurses finishing round 2.

    Out of 67 items from the initial round, 58 were agreed on by the participating doctors to include in the final list. Nine items were eliminated since less than 80% rated these items important or very important. Table 2 presents the items included in round 2, including the mean score.

    View this table:
    Table 2

    Final list of items to include in a training curriculum for doctors

    For the nurses, 58 items were agreed on out of 70 from the initial round. Twelve items did not meet the consensus criterion and were therefore excluded. The final list of included items is presented in table 3, including the mean rating scores.

    View this table:
    Table 3

    Final list of items to include in a training curriculum for nurses

    Discussion

    In this study, we conducted a systematic process using a modified Delphi method to identify the training needs of doctors and nurses on the front lines of the COVID-19 pandemic. This resulted in a list of procedures that informs the development of training programmes regarding COVID-19 management.

    The high percentage of participants indicated unequivocal support in identifying and establishing standardised training programmes to better equip healthcare workers with specialised knowledge and skills to manage highly infectious disease. This was evident in the second round with more than 90% response rate even though most of the participants had returned home following their deployment in Wuhan. This is one of the advantages of the Delphi process, where subsequent questionnaires can be sent electronically notwithstanding geographical location.7 The online format of this process facilitated the gathering of data from each participant and eliminated the influence of dominant individuals, which may occur in other group settings such as face-to-face meetings.7

    The procedures that were included were aligned in both groups, with a few exemptions specific to either group.

    Treating patients with COVID-19

    Knowledge and technical skills in critical care management have been identified, including tracheal intubation, use of invasive and non-invasive mechanical ventilation, central venous access, and haemodynamic monitoring. One of the life-threatening complications of COVID-19 is severe acute respiratory failure, which may require intensive care treatment.8 Important considerations when performing some of these treatments may arise in order to decrease risk of infection in healthcare workers (eg, the use of open systems for oxygen delivery such as high-flow nasal oxygen, the correct timing of intubation, use of bronchoscopy and performing tracheostomy).9

    Veno-venous extracorporeal membrane oxygenation (ECMO) was recommended as a life-saving modality for patients with the most severe of acute respiratory distress syndrome.9 10 To perform this complex therapeutic intervention, specialised and highly trained staff are required. Interestingly, the doctors eliminated ECMO in the final round; however, the nurses rated the training of how to assist during ECMO management as important and therefore was retained. Because it is a resource-intensive procedure that requires expert skills, it was rated low by the doctors. Furthermore, the WHO interim guidelines recommend that the administration of ECMO to eligible patients should only be offered at specialised centres.11 For nurses, the provision of sound nursing care for ECMO patients is fundamental and has to be trained.12

    Performing flexible bronchoscopy in patients with COVID-19 was also eliminated by doctors in the final round. This is consistent with the fact that its utility is so far uncertain, with the American Association for Bronchology and Interventional Pulmonology emphasising that bronchoscopy is not an appropriate tool for diagnosis of SARS-CoV-2 infection and should only be performed if strictly indicated given the high risk of aerosol-transmitted infection.13

    Other procedures that were not deemed necessary to include in a COVID-19 course were skills that many doctors and nurses already master, such as cardiac pulmonary resuscitation, arterial puncture and nasogastric tube insertion.

    Infection prevention and control

    SARS-CoV-2 is transmitted from person to person through respiratory droplets and close contact. According to the WHO, airborne transmission is also possible, especially when performing procedures where aerosol is produced, such as endotracheal intubation, open suctioning, tracheostomy or during ventilation.14

    Donning and doffing of PPE was suggested by the majority of the participants and was rated highly as being one of the most important life-saving procedures for staff safety when dealing with the highly infectious respiratory disease. There are existing training modules on PPE use as well as recommendations from the Centers for Disease Control and Prevention outlining key concepts regarding prevention of spread and protection of healthcare personnel who have direct and indirect contact with patients and infectious substances.15 Several studies have reported increase in confidence, knowledge and skills after training of proper donning and doffing of PPE in a safe environment.16 17

    The importance of isolation measures was also emphasised by both participant groups, including screening and triage protocols of suspected and infected patients, as well as adhering to standard precautions to protect themselves from infection. The WHO has provided operational guidance to promote the early identification and prevention of transmission of the COVID-19 virus to patients and healthcare workers.18

    Care for the healthcare worker

    Physical and psychological well-being were identified by the participants to be included in a comprehensive training programme in the management of COVID-19. The impact of the COVID-19 pandemic has taken its toll, posing strain not only on the healthcare system but also on the healthcare worker. Currently, there is a pressing need to focus on critical issues surrounding mental health of healthcare workers.19 Previous outbreaks have shown severe emotional distress among healthcare workers on the front line resulting in stress, post-traumatic disorders, anxiety and burnout.20 Educational interventions to promote well-being among healthcare workers are needed, such as psychological first aid.21

    Implications

    Training and education of healthcare workers before and during an outbreak are crucial in order to ensure that all health workers have the knowledge and skills to treat infected patients, and to prevent and control the spread of infection to other patients or themselves. Pandemic outbreaks are rare events and this COVID-19 is unprecedented with a potential second wave of infections. This presents a unique window of opportunity to continue preparing and upskilling healthcare workers to manage the upcoming crisis.

    This needs assessment is an important first step to ensure that training programmes are aligned to current needs of healthcare workers on the front lines. Following the Kern's six-step approach of curriculum development,6 the next step would be to develop training programmes on these identified procedures depending on the local context, that is, targeted learners (targeted needs assessment). Goals and objectives should be well defined and clearly stated, including assessment of learning with set criteria for passing to ensure that the set competences are achieved. Deliberate selection of educational strategies is also vital prior to efficient implementation and lastly evaluation of training delivery.6

    Gathering from previous epidemic experiences, some of the most efficient educational strategies include e-learning and simulation-based training. e-Learning is an effective strategy with large positive effects on learning. This has been used in the past for fast and flexible dissemination of information to a large group across different countries including remote locations.22

    Simulation-based training is a powerful educational strategy in the midst of an epidemic, providing a safe environment for healthcare workers to train different skillsets, while protecting patients from further harm.23 Based on the identified skills for doctors and nurses in this study, simulation-based training can be used to fast-train healthcare workers on specific technical skills (eg, performing clinical procedures while wearing PPE), on working closely and collaboratively with a team composed of different professional roles and responsibilities, and on optimising workflows and systems.16 24

    Strengths and limitations

    The main strength of this study is the involvement and active engagement of front-line healthcare workers fighting COVID-19 in Wuhan, China, indicating that the suggestions were based from first-hand experiences. However, this also poses a limitation as the experiences, skills and resources in China might be different from other countries. Nevertheless, we believe that these lists of core knowledge, technical and behavioural skills are generalisable and can inform healthcare workers across the world who are fighting the same disease.

    Conclusion

    This needs assessment process has resulted in a consensus document listing what healthcare workers should be taught in order to manage patients with COVID-19. We propose that educators use these lists as a guide when deciding the content of their training programmes to integrate in a comprehensive curriculum, not only during this unprecedented COVID-19 crisis but also in preparation for future viral pandemic outbreaks.

    Data availability statement

    The Chinese version of the questionnaires, along with the deidentified data, will be made available upon reasonable request (contact email: huwjie@mail.sysu.edu.cn).

    Ethics statements

    Ethics approval

    The study was approved by the Independent Ethics Committee (IEC) of The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China, with reference number (2020) 198.

    Acknowledgments

    The authors would like to thank Dr Shaoting Feng for assisting with the generation and collection of Wen Juan Xing questionnaires and Karlen Bader-Larsen for English language editing of the manuscript. Ultimately, the authors extend their gratitude to the doctors and nurses who took their time and participated in this study.

    References

    1. ↵
      1. Wu Z,
      2. McGoogan JM
      . Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese center for disease control and prevention. JAMA 2020;323:1239–42.doi:10.1001/jama.2020.2648pmid:http://www.ncbi.nlm.nih.gov/pubmed/32091533
      OpenUrlCrossRefPubMed
    2. ↵
      1. World Health Organization
      . WHO coronavirus disease (COVID-19) dashboard, 2020. Available: https://covid19.who.int/ [Accessed 15 Oct 2020].
    3. ↵
      1. Li L,
      2. Xv Q,
      3. Yan J
      . COVID-19: the need for continuous medical education and training. Lancet Respir Med 2020;8:e23. doi:10.1016/S2213-2600(20)30125-9pmid:http://www.ncbi.nlm.nih.gov/pubmed/32192586
      OpenUrlPubMed
    4. ↵
      1. World Health Organization
      . Health systems respond to COVID-19 technical guidance #2: creating surge capacity for acute and intensive care recommendations for the WHO European Region, 2020. Available: https://www.euro.who.int/__data/assets/pdf_file/0006/437469/TG2-CreatingSurgeAcuteICUcapacity-eng.pdf?ua=1 [Accessed 5 May 2020].
    5. ↵
      1. Xu S,
      2. Li Y
      . Beware of the second wave of COVID-19. Lancet 2020;395:1321–2.doi:10.1016/S0140-6736(20)30845-Xpmid:http://www.ncbi.nlm.nih.gov/pubmed/32277876
      OpenUrlCrossRefPubMed
    6. ↵
      1. Thomas PA,
      2. Kern DE,
      3. Hughes MT, et al
      . Curriculum development for medical education: a six-step approach. JHU Press, 2016.
    7. ↵
      1. Hsu C-C,
      2. Sandford BA
      . The Delphi technique: making sense of consensus. Prac Asses Res Evaluat 2007;12:1–8.
      OpenUrl
    8. ↵
      1. Wilcox SR
      . Management of respiratory failure due to covid-19. BMJ 2020;369:m1786. doi:10.1136/bmj.m1786pmid:http://www.ncbi.nlm.nih.gov/pubmed/32366375
      OpenUrlFREE Full Text
    9. ↵
      1. Ramanathan K,
      2. Antognini D,
      3. Combes A, et al
      . Planning and provision of ECMO services for severe ARDS during the COVID-19 pandemic and other outbreaks of emerging infectious diseases. Lancet Respir Med 2020;8:518–26.doi:10.1016/S2213-2600(20)30121-1pmid:http://www.ncbi.nlm.nih.gov/pubmed/32203711
      OpenUrlPubMed
    10. ↵
      1. Prekker ME,
      2. Brunsvold ME,
      3. Bohman JK, et al
      . Regional planning for extracorporeal membrane oxygenation allocation during coronavirus disease 2019. Chest 2020;158:603–7.doi:10.1016/j.chest.2020.04.026pmid:http://www.ncbi.nlm.nih.gov/pubmed/32339510
      OpenUrlPubMed
    11. ↵
      1. World Health Organization
      . Clinical management of severe acute respiratory infection when COVID-19 is suspected, 2020. Available: https://www.who.int/publications-detail/clinical-management-of-severe-acute-respiratory-infection-when-novel-coronavirus-(ncov)-infection-is-suspected [Accessed 8 May 2020].
    12. ↵
      1. Umeda A,
      2. Sugiki Y
      . Nursing care for patients with COVID-19 on extracorporeal membrane oxygenation (ECMO) support. Glob Health Med 2020;2:127–30.doi:10.35772/ghm.2020.01018pmid:http://www.ncbi.nlm.nih.gov/pubmed/33330790
      OpenUrlPubMed
    13. ↵
      1. Ost DE
      . The future of interventional pulmonology and the role of the Journal. J Bronchology Interv Pulmonol 2018;25:161–4.doi:10.1097/LBR.0000000000000531pmid:http://www.ncbi.nlm.nih.gov/pubmed/29944586
      OpenUrlPubMed
    14. ↵
      1. World Health Organization
      . Modes of transmission of virus causing COVID-19: implications for IPC precaution recommendations. Available: https://www.who.int/news-room/commentaries/detail/modes-of-transmission-of-virus-causing-covid-19-implications-for-ipc-precaution-recommendations [Accessed 8 May 2020].
    15. ↵
      1. Centers for Disease Control and Prevention
      . Interim infection prevention and control recommendations. Available: https://www.cdc.gov/coronavirus/2019-ncov/hcp/infection-control-recommendations.html?CDC_AA_refVal=https%3A%2F%2Fwww.cdc.gov%2Fcoronavirus%2F2019-ncov%2Finfection-control%2Fcontrol-recommendations.html [Accessed 21 Apr 2020].
    16. ↵
      1. Christensen L,
      2. Rasmussen CS,
      3. Benfield T, et al
      . A randomized trial of Instructor-Led training versus video lesson in training health care providers in proper Donning and Doffing of personal protective equipment. Disaster Med Public Health Prep 2020;14:514–20.doi:10.1017/dmp.2020.56pmid:http://www.ncbi.nlm.nih.gov/pubmed/32223776
      OpenUrlPubMed
    17. ↵
      1. Harrod M,
      2. Petersen L,
      3. Weston LE, et al
      . Understanding workflow and personal protective equipment challenges across different healthcare personnel roles. Clin Infect Dis 2019;69:S185–91.doi:10.1093/cid/ciz527pmid:http://www.ncbi.nlm.nih.gov/pubmed/31517971
      OpenUrlPubMed
    18. ↵
      1. World Health Organization
      . Operational considerations for case management of COVID-19 in health facility and community. Available: https://www.who.int/publications-detail/operational-considerations-for-case-management-of-covid-19-in-health-facility-and-community [Accessed 7 May 2020].
    19. ↵
      1. Shah K,
      2. Kamrai D,
      3. Mekala H, et al
      . Focus on mental health during the coronavirus (COVID-19) pandemic: applying Learnings from the past outbreaks. Cureus 2020;12:e7405. doi:10.7759/cureus.7405pmid:http://www.ncbi.nlm.nih.gov/pubmed/32337131
      OpenUrlPubMed
    20. ↵
      1. Lee SM,
      2. Kang WS,
      3. Cho A-R, et al
      . Psychological impact of the 2015 MERS outbreak on hospital workers and quarantined hemodialysis patients. Compr Psychiatry 2018;87:123–7.doi:10.1016/j.comppsych.2018.10.003pmid:http://www.ncbi.nlm.nih.gov/pubmed/30343247
      OpenUrlCrossRefPubMed
    21. ↵
      1. Sijbrandij M,
      2. Horn R,
      3. Esliker R, et al
      . The effect of psychological first aid training on knowledge and understanding about psychosocial support principles: a cluster-randomized controlled trial. Int J Environ Res Public Health 2020;17doi:10.3390/ijerph17020484pmid:31940865
    22. ↵
      1. Otu A,
      2. Ebenso B,
      3. Okuzu O, et al
      . Using a mHealth tutorial application to change knowledge and attitude of frontline health workers to Ebola virus disease in Nigeria: a before-and-after study. Hum Resour Health 2016;14:5. doi:10.1186/s12960-016-0100-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/26872824
      OpenUrlPubMed
    23. ↵
      1. Li L,
      2. Lin M,
      3. Wang X, et al
      . Preparing and responding to 2019 novel coronavirus with simulation and technology-enhanced learning for healthcare professionals: challenges and opportunities in China. BMJ Simul Technol Enhanc Learn 2020;6:196–8.doi:10.1136/bmjstel-2020-000609pmid:http://www.ncbi.nlm.nih.gov/pubmed/32832099
      OpenUrlPubMed
    24. ↵
      1. Choi GYS,
      2. Wan WTP,
      3. Chan AKM, et al
      . Preparedness for COVID-19: in situ simulation to enhance infection control systems in the intensive care unit. Br J Anaesth 2020;125:e236–9.doi:10.1016/j.bja.2020.04.001pmid:http://www.ncbi.nlm.nih.gov/pubmed/32307117
      OpenUrlPubMed

    Supplementary materials

    Footnotes

    • XH and WH are joint first authors.

    • Contributors XH, WH and LJN are the guarantors of this study. XH, WH and MK led the recruitment of participants and data collection and were supported by LR, LK and LJN. XH, WH, LR, MK, LK and LJN contributed to data processing, data analysis and data interpretation. XH, WH and LJN drafted the manuscript and were supported with literature search by LR, MK and LK. All authors contributed equally to the revision and quality improvement of this paper and have approved the final version of the manuscript. The corresponding author attests that all authors meet the authorship criteria.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.