Evidence-based training and certification: the ERS thoracic ultrasound training programme

Identification of needs, target audience, and TUS task force group

In 2016, a Danish national general needs assessment identified focused ultrasound scanning of the lungs as one of the 11 technical procedures in pulmonary medicine that should be integrated into a simulation-based curriculum [12]. 2 years later, the ERS published the “Update of the ERS international Adult Respiratory Medicine syllabus for postgraduate training” and the ERS Monograph on “Thoracic Ultrasound”, both establishing TUS as a core skill for respiratory physicians [1, 13]. A task force comprised of international clinicians was established to oversee the development of a TUS curriculum and training programme. Members included advanced TUS practitioners and educators with many years of experience in delivering TUS teaching, medical and simulation educational experts, and those who helped to develop the successful ERS EBUS programme [9].

Finally, after working on the training programme, the ERS statement on TUS from 2021 confirmed the growing evidence supporting the use of TUS and underlined the importance of evidence-based training and certification [14].

Establishment of objectives and content, and determination of educational strategies

To gather information about current training programmes, educational strategies and methods of assessment, a literature search was undertaken based on members' existing knowledge of the topic and through reference screening, but without the use of a structured or evidence-based approach [15]. Medical educational theory and research in other educational areas suggested training would be optimal using a three-step approach, structured to have gradually increasing complexity [15–17]. Accordingly, the task force ratified a programme structure beginning with theoretical learning to establish the basic principles of TUS. This would be followed by a practical component, ensuring acquisition of ultrasound skills following hands-on training, and the creation of a TUS portfolio. The programme would end with an objective structured clinical examination (OSCE) for final certification and confirmation of TUS competencies (figure 1). The approach aligns with the framework for the assessment of clinical competencies proposed by Miller [18], which recognises the need for consecutive development when learning a new skill, with increased complexity when moving from the basics at the bottom of the pyramid towards the complexity at the top.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

Thoracic ultrasound training programme according to Miller's pyramid. Reproduced and modified from [18]. The Creative Commons license does not apply to this content. Use of the material in any format is prohibited without written permission from the publisher, Wolters Kluwer Health, Inc. Please contact permissions@lww.com for further information. OSCE: objective structured clinical examination; MCQ: multiple choice question.

As well as basic theoretical knowledge and the ability to practically perform the ultrasound examination being key objectives for learners, the task force also agreed that the training programme should prioritise the integration of TUS findings into a clinical context, driven by its real-world use which sees it complement other diagnostic tests, clinical examinations and patient history. Thereby, the goals of the training programme are broad, whereas the objectives for each specific part and module of the programme are clear and measurable; striving for competence-based training, that accommodates trainees’ different learning paces but ensures specific minimum requirements for each part. This structure advocates for mastery learning and the principles of “excellence for all”, meaning that all trainees can learn the established goals and objectives to a high standard with minimal variation of the learning outcomes, but that individual trainees have different learning paces and prerequisites [19].

The curriculum evolved through several rounds of feedback and assessment by the task force, who needed to reach consensus at each stage to ensure that the objectives and content were robust across institutions and countries [11, 20].

The TUS curriculum and training programme structure

The TUS training curriculum comprises three units that cover the fundamentals of the topic required for clinical practice. As need and/or new evidence evolves over time, the number and content of the units are anticipated to adapt and expand. A complete list of the TUS curriculum topics, including specific learning objectives for knowledge, skills and attitude, is included in the supplementary material, with an overview presented in table 1.

The content of the curriculum is delivered via a training programme, which is split into three parts: theoretical, practical, and final assessment (see figure 1). Part 1 (theory) comprises a package of six online educational modules, covering basic physics and principles, focused TUS, basic chest sonography, ultrasound-guided procedures, and basic cardiac ultrasound (table 2). The educational programme is open to all for whom it is of interest, and for the part 1 there is no maximum number of participations. The anticipated time to complete the six modules is estimated to be between 12 and 20 h, ending with a timed, 30-question multiple-choice assessment required for part 1 certification, and for continuation to part 2.

For high-stake and summative assessments, solid validity evidence needs to be explored to ensure that the test actually measures what it is supposed to measure, that the internal consistency reliability is high enough, and to establish a credible pass/fail score. For part 1, the initial 30 multiple-choice questions (MCQ) have been tested using recognised validity frameworks [21]. As a part of the continuous improvement and development of the course, the task force will continue to expand the bank of MCQs, ensuring variability in the test from course to course. When more questions are added, item analysis will be undertaken to ensure a continued evidence-based approach.

The part 2 (practical) component of the programme was originally intended to be delivered as a 2-day, hands-on course featuring clinical observation, active training on simulated patients, scanning real patients with sonographic pathologies, and simulation-based training. The structure of this course was based largely on a successful model developed in Odense, Denmark, with sessions delivered by a broad, multidisciplinary clinical faculty. To increase the time available for the development of practical skills, and to ensure sufficient experience for each trainee, courses were to be set up with multiple opportunities for hands-on training over the 2 days. The student to instructor ratio was set at 3:1 to ensure close supervision of each trainee, with evidence showing a strong correlation between the time given for hands-on training and competence [22]. At the end of the 2 days, a pass/fail practical examination was to be held for participants, based on the course material [23].

However, due to the impact of the COVID-19 pandemic (lockdowns and the risk of disease spread in particular), the part 2 material was adapted into an online, interactive course to ensure TUS education remained accessible despite restrictions imposed by local, national and international regulations. A survey among pulmonologists following COVID-19 found that respondents felt well-trained in handling COVID-19-positive patients, but 36.3% responded that they had performed procedures that they did not feel competent with as a direct result of the pandemic [24]. Although it was acknowledged by the task force that online TUS courses cannot completely replace in-person courses, through the use of interactive features like live demonstrations, short lectures with time for debate, interactive question and answer sessions, and break-out sessions with case discussions, the course remained feasible and its content remained relevant to participants. Two on-site and two online part 2 courses are held per year, for the on-site courses there is a limit of 18 course participants whereas the online course can hold up to 50 participants. As for most multistep educational programmes, a drop out from part 1 to part 2 is expected, as well as a drop out from part 2 to part 3. This is an important topic of evaluation in the ERS TUS task force and educational office.

Following the part 2 course, independent of whether it is undertaken online or on-site, participants are to complete self-directed, local training and create a portfolio of ultrasound cases and clips in an online platform. At least 25 ultrasound scans are required, and the cases are peer-reviewed by fellow course participants and course faculty. For certification of part 2, the online or on-site course is to be completed, 25 cases must be uploaded, and at least 10 peer reviews must be completed.

To gain final, overall TUS certification, an OSCE assessment must be successfully completed. This OSCE is held yearly at the ERS Congress and makes up part 3 of the TUS programme. To be eligible, participants must have completed both parts 1 and 2.

The OSCE was developed specifically for the ERS TUS programme using accepted educational principles and through gathering of multicentre validity evidence. It comprises four practical stations (image optimisation and knobology, pleural effusion, interstitial syndrome, and pneumothorax) and two theoretical stations where participants answer a series of MCQs based on ultrasound images and clinical information [25].

Implementation, evaluation, and further development

As the quality of implementation directly affects training programme outcomes, implementation research is gaining prominence within the fields of medical education and course development [26, 27]. Several frameworks have been proposed, including those useful to planning and assessing quality of implementation, e.g. identification of stakeholders and an implementation team, determining institutional and organisational back-up, planning course and curriculum monitoring and evaluation [28]. The ERS education team, along with members of the ERS TUS task force group, served as an implementation team and assigned dedicated time for the realisation of the training programme with well-defined roles and responsibilities. Parts 1 and 2 were implemented in 2019 and, following delays resulting from the COVID-19 pandemic, part 3 was implemented in 2022. At the time of writing, a total of four online part 2 courses have been delivered since early 2020. Feedback from participants has been positive and, as we move into the post-pandemic period, it has been decided that two in-person courses and two online courses will be held each year, increasing both flexibility and access to as wide a group of learners as possible.

The task force remains committed to keeping content and educational strategies up to date, with the training programme thoroughly evaluated after each part by both trainees and faculty. Additionally, the task force meets regularly to review the existing curriculum and to determine if new curriculum modules are needed. New faculty members are being continuously invited to ensure uniformity and generalisation beyond the institutions that developed and started the programme, thereby ensuring the TUS programme remains current and of relevance to the whole ERS community. Following last year's part 2 courses and the first part 3 OSCE, survey data were collected on the number of ultrasound examinations performed by trainees, their self-assessment and confidence, in order to evaluate trainee behaviour during the course. These data are soon to be published.

Conclusions and next steps

The ERS TUS task force has successfully developed a TUS educational programme based on structured and evidence-based frameworks for curriculum development and assessment of clinical competencies. The TUS training programme has been implemented with part 1 courses that are accessible all year, four part 2 courses (two on-site and two online) and one part 3 OSCE each year.

The ERS and the TUS task force group strive to continuously improve the training programme by keeping up to date with new research and new trends, and by monitoring course quality and feedback with a view to expanding the curriculum, improving the programme, and increasing the accessibility of TUS.

Supplementary material