A qualitative study of provider feedback on the feasibility and acceptability of virtual patient simulations for suicide prevention training
Original Article

A qualitative study of provider feedback on the feasibility and acceptability of virtual patient simulations for suicide prevention training

Kimberly H. McManama O’Brien1,2^, Kristen Quinlan3, Laura Humm4, Andrea Cole5, Warren Jay Pires6, Ariel Jacobs6, Julie Goldstein Grumet3

1Department of Psychiatry, Boston Children’s Hospital, Boston, MA, USA; 2Department of Psychiatry, Harvard Medical School, Boston, MA, USA; 3Department of Health Promotion, Practice, and Innovation, Education Development Center, Waltham, MA, USA; 4SIMmersion, LLC, Columbia, MD, USA; 5School of Pharmacy and Health Sciences, Fairleigh Dickinson University, Florham Park, NJ, USA; 6Harlem Residency in Family Medicine, Institute for Family Health, New York, NY, USA

Contributions: (I) Conception and design: KHM O’Brien, K Quinlan, L Humm, J Goldstein Grumet; (II) Administrative support: A Jacobs; (III) Provision of study materials or patients: L Humm, WJ Pires, A Cole; (IV) Collection and assembly of data: KHM O’Brien, K Quinlan, L Humm, WJ Pires, A Cole, A Jacobs; (V) Data analysis and interpretation: KHM O’Brien, K Quinlan; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

^ORCID: 0000-0001-7143-9238.

Correspondence to: Kimberly H. McManama O’Brien, PhD, LICSW. Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115, USA. Email: kimberlyh.m.obrien@childrens.harvard.edu.

Background: Healthcare organizations are often committed to preventing suicide among their patients, but they can struggle to adequately train providers and implement strategies grounded in evidence-based suicide prevention practices. Virtual patient simulations (VPS) offer the opportunity for providers at healthcare organizations and educational institutions to learn suicide prevention strategies using a realistic and risk-free environment. The purpose of this study was to gather feedback from leaders in the healthcare field regarding the feasibility and acceptability of VPS for their organizations.

Methods: Participants (N=9) included administrators, managers, and educators from a variety of health care settings. They were invited to independently test the VPS and participate in a subsequent focus group to provide feedback. Participants were asked about VPS acceptability, satisfaction, potential fit within the intended context, feasibility of delivery, motivation to use, and likelihood of adoption. Responses were audio recorded and transcribed for coding and thematic analysis.

Results: Themes emerged regarding perceived benefits of the VPS, considerations related to cost, barriers to implementation, and suggestions for improvement. Participants reported VPS trainings were acceptable and feasible, filling an important gap in the field especially around suicide safety planning, particularly for newer clinicians and students in training. Participants felt that this type of virtual training was particularly feasible given the recent increase in need for online trainings. Suggested improvements included the need to normalize the trial-and-error nature of the VPS for trainees prior to the start of the training, and to consider shortening the duration of the simulation due to learners not being able to bill for time while training.

Conclusions: VPS may help to fill an important training need in the field of suicide prevention. The training suite may be best suited for certain settings, such as educational institutions, and most useful for populations including students and new clinicians. VPS may be particularly feasible for organizations that already utilize remote options for work and training.

Keywords: Suicide prevention; virtual patient simulation (VPS); training; provider feedback


Received: 30 March 2022; Accepted: 28 July 2022; Published: 30 October 2022.

doi: 10.21037/mhealth-22-15


Introduction

Despite significant investment in suicide prevention strategies in the United States, rates of suicide have continued to increase and remain the tenth leading cause of death among adults (1). One systematic review found that 80% of all individuals who die by suicide have had an encounter with a primary care provider in the past year, suggesting that healthcare providers could play a significant role in suicide prevention (2). However, healthcare industry gaps in suicide prevention skills are staggering. Two-thirds of healthcare providers (n=15,000) who completed an optional 2019 self-test for the Zero Suicide initiative reported lacking knowledge about suicide risk warning signs. Only one third reported confidence in their ability to respond to a suicidal patient (3). The majority of 2,257 outpatient clinicians in a 2019 New York State survey reported only moderate self‐efficacy working with suicidal clients and acknowledged inconsistent use of recommended intervention practices (4). Furthermore, a 2016 survey of over 16,000 U.S. behavioral health care professionals indicated that 52.9% had no previous suicide prevention or assessment training (5). Thus, training of both medical and mental health providers focused on suicide prevention among patients is sorely needed.

Online learning and patient simulations for suicide prevention

Online professional health care education has over a two-decade history (6). In 2014, Walsh and colleagues predicted case histories would become interactive, learners would be self-directed, and modules would offer varied levels of difficulty with branched narratives that pitted the user against the consequences of their own choices (6). Even prior to coronavirus disease (COVID), the global health care education solutions market was projected to reach $13.3 billion by 2023 from $9.3 billion in 2018 (7), warranting the need for innovative online education and training in health care.

The advantages of online training and education for learners are self-evident as they are accessible, self-paced, interactive, and personalized. For organizations, web-based training is increasingly affordable relative to in-person training and can offer tracking of participants and broad reach for staff separated geographically or working asynchronously (8). Some disadvantages of online learning include a lack of real patient interactions, no direct interactions with peers, and a shift of responsibility for technology from agency to learner (9). In addition, not all learners have ergonomically or technically optimal learning environments and web-based learning can blur the home-work divide (8). Furthermore, home environments come with numerous distractions (10) which may impact the effectiveness of online learning. Disadvantages for agencies include not being able to have a group discussion applying the training and lack of community building through training. Workplace trainings allow employees to gather socially while at work and offer agencies a chance to nurture employees both practically and emotionally by overtly validating and investing in employees and connecting with shared goals and values in patient care.

Another training technique often used historically in the medical field is patient simulations (11). Previously and typically, patient simulations were primarily provided in-person, but this has changed since the onset of the COVID pandemic in favor of online trainings. Although a variety of organizations including health care agencies, hospitals, and colleges currently use patient simulations as well as suicide prevention trainings, few use online suicide prevention trainings that include simulations.

The virtual patient simulations (VPS) training program—Suicide Prevention Training Suite

To fill a training need in the field, we created and tested a VPS training program for suicide prevention using SIMmerson® PeopleSim® technology. The VPS Suicide Prevention Training Suite consists of three modules to train practitioners in key areas of suicide prevention: risk assessment, safety planning, and motivating the patient to engage in treatment. The training offers practitioners the opportunity to practice their communication and risk assessment skills with patients in a safe, risk-free environment with a video-recorded actor. When users select options from the comprehensive list of programmed dialogue, the software evaluates the dialogue chosen as well as how the statement will impact the relationship with the patient to select an appropriate response. The use of non-branching logic and multiple patient needs in the software creates a dynamic conversational structure, allowing users to try varied approaches and experience different outcomes during each play through. An on-screen help coach provides guidance as users navigate through the program, offering real time feedback on performance and insight into the patient’s needs.

In 2019, The Joint Commission (TJC) released an updated National Patient Safety Goal (NPSG) 15.01.01 which was designed to improve the quality and safety of care for individuals being treated for behavioral health conditions and those who are identified as high risk for suicide. The NPSG addresses screening, risk assessment, and plans to mitigate risk, including policies and procedures to improve compliance with these practices. The VPS training in this study picks up after an individual has screened positive for suicide by providing the skills and practice to do a thorough risk assessment and risk mitigation practices. The training does not explicitly name any tool for clinicians to use but the training is based on the Columbia Suicide Severity Rating Scale. As such, the VPS training in this study is compatible with TJC requirements and offers health care providers a chance to gain confidence in risk assessment and best practices associated with mitigation of suicide risk. Organizations that use this training would be equipping their teams with the skills to meet these standards.

Study purpose

After developing the VPS, the research team set out to understand organizations’ need and/or interest in the training. Given that factors such as perceived usefulness, organizational compatibility, ease of use, and support from management have been shown to affect the utilization of m-health technology among healthcare professionals, the research team chose to conduct a focus group to explore these considerations among stakeholders with an interest in suicide prevention (12). The purpose of this study was to gather feedback from leaders in the health care field regarding the feasibility and acceptability of VPS training for their organizations.


Methods

Focus group setting and population

Because the SIMmerson® program was intended to train a variety of health professionals from licensed behavioral health providers to primary care physicians, we selected focus group participants from a variety of training institutions and health care systems. To ensure balanced representation and achieve optimal group size, we sought to identify individuals in leadership positions who could speak on behalf of their organization, as well as on-the-ground service providers who could speak to the practicalities and logistics of implementation of the training program. Best practices in focus group research indicate that groups can range in size from as small as three to as many as fourteen. Size is determined by logistical considerations, existing acquaintance between participants, and the breadth of the topic being discussed. Smaller groups can have more limited, but more in-depth discussion, while larger groups benefit from broader perspective (13). For the purposes of our study, we opted for 13 leaders from diverse sectors. Participants therefore included a Chief Operating Officer for behavioral health services, a NIMH researcher, a Dean of Social Work in a university, a Chief Medical Officer, two Vice Presidents of Clinical Operations, a mobile crisis Clinical Officer, a Compliance Officer for a health care system, and a wellness advocate. The disciplines represented by the participants included social work, clinical psychology, psychiatry, and care management. Many are experts in their field and have ten or more years of practice experience. Several are involved with national suicide prevention organizations and have expertise in suicide prevention training and workflows. The focus group took place via Zoom in July of 2020.

Focus group protocol

The project was classified as exempt as defined by 45CFR46 and did not require IRB review. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). Informed consent was taken from all participants verbally at the beginning of the Zoom. Prior to participation, participants used the VPS on their own. The site Principal Investigator (PhD, LICSW) conducted the focus group using a semi-structured interview guide developed by the research team. The interview guide asked participants questions that reflected on the VPS’ alignment with existing organizational values, processes, and training protocols. Participants were also asked to reflect on the importance of the VPS for their organization, the acceptability of its set proficiency scoring methodology, and thoughts on setting the price point for the training tool (see Table 1).

Table 1

Focus group questions for testing the feasibility and acceptability of virtual patient simulations for suicide prevention training

How well do the objectives of the Suicide Prevention Training Suite align with the clinical mission of your setting?
How relevant are the goals and objectives of the Suicide Prevention Training Suite to the trainees that would use it in your setting?
Discuss the value of using the Suicide Prevention Training Suite for trainees within your program. How well does it meet the needs of your organization? What needs are met and what needs are missed?
How well does the Suicide Prevention Training Suite align with the current training efforts in your setting? In what ways does it enhance existing efforts? In what ways does it duplicate existing efforts?
In what ways is implementing these trainings feasible in your setting? What might get in the way of implementing these trainings in your setting in the short-term and long term?
How important do you feel it is to have a tool like the Suicide Prevention Training Suite in your setting?
How feasible do you think it is for trainees to complete three plays of each training module with a score of 80% or higher in order to achieve proficiency? How many plays and how much time do you think is reasonable to ask your staff to complete on a training such this?
The three training modules will be available for purchase as a set at a price point of $150 per student. Would you be interested in purchasing these trainings as a package at this price point? How interested would you be in purchasing these trainings separately?
What considerations might impact the price you would be willing to pay for the Suicide Prevention Training Suite? For example, would the option to earn continuing education units (CEUs) by completing the training impact the price that you would be willing to pay?

Focus group analysis

The focus group was recorded and notes were taken during the session by a Ph.D.-level, trained qualitative analyst. Notes and recordings were reviewed by two independent, Ph.D.-level, trained assessors (KQ and KO) and a thematic analysis was conducted. Narrative text was broken up to reflect novel ideas, which were considered novel if they were substantively different from other thoughts previously mentioned. Text was identified as supportive if it reflected, underscored, or built on an idea that had already been stated. Repeat mentions of the same idea by the same participant were noted to ensure that a single point by a single participant was appropriately weighted (14). Pulling together those ideas identified as novel, themes were established first by a single reviewer (KQ) and then verified by a second reviewer (KO). Themes were identified as either related to the “feasibility and acceptability of a VPS training program”, “improvements applicable to the VPS training program”, or “considerations for the future of the VPS training program”.


Results

Feasibility and acceptability of a VPS training program

The VPS training program was convenient, feasible, and beneficial for trainees and their organizations

Participants noted that the online nature of the program allowed students to do the training on their own time, which was a strong benefit. As the trainings often took place while the U.S. was in active lock-down to avoid coronavirus disease 2019 (COVID-19) transmission, participants were particularly enthusiastic about the program’s online nature, which permitted social distancing without stalling training efforts. The easy-to-access “coach” freed time for clinical trainers. Because students typically have limited access to medical actors, the online program ensured that they were well-prepared with unlimited practice prior to meeting the medical actor in the classroom. Participants felt students were more confident and better prepared for future clinical encounters after using the program. Participants felt that the multitude of selection options offered through the virtual simulation allowed for unique clinical responses, meaning that students could not easily deduce the “correct” answer, instead having to focus on nuanced decision making to pick the best possible answer.

The VPS training program can fulfill critical training gaps in the field

Focus group participants felt that there is “no such thing as too much training” and were therefore not worried that a training program might duplicate existing training efforts. At the same time, focus group participants felt that this virtual simulation training program could fulfill gaps in motivational interviewing training and safety planning, both of which are much needed in clinical practice.

The VPS training program may be useful for management purposes

Participants in our study spoke about the easy fit between VPS training and the need of organizations to easily onboard new staff and early career clinicians. Participants noted that the well-rounded tool allowed for consistency of training across their teams, ensuring that everyone received the same information. The scoring component was particularly useful for managers, allowing for tailored feedback.

Improvements applicable to the VPS training program

Ensure a fit between the program and the audience

Participants clearly articulated that the training needs of experienced clinicians differ from those who are new to clinical work. Some felt that experienced clinicians might find a virtual “coach” who provides immediate feedback on their clinical response to the virtual client to be distracting. Experienced clinicians have established “go-to” responses, which might not be reflected in an online simulation, where they are limited to a finite number of predetermined responses. Further, while a VPS is helpful for encouraging clinicians to carefully examine patient’s nonverbal cues, it may have less applicability for telephonic crisis services, like telephone support lines, where nonverbal cues are unavailable but background noises, changes in voice, and pauses provide important information. Participants encouraged training developers to think about having a clear option for turning feedback off, avoid providing too much feedback too soon, and making the voices and response options less “computerized”.

Ensure sensitivity to trainees’ cultural and contextual experiences

An online clinical training program requires extra consideration of the context in which the training is occurring. Topics like suicide are emotionally challenging to address, and participants in this study spoke about the need for concurrent supervision and a real-time crisis number for clinicians who may need immediate support. Further, developers of virtual training content should be aware of the race/ethnicity during casting, avoid reinforcing negative racial or ethnic stereotypes, and watch for a disconnect between online actors, who may be discussing challenging content, and online clinical feedback providers, who are simultaneously smiling and clapping when an appropriate clinical response is provided. Branching logic might allow for additional flexibility, offering users the opportunity to select between a longer onboarding program versus a shorter booster session, in order to meet trainees where they are in their level of experience.

Address issues of assessment up front

Participants suggested that when designing an online VPS training program, trainees need to understand assessment up front, to ensure clinical growth and avoid frustration. The process of trial and error inherent in clinical training should be normalized and trainees should be encouraged to take the course many times to optimize their experience. Trainees need to understand how feedback on their training sessions will be used, who will receive it, and how or whether it will be shared.

Considerations for the future of the VPS training program

Cost

When running a health care facility, crisis call center, or other 24-hour care system, agency leaders must cover shift time plus the cost of the training. This magnifies the cost of the training and must be accounted for in determining the pricing of any clinical training program. Agencies might be additionally motivated to participate if there was recognition that some portion of the proceeds would support a non-profit organization that promotes national suicide prevention or mental health promotion efforts. Participants suggested that partnerships—including with organizations like the Zero Suicide Institute or American Foundation for Suicide Prevention—might allow for additional marketing and/or funding opportunities. Participants encouraged program developers to think about issuing continuing education credits to increase participant benefit, developing a cost structure that allows smaller non-profits to purchase training at a discount, and creating semester-based licensing (instead of annual licensing) to support educational institution participation.

Logistics

Participants encouraged program developers to consider issues related to access to laptops and Wi-Fi for trainees, as access will need to be built into any system that adopts the training. The training should be tested on a variety of browsers to ensure compatibility across platforms. Participants worried about time and suggested that the program might be improved by quicker start-up instructions that allow users to bypass unnecessary information but also offer accessible, detailed instructions for those who need more guidance on how the interface works (see Table 2 for a summary of focus group findings).

Table 2

Focus group reflections on feasibility and acceptability of an online virtual simulation clinical training program, areas for improvement, and cost/logistical considerations

Theme Sub-theme Examples
Feasibility and acceptability of a virtual simulation clinical training program Virtual simulation clinical training programs are convenient, feasible, and beneficial for trainees and their facilities Allows students to do it on their own time
Available response options in the program are excellent. They are not simply options that can be deduced by a “skilled test taker”
Allows students to be more deliberate in their practice. Extremely beneficial in a classroom setting
Students feel more confident after using the suite and that clinical supervisors feel students are better prepared when they enter the clinical encounter
Easy onboarding tool
Faculty and staff were thankful to have this during coronavirus disease (COVID) (social distancing)
The easy-to-access “coach” was beneficial to students
Students have limited access to medical actors. The program allows unlimited practice before they test their clinical skills with a medical actor
Virtual simulation clinical training programs can fulfill critical training gaps in the field Safety planning is both a critical, and unfulfilled, need in the field
Motivational interviewing is a particularly good module—one that is needed in the field
No such thing as too much training—not worried about duplication
Seen nothing like this in terms of motivational interviewing (MI). This was the strongest module
Virtual simulation clinical training programs can be useful for management purposes The training is a good fit for early career clinicians, for onboarding of new staff, and for college student trainings
Having a well-rounded tool helps for consistency across the team—everyone getting the same information
Scoring is the most useful part for managers, to allow for tailored feedback.
Improvements required for the program Ensure a fit between the program and the audience The visuals may not fit with a call center. Telephonic case service requires clinicians to pay attention to pauses, background noises, and other nonvisual cues
Experienced clinicians find the “coach” distracting
Experienced clinicians wanted the opportunity to write their own options when none of those in the menu matched their experience
The voice sometimes felt “computerized”
Ensure sensitivity to trainee’s cultural and contextual experiences Students are taking the course independently but the content is emotionally charged. We need an interface that will allow an “off ramp” for additional clinician safety
There is a disconnect between a black actor, discussing difficult content, and a smiling, thumbs up, white actress in the corner. The same actor discussing the content should give the feedback. Thumbs up and clapping felt condescending, distracting at times
It would be helpful to have branch logic that matches the tool with needs. For example, branching to see if the user is onboarding or doing a booster, if they have 20 minutes vs. 1 hour available, etc.
Address issues of assessment up front Students and clinicians need to know that it’s not uncommon to fail the first time. They should be given the grading matrix up-front, so they understand how they’ll be assessed
Need guidance on how we would operationalize the feedback from the transcript. Who would get it? What would we do with it? Need to be able to close that loop
Students may feel anxious about needing to achieve this level
Build expectations. Let people know that it is a system that promotes learning by failing. Let users know that you will get multiple tries. Normalize the trial and error process through a pre-dialogue
Users should have up-front access to a scoring matrix
Considerations for the future of the program Cost Annual licensing may not make sense for educational institutions. If you are a school and you pay for an annual license, you should be able to use those licenses for the spring and fall semesters
If some portion of proceeds were doing to American Association of Suicidology or American Foundation for Suicide Prevention…Knowing where the proceeds are going and whether there is an aspect of “public good” for the proceeds
Consider partnering with Zero Suicide for marketing
Possibly partner with American Foundation for Suicide Prevention for additional funding opportunities
Having a grant that covers implementation is helpful
Consider different price points, where educational institutions have a different price than for-profit locations. Separate pricing for mass buying. In the nonprofit world, you cannot have a high price point
Having a grant that covers implementation is helpful
Cost. When running a 24-hour system (like a call center), we have to cover shift time and the cost of the training
Availability of continuing education units (CEUs) will play a role. 8–10 CEUs typically cost less than $100
Logistics Time is an issue. The text at the beginning might be simplified, perhaps with a few short videos explaining how the platform runs. A help button would also be beneficial. A quickstart option would allow users to bypass unnecessary information
Time pressure is an issue. Having only one hour to do the training means clinicians may balance what they would actually do vs. trying to move through the system
Be sure this has been tested on a variety of browsers
It’s a matter of how it gets prioritized when people have a laundry list of “must do’s”
For students, it is not guaranteed that they will have access to laptops, Wi-Fi, etc. We have to build in classroom time for access

Discussion

The VPS training program discussed in this focus group was described as convenient, feasible, and beneficial for trainees and their facilities, and was seen as able to fill critical training gaps in the field and may be useful for management purposes. An asynchronous, online suicide prevention training option was seen as particularly useful due to the COVID pandemic. Offering online education that includes simulated patient interaction may be even more critical now that healthcare disciplines have less opportunity to train in-person (15).

Suggestions for improvement to the VPS training program included the need to ensure a fit between the program and the audience, uphold the sensitivity to trainees’ cultural and contextual experiences, and address issues of assessment up-front. Another improvement that was suggested included the need to normalize the trial-and-error nature of the VPS prior to the start of the training. In addition, issues of cost and logistics were potential barriers that participants felt needed to be addressed for VPS training program to be widely adopted.

Cost benefit analyses of suicide prevention training programs have demonstrated that training is cost effective and results in cost savings based on averted suicide deaths (16), fewer expenditures on emergency interventions and higher levels of care, and lost worker productivity. Further, health care systems that offer greater support for clinical work have shown less burnout among physicians (17). Although most health care providers have never received suicide specific training, the majority will encounter a patient at risk for suicide. Therefore, an investment in training for staff is likely to improve quality of care and outcomes for patients. However, research on the impact of training on suicide-related outcomes is needed to make this determination.

One limitation of the study is the small sample size which is mitigated by the nature of participants who could speak to the primary research questions (i.e., management and upper management at health care and educational institutions). Another limitation is the lack of front-line providers and other who would be the target end users of the program. Future research can and should explore the feasibility, acceptability, and impact of the actual implementation of the VPS training program in real-world health care settings with a variety of disciplines engaging the training.

In sum, VPS training programs may help to fill an important need in the field of suicide prevention. The suicide prevention training suite may be best suited for certain settings, such as educational institutions, and most useful for particular populations including students and new clinician hires. This suicide prevention training may be particularly feasible for organizations that conduct a lot of their work remotely, especially if modules can be shortened.


Acknowledgments

Funding: This work was supported by the National Institute on Mental Health of the National Institutes of Health under Award Number R44MH114710. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.


Footnote

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-15/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-15/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-15/coif). SIMmerson® was the primary grant holder with Education Development Center (EDC) as a sub and with The Institute for Family Health (IFH) as a sub. SIMmerson® was the primary awardee who owns the training program developed which is actively being licensed for fees. KHMOB, KQ, and JGG were employed by EDC throughout the project. LH is employed by and owns shares in SIMmerson®, LLC and that both Education Development Center and SIMmerson® may benefit from sales of the product once commercialized. AC was employed by IFH for the majority project. During this time, AC was also an employee at the Center for Practice Innovations (CPI) at New York State Psychiatric Institute. Since August, 2021, AC resigned from her role at IFH and CPI. WJP and AJ were employed by IFH throughout the project. JGG is the Director of the Zero Suicide Institute (owed by EDC) which may also benefit from sales of the product once commercialized. JGG sits on NQF Board and Center for School Mental Health Advisory Board, and Centerstone Zero Suicide Grant (SAMHSA) Advisory Board. All authors had salary support from the grant (R44MH114710) from which this project was funded. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The project was classified as exempt as defined by 45CFR46 and did not require IRB review. Informed consent was taken from all participants verbally at the beginning of the Zoom.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Suicide [Internet]. National Institute of Mental Health. U.S. Department of Health and Human Services; [cited 2022 Mar 25]. Available online: https://www.nimh.nih.gov/health/statistics/suicide
  2. Stene-Larsen K, Reneflot A. Contact with primary and mental health care prior to suicide: A systematic review of the literature from 2000 to 2017. Scand J Public Health 2019;47:9-17. [Crossref] [PubMed]
  3. Harmer B. Suicidal ideation [Internet]. StatPearls [Internet]. U.S. National Library of Medicine; 2021 [cited 2022 Mar 25]. Available online: https://www.ncbi.nlm.nih.gov/books/NBK565877/
  4. Labouliere CD, Green KL, Vasan P, et al. Is the outpatient mental health workforce ready to save lives? Suicide prevention training, knowledge, self‐efficacy, and clinical practices prior to the implementation of a statewide suicide prevention initiative. Suicide and Life‐Threatening Behavior 2021;51:325-33. [Crossref] [PubMed]
  5. Silva C, Smith AR, Dodd DR, et al. Suicide-Related Knowledge and Confidence Among Behavioral Health Care Staff in Seven States. Psychiatr Serv 2016;67:1240-5. [Crossref] [PubMed]
  6. Walsh K. The future of e-learning in healthcare professional education: some possible directions. Annali dell'Istituto superiore di sanita 2014;50:309-10. [PubMed]
  7. Research and Markets. Healthcare Education Solutions Market - Global forecasts to 2023: CME programs gaining momentum due to stringent regulatory mandates [Internet]. GlobeNewswire News Room. Research and Markets; 2019 [cited 2022Mar22]. Available online: https://www.globenewswire.com/news-release/2019/02/27/1743169/0/en/Healthcare-Education-Solutions-Market-Global-Forecasts-to-2023-CME-Programs-Gaining-Momentum-Due-to-Stringent-Regulatory-Mandates.html
  8. Seymour-Walsh AE, Bell A, Weber A, et al. Adapting to a new reality: COVID-19 coronavirus and online education in the health professions. Rural Remote Health 2020;20:6000. [Crossref] [PubMed]
  9. Althwanay A, Ahsan F, Oliveri F, et al. Medical Education, Pre- and Post-Pandemic Era: A Review Article. Cureus 2020;12:e10775. [Crossref] [PubMed]
  10. Seymour-Walsh AE, Weber A, Bell A. Pedagogical foundations to online lectures in health professions education. Rural Remote Health 2020;20:6038. [PubMed]
  11. Oliveira LM, Figueiredo EG. Simulation Training Methods in Neurological Surgery. Asian J Neurosurg 2019;14:364-70. [Crossref] [PubMed]
  12. Gagnon MP, Ngangue P, Payne-Gagnon J, et al. m-Health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc 2016;23:212-20. [Crossref] [PubMed]
  13. Bloor M. editor. Focus groups in social research. Sage Publications, 2001.
  14. Thurgur L, Bandiera G, Lee S, et al. What do emergency medicine learners want from their teachers? A multicenter focus group analysis. Acad Emerg Med 2005;12:856-61. [Crossref] [PubMed]
  15. Newman NA, Lattouf OM. Coalition for medical education—A call to action: A proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during COVID‐19. Journal of Cardiac Surgery 2020;35:1174-5. [Crossref] [PubMed]
  16. Vasiliadis HM, Lesage A, Latimer E, et al. Implementing Suicide Prevention Programs: Costs and Potential Life Years Saved in Canada. J Ment Health Policy Econ 2015;18:147-55. [PubMed]
  17. West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018;283:516-29. [Crossref] [PubMed]
doi: 10.21037/mhealth-22-15
Cite this article as: O’Brien KHM, Quinlan K, Humm L, Cole A, Pires WJ, Jacobs A, Goldstein Grumet J. A qualitative study of provider feedback on the feasibility and acceptability of virtual patient simulations for suicide prevention training. mHealth 2022;8:31.

Download Citation