qualitative methods in rapid turn around health services research

This website may not work correctly because your browser is out of date. Please update your browser .

Qualitative Methods in Rapid Turn-Around Health Services Research

This video presentation by Alison B. Hamilton for the US Department of Veterans Affairs  (VA) focuses on qualitative study design, data collection, and fundamentals of data analysis. The presentation uses examples from a VA women's health services project in order to demonstrate how the rapid collection of data can be used generate findings that could be used immediately.

The video presentation includes a downloadable handout of the slides which can be accessed here: PDF Handout

Hamilton, A. B. (2013).  Qualitative Methods in Rapid Turn-Around Health Services Research, VA Women's Health Research Network. Retrieved from:  http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780

Related links

• http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780

Back to top

© 2022 BetterEvaluation. All right reserved.

Practical Applications of Rapid Qualitative Analysis for Operations, Quality Improvement, and Research in Dynamically Changing Hospital Environments

• PMID: 36585315
• DOI: 10.1016/j.jcjq.2022.11.003

Background: Health care systems are in a constant state of change. As such, methods to quickly acquire and analyze data are essential to effectively evaluate current processes and improvement projects. Rapid qualitative analysis offers an expeditious approach to evaluate complex, dynamic, and time-sensitive issues.

Methods: We used rapid data acquisition and qualitative methods to assess six real-world problems the hospitalist field faced during the COVID-19 pandemic. We iteratively modified and applied a six-step framework for conducting rapid qualitative analysis, including determining if rapid methods are appropriate, creating a team, selecting a data collection approach, data analysis, and synthesis and dissemination. Virtual platforms were used for focus groups and interviews; templated summaries and matrix analyses were then applied to allow for rapid qualitative analyses.

Results: We conducted six projects using rapid data acquisition and rapid qualitative analysis from December 4, 2020, to January 14, 2022, each of which included 23 to 33 participants. One project involved participants from a single institution; the remainder included participants from 15 to 24 institutions. These projects led to the refinement of an adapted rapid qualitative method for evaluation of hospitalist-driven operational, research, and quality improvement efforts. We describe how we used these methods and disseminated our results. We also discuss situations for which rapid qualitative methods are well-suited and strengths and weaknesses of the methods.

Conclusion: Rapid qualitative methods paired with rapid data acquisition can be employed for prompt turnaround assessments of quality, operational, and research projects in complex health care environments. Although rapid qualitative analysis is not meant to replace more traditional qualitative methods, it may be appropriate in certain situations. Application of a framework to guide projects using a rapid qualitative approach can help provide structure to the analysis and instill confidence in the findings.

Copyright © 2022 The Joint Commission. Published by Elsevier Inc. All rights reserved.

• Focus Groups
• Qualitative Research
• Quality Improvement

Adaptability on Shifting Ground: a Rapid Qualitative Assessment of Multi-institutional Inpatient Surge Planning and Workforce Deployment During the COVID-19 Pandemic

• Original Research: Qualitative Research
• Published: 22 March 2022
• Volume 37 , pages 3956–3964, ( 2022 )

Cite this article

• Angela Keniston MSPH   ORCID: orcid.org/0000-0003-1399-2881 1 ,
• Matthew Sakumoto MD 2 ,
• Gopi J. Astik MD/MS 3 ,
• Andrew Auerbach MD, MPH 4 ,
• Shaker M. Eid MD/MBA 5 ,
• Kirsten N. Kangelaris MD/MAS 6 ,
• Shradha A. Kulkarni MD 6 ,
• Tiffany Lee MA 7 ,
• Luci K. Leykum MD/MBA/MSc 8 ,
• Anne S. Linker MD 9 ,
• Devin T. Worster MD MPH 10 &
• Marisha Burden MD 11  

1543 Accesses

9 Citations

9 Altmetric

Explore all metrics

During the initial wave of COVID-19 hospitalizations, care delivery and workforce adaptations were rapidly implemented. In response to subsequent surges of patients, institutions have deployed, modified, and/or discontinued their workforce plans.

Using rapid qualitative methods, we sought to explore hospitalists’ experiences with workforce deployment, types of clinicians deployed, and challenges encountered with subsequent iterations of surge planning during the COVID-19 pandemic across a collaborative of hospital medicine groups.

Using rapid qualitative methods, focus groups were conducted in partnership with the Hospital Medicine Reengineering Network (HOMERuN). We interviewed physicians, advanced practice providers (APP), and physician researchers about (1) ongoing adaptations to the workforce as a result of the COVID-19 pandemic, (2) current struggles with workforce planning, and (3) evolution of workforce planning.

Key Results

We conducted five focus groups with 33 individuals from 24 institutions, representing 52% of HOMERuN sites. A variety of adaptations was described by participants, some common across institutions and others specific to the institution’s location and context. Adaptations implemented shifted from the first waves of COVID patients to subsequent waves. Three global themes also emerged: (1) adaptability and comfort with dynamic change, (2) the importance of the unique hospitalist skillset for effective surge planning and redeployment, and (3) the lack of universal solutions.

Conclusions

Hospital workforce adaptations to the COVID pandemic continued to evolve. While few approaches were universally effective in managing surges of patients, and successful adaptations were highly context dependent, the ability to navigate a complex system, adaptability, and comfort in a chaotic, dynamic environment were themes considered most critical to successful surge management. However, resource constraints and sustained high workload levels raised issues of burnout.

Similar content being viewed by others

The impact of surge adaptations on hospitalist care teams during the COVID-19 pandemic utilizing a rapid qualitative analysis approach

Description of a Multi-faceted COVID-19 Pandemic Physician Workforce Plan at a Multi-site Academic Health System

The Impact of Hospital Capacity Strain: a Qualitative Analysis of Experience and Solutions at 13 Academic Medical Centers

Avoid common mistakes on your manuscript.

INTRODUCTION

Hospitalists have been at the forefront of the pandemic, serving as clinicians and operational leaders. 1 , 2 The COVID-19 pandemic required addressing the influx of patients to not only emergency departments, but also to medical wards and intensive care units. As a result, existing disaster plans had to be rapidly modified and deployed to address surges in inpatient volume, often by hospitalists in collaboration with other stakeholders across healthcare organizations. 1 A variety of organization-, team-, and individual-level adaptations were rapidly implemented in response to surges of patients during the first wave of the COVID-19 pandemic in March and April 2020. 3 , 4 Initial strategies included reduction of non-essential services, geographic cohorting of patients in respiratory isolation units (RIUs), implementing technology for communicating with and evaluating patients to reduce clinical staff exposure, allowing healthcare workers to opt out of direct care of COVID-19 patients, and deployment of healthcare workers from other specialties. 3 , 4 , 5 However, many of these strategies may not be sustainable practices.

In preparation for surge events, logistical planning for diagnostic testing, ensuring the availability of PPE, developing strategies for patient triage and cohorting, developing clinical protocols, addressing the physical and mental wellness of healthcare workers, developing communication plans, and surge planning specifically around key resources—physical space and beds, clinical and operational staff, equipment, and system coordination—should be addressed. 6 , 7 , 8 However, plans addressing these domains must be both systematic and highly adaptable. 6

In the subsequent months, most areas of the USA experienced a second, and sometimes third, wave of patients requiring hospitalization for COVID-19. 9 In response to these additional surges and continual significant challenges with the testing and treatment of COVID patients while also maintaining a safe work environment, institutions have updated, modified, and/or discontinued adaptations made earlier in the pandemic. 4 , 10 , 11 Our work provides novel insights regarding the ongoing challenges of sustained surges, the types of adaptations that have not been sustainable, and the new ways that the hospitalist skillset has been applied as the pandemic continues.

While a growing literature describes initial adaptations employed by hospitals and hospitalists, 1 , 3 , 12 , 13 , 14 further updates to workforce deployment and care processes with subsequent COVID-19 surges have not been described. This rapid qualitative evaluation of the inpatient surge planning and workforce deployment across multiple hospital medicine groups provides insight into the direct experience of hospital medicine clinicians and leaders who were responsible for both the development of surge plans and the delivery of care to patients in the setting of the implementation of surge plans. These focus groups were conducted as a part of efforts by the Hospital Medicine Reengineering Network (HOMERuN) collaborative to rapidly collect and disseminate information needed by hospitalists to respond to the COVID-19 pandemic. 15 Rapid qualitative methods are uniquely suited for quick assessment and evaluation while ensuring the same rigor of more traditional qualitative methods in time-sensitive situations. 16 , 17 , 18 , 19

The constraints and demand on hospital medicine clinicians are different from those felt by intensivists and emergency department staff. The participants of these focus groups describe adaptations of different groups, insight into the challenges of an evolving pandemic and continual surges of patients, and insight into their experience, as both surge planners and frontline clinicians, with the solutions implemented.

Human Subjects

The Colorado Multiple Institutional Review Board reviewed and approved this study as exempt from IRB review (COMIRB #: 21-4873).

Study Design

On December 4, 2020, we conducted five semi-structured focus groups with hospitalist physicians, advanced practice providers (APP), and hospitalist physician researchers participating in the Hospital Medicine Reengineering Network (HOMERuN), 20 a collaborative of hospitals, hospitalists, and multidisciplinary care teams founded in 2011. HOMERuN is a consortium of academic medical centers, primarily in urban settings, though geographically diverse with participating sites from the Northeast, Southeast, Midwest, West, and Northwest, as previously described by Auerbach et al. 20 This group mobilized to create workgroups that collated and shared best practices for the COVID-19 pandemic. These focus groups explored the changes in each participating hospital’s approach to workforce deployment and organization of care during the COVID-19 pandemic, and identified the types of workforce and surge planning issues with which hospitalists are currently grappling.

Setting and Participants

Participants of the monthly HOMERuN collaborative call were electronically notified in advance that focus groups would be conducted during the next regularly scheduled Zoom call (December 4, 2020). Hospitalist physicians, APPs, hospitalist physician researchers, residents, and patient representatives participating in the Hospital Medicine Reengineering Network (HOMERuN) and present for the monthly collaborative call were included in the focus groups. The only exclusion criterion was refusal to participate. Individuals present for the call were not offered any incentive to participate.

At the start of the meeting, individuals who elected to call in were informed again of the plan to conduct focus groups, which would be recorded, and offered the chance to ask questions of moderators. If the attendees agreed to proceed, they were placed in a separate virtual breakout room for focus group participation, with a moderator assigned to each room. Each focus group was approximately 30 min in duration and had approximately six participants.

Interview Guide

The focus group guide was developed by the members of the HOMERuN workforce planning workgroup, convened in March 2020 to assess workforce and organizational adaptations undertaken in response to COVID-19. We asked participants to consider the following questions: (1) What adaptations have proved most useful to you? (2) What are you struggling with right now? (3) What are you changing now? (4) What important changes occurred between your first surge and later waves? Moderator Guide shown in the Appendix .

Data Collection

Prior to beginning each focus group, participants granted permission to record the conversation. During the focus groups, the moderators (MB, DW, GA, SK, AL, MS, AK) made field notes and observations to supplement the audio recordings. The audio recordings and field notes were used for the analysis rather than transcriptions of the focus groups.

Our analysis was conducted in a two-step process using a rapid qualitative analytic approach. 19 , 21 , 22 , 23 , 24 , 25 First, a team member who did not participate in the specific focus group used a standard template to create a summary of each group’s session, incorporating both the audio recording and the moderator’s field notes (MB, GA, SK, AL, MS, AK, LL). Second, all workgroup team members participated in creating an analysis matrix 21 of the summaries of each focus group. Each row was a focus group and each column referred to a unique question we asked each focus group. One workgroup member completed the matrix by logging key points summarized for each focus groups’ discussion of each question into the matrix (AK). Individually, workgroup members then identified themes and subthemes across all focus group discussions of each question (MB, GA, SK, AL, MS, AK, LL). As a group, workgroup team members then met to discuss and reach consensus regarding themes identified (MB, DW, GA, SK, AL, MS, AK, LL). As the focus groups were conducted simultaneously, all data were used in the analysis rather than in considering data saturation. Member checking, a technique for confirming the credibility of results, was conducted. 26 Two members of the workgroup who participated in a focus group but who did not moderate a focus group, create a summary, or participate in the analysis reviewed the findings to confirm the themes reflected their experience as a focus group participant (SE, KK).

Physicians, APPs, and physician researchers from 24 hospitals participated, representing 52% of HOMERuN sites, with 29 (88%) hospitalist physicians, three (9%) in another category (an APP, a resident, and a patient representative), and one (3%) unidentified participant. All but one of the hospitals represented were academic hospitals. One participant reported working at a VA hospital. Four hospitals had two participant representatives, one hospital had six participants, and 19 had one participant representative in attendance.

Participants described a variety of adaptations, some common across institutions and others specific to the institution’s location and context. Adaptations implemented shifted from the first waves of COVID patients to subsequent waves. Table 1 summarizes these adaptations and Figure 1 illustrates changes from initial to subsequent waves of patients. Table 2 highlights exemplar quotes for themes identified across the domains explored during the focus groups.

Changes over time.

Adaptations That Have Proved Most Useful

Managing high-capacity situations.

All groups discussed how they approached the decision to add capacity to care for a rapid increase in patients. Overall occupancy across the system or department census was often used to make decisions about adding providers to a particular site or deploying providers to other hospitals in the system. Often a decision to add capacity was dependent on how stretched providers felt, as opposed to specific triggers based on provider-to-patient ratios (which was felt to be challenging to define). Tiered surge plans were a commonly used adaptation for adding capacity, developed with guidance from institutional stakeholders, although participants described significant variability in such plans, with a range in the number of tiers from three to 36 levels. One participant described using triagists to direct patient flow and manage capacity, and this was noted to be helpful.

Recruitment and Staffing Strategies

Goodwill and volunteerism were insufficient to maintain adequate staffing over time. Some participants noted additional payments or compensation for working additional shifts. A number of participants described shifting from an opt-in approach for caring for COVID patients to an opt-out approach. Participants also described a shift from trying to cohort COVID patients on a small number of teams to distributing COVID patients to specialty teams depending on the patient’s primary complaint. This shift was intended to more uniformly distribute work across a broader group of clinicians. From initial to subsequent waves, geographic cohorting was reported as less operationally feasible because numbers of non-COVID patients were rising concomitantly with COVID numbers, and hospitals were typically at or beyond 100% utilization. Both APPs and non-hospitalists were deployed to extend admitting capacity, with varied models including direct care for COVID patients under the supervision of hospitalists or working remotely to write notes for the primary team. Several innovative staffing models were described. Examples included a virtualist model, 27 in which attending physicians rounded from home using iPads and called families to provide updates, or hiring “COVID-ists.” A flexible APP deployment model was also described where assignments were made based on both clinical expertise and patient census. When patient volumes were high, APPs were used for independent clinical care of patients, while at other times APPs were redeployed to care coordination tasks.

At first, hospitals sought to protect resident education, using physician attendings or APPs to care for COVID patients. Some participants described residents being asked to flex up to meet demand, but this request was felt to be at the expense of education. One participant described distributing housestaff across all teams rather than maintaining teaching and non-teaching teams. Other participants reported that housestaff were only in the ICUs. Most participants described excluding learners from COVID care, at least at first, because using residents required ACGME emergency authorization, though many reported wanting to include learners in the care of COVID patients particularly during subsequent waves. Participants recognized the value of residents or high-functioning interns who know how to manage inpatient logistics to support the care team.

Delivery Settings Outside of Hospital

One participant described setting up a field hospital but only for certain patients who were mobile and did not have any behavioral health issues. Another participant described converting a long-term acute-care (LTAC) hospital to a COVID hospital, which was considered very successful. A number of participants reported redistributing both patients and providers across a system of hospitals to manage surges in volume.

Communication Strategies

Participants described the importance of robust communication, including checking in with hospitalist and ICU colleagues and communication about current COVID-19 evidence and treatment guidelines. However, participants reported struggling to decide what the right frequency of communication might be, titrated to surge level, anxiety level, and knowledge level. Clinical pathways, order sets, and protocols were used to communicate current treatment guidelines as new clinical staff were deployed.

Persistent Struggles

Resource constraints.

These included insufficient negative pressure rooms, limited ICU capacity, and shortages in nursing and respiratory therapy staff. The most common concerns noted were nursing and respiratory therapy turnover. Organizations described significant attention paid to maintaining nursing ratios. Participants were concerned that even if there were sufficient beds or provider workforce, other disciplines within the hospital, such as nursing or respiratory therapy, were short-staffed and unable to adequately handle surges in volume. From the first wave to subsequent waves, participants described struggling with a surge in non-COVID patients and a concomitant resumption of elective surgeries and outpatient clinics, which decreased available beds, staff, and other resources for COVID patients. Space limitation was an issue not only for patient care, but also for providers attempting to distance from each other. Clinicians were in need of space to practice social distancing, especially given institutional rules about where staff were allowed to eat or take breaks. Finally, there was concern that a lack of redundancy in staffing plans made it difficult to flex up to cover shifts when hospitalists were unable to work.

Ongoing Struggle to Determine the Best Workforce Deployment Strategies

This was particularly true regarding the role of residents and balancing resident service and educational activities. Some found over time that trainees were more eager to care for COVID patients than originally thought, that leaving residents out of the workforce had unforeseen consequences, and that they could be included in the workforce safely. Participants reported having to continue to work on methods to determine the best workforce redeployment strategies and how to most effectively reorient new or returning workforce members. The higher non-COVID patient volume (including surgical/procedural and non-COVID medical patients) after the first wave complicated decision-making. Ultimately, there was a fixed workforce with limited ability to flex upwards without major structural changes (i.e., the workforce that was originally available from canceling clinics, canceling surgeries, etc. became less available and were now also facing increased volumes).

Important Changes that Occurred Between the First Surge and Later Waves

Changes in attitudes/moral issues/burnout.

Participants described heavy reliance on goodwill and volunteerism with the first wave, but that with later waves, providers were fatigued and goodwill had faded. Local factors that influenced decisions about team size, number of teams, and which providers staffed teams included the use of care protocols, order sets, and guidelines to support redeployed clinicians and hospitalist supervision of redeployed clinicians as well as burnout among providers. There were differing opinions on running workloads higher than normal versus trying to find/add in additional providers to manage the high numbers of patients. Participants reported that deploying subspecialists with historically less inpatient experience was challenging because subspecialists often lacked the hospital systems knowledge required to deliver inpatient care (i.e., working knowledge of how to navigate the electronic health record and other operational factors). To support specialists caring for COVID patients, participants described creating a COVID consult service to answer any COVID-specific questions and provide COVID-specific medical management as opposed to admitting patients to a COVID-specific team.

Burnout Increasingly Constrained the Ability to Adapt

Participants discussed the challenge of continued changes on a workforce experiencing burnout. Participants also described a normalization of caring for COVID patients that allowed a larger group of clinicians to be involved in COVID care. However, there was concern that these continual higher volumes are contributing to reduced morale, fatigue, and burnout, though it is unclear whether COVID or non-COVID volumes are the bigger issue.

Global Themes

Three global themes emerged across discussions of all questions: (1) adaptability and comfort with dynamic change, (2) the importance of the hospitalist skillset to effective surge planning and workforce deployment, and (3) the lack of universal solutions, in which there is no easy way to surge.

The healthcare workforce was redeployed in a variety of ways as the situation evolved. Iterative improvements were made with each fluctuation in COVID-19 patient volumes, and participants reported that their hospitalist groups become more comfortable with dynamic change over time. Factors affecting level of comfort with change included communication, degree of burnout, and the number and types of innovations. Adaptations both fostered and reinforced more functional collaborations and partnerships with clinical colleagues, and participants reported being able to continue to leverage improved collaborations in the future. Systems knowledge and systems process improvement have always been central to hospitalist work. 28 , 29 , 30 Participants felt that this skillset was critical to successful adaptations and augmented the hospitalist clinical skillset. The importance of tacit, implicit contextual knowledge in a time of rapid change was also apparent, and was felt to be a key reason why some clinicians were highly valued. However, there were no universal solutions described—the success of any one tactic for surge planning was highly dependent on the context in which it was applied. Challenges resulted from insufficient staffing and resources, often requiring clinical staff to flex up to meet demand, or flex into new roles that are not familiar or comfortable.

In this rapid qualitative evaluation of continued institutional adaptations in response to second and third waves of COVID-19, participants described a variety of useful adaptations but also described continued ongoing struggles. Despite multiple iterations of surge practices across multiple institutions, there were limited universal solutions to manage the surges beyond ensuring sufficient staffing as the ultimate crisis was a lack of resources (providers and other ancillary staff and sometimes critical other resources) to match the magnitude of the surge.

Prior to the COVID-19 pandemic, hospitals associated with academic medical centers commonly struggled with hospital capacity strain, or patient volume in excess of the available beds, clinical staff, and equipment. 31 Previous work has noted that hospital capacity strain is complex and difficult to predict and the interventions that were perceived to have worked the best when facing strain were to ensure appropriate resources; however, less costly solutions were often deployed. 31 These decisions unfortunately negatively impact the workforce, and some studies have suggested that they may lead to negative patient outcomes as well as operational outcomes. COVID-19 has further stressed an already strained system. Work by Aiken et al. as well as Elliot et al. has highlighted that when workloads exceed certain thresholds, quality and operational outcomes decline. 32 , 33 , 34 Kamalahmadi et al. noted that it may actually be in the institution’s financial interest to lower census thresholds to optimize patient flow. 35

Despite continual innovation and a comfort with dynamic change, hospitals and hospitalists struggled to figure out how to best maximize the workforce for current and future surges in the setting of insufficient workforce, primarily because there is no perfect approach to navigating surges in patient volume without having a sufficient workforce supply. Predicting when the surge occurs is also challenging especially when complicated by a baseline increase in patients needing hospital care. Additionally, communicating surge needs was complicated as thresholds varied and were challenging to define in the setting of a continually evolving situation.

It was starkly apparent that agile systems that are capable of rapid adaptation were vital for meeting the demands of the dynamic US healthcare environment during the first and subsequent waves of COVID-19 hospitalizations and the hospitals in this study clearly adapted rapidly. Participants described the importance of creativity in designing approaches for local problems and comfort with a dynamic atmosphere in which consistent change was accepted as the new normal. Although there were a number of commonalities, such as using volunteers, APP staff, or the use of field hospitals, no single adaptation emerged across focus groups as a universal approach. High-level recommendations exist in the literature for managing an influx of patients due to a disaster or pandemic 36 , 37 ; however, a one-size approach fitting all situations does not exist. Hospitals can learn from one another, but will have to adapt in response to the contextual factors at their hospital.

The hospitalist skillset, beyond the clinical knowledge required for delivering high-quality inpatient care to medically complex patients, includes operational expertise and an ability to navigate complex systems. 28 , 38 In rapidly evolving, high-uncertainty situations like the pandemic, relationships provide the basis for effective communication, sense-making, and learning. 28 , 38 Hospitalists uniquely hold the relational and operational knowledge to be most effective under such conditions since they constantly navigate healthcare systems issues and are involved in managing process improvements for the inpatient setting.

As identified by participants in our focus groups, system constraints like staffing shortages and insufficient or irregular communication inhibited the ability of the workforce to innovate. As COVID-19 unfortunately becomes the norm of hospital care with likely intermittent upticks in patient numbers, hospital systems and hospitalists groups must begin to evolve their surge strategies to ensure proper staffing with sufficient flexibility to manage these surges in less disruptive ways.

Adaptations considered useful across participants include creating tiered surge plans, redeploying non-hospitalist physicians, APPs, and subspecialists to care for COVID-19 patients, redistributing COVID patients to specialist consult teams based on patients’ primary disease complaint, frequently collating and disseminating updated COVID-related evidence and guidelines, and creating and sharing COVID care pathways and order sets to standardize treatment. While most participants described excluding learners from COVID care, participants recognized the value of residents or high-functioning interns and many reported seeking to include learners in the care of COVID patients during subsequent waves. Finally, some participants described developing plans to care for COVID patients outside of the traditional hospital setting, including field hospitals and long-term acute-care hospitals converted to COVID patient care.

Our work has several strengths. This study employed rapid qualitative methods, useful in dynamic, real-world situations where the insights gathered are vital for immediate real-time application. 19 , 21 , 22 , 23 , 24 , 25 While the methods we used were not designed to quantify the strategies described by participants, qualitative analysis allows a deeper understanding of the context in which various strategies were implemented, the perspectives of frontline physicians and APPs as well as those developing operational plans, and the role of hospital medicine in the response to the COVID-19 pandemic. We interviewed a diverse cross section of hospitalists including physicians involved not only in frontline clinical care but also in COVID-related adaptations, APPs, and physician researchers. At the time the focus groups were conducted, each participant had been involved in one or more surges of the COVID-19 pandemic.

Because our participants were limited to members of the HOMERuN, our results may not be completely applicable to non-academic settings. In addition, there was a potential for participation bias if hospitalists who attended the meeting and participated in the focus groups were different in some way from those who did not attend and/or participate in the focus groups. The focus groups included physicians, APPs, and physician researchers working in the field of hospital medicine, so we did not capture the voice of the providers from other specialties. While participants represented hospitals from across the USA, individuals who elected to call in were assured during the focus groups that we would protect their confidentiality so we did not collect and analyze the qualitative data in such a way that we can assign specific institutions to specific solutions described.

Finally, while these focus groups were conducted more than a year ago, continual surges of patients and diminishing resources including space and clinical staff have necessitated adapting and evolving surge plans. Disseminating the findings from these focus groups would provide additional information, ideas, and potentially useful adaptations as hospitals and hospitalists across the country are faced with the ongoing challenges of the COVID-19 pandemic.

Hospitals continued to evolve in the ways they have adapted to the challenges of the COVID pandemic. Few approaches were universally effective in managing surges of COVID-19 patients, and successful adaptations were highly context dependent. Hospitalists’ local systems knowledge has uniquely positioned them to manage ongoing adaptations in response to COVID-19, but resource constraints and sustained high workload levels raised issues of burnout. The findings of this rapid qualitative evaluation bring to light the challenge of creating single solutions that will be applicable across hospitals that operate in different ways, and underscore the need for further research to identify particular workflows that are associated with improved patient-relevant outcomes.

Bowden K, Burnham EL, Keniston A, et al. Harnessing the Power of Hospitalists in Operational Disaster Planning: COVID-19. J Gen Intern Med. 2020;35(9):2732-2737.

Article   PubMed   PubMed Central   Google Scholar  

Persoff J, Ornoff D, Little C. The Role of Hospital Medicine in Emergency Preparedness: A Framework for Hospitalist Leadership in Disaster Preparedness, Response, and Recovery. J Hosp Med. 2018;13(10):713-718.

Article   PubMed   Google Scholar  

Auerbach A, O'Leary KJ, Greysen SR, et al. Hospital Ward Adaptation During the COVID-19 Pandemic: A National Survey of Academic Medical Centers. J Hosp Med. 2020;15(8):483-488.

Linker AS, Kulkarni SA, Astik GJ, Keniston A, Sakumoto M, Eid SM, Burden M, Leykum LK; HOMERuN COVID-19 Collaborative Working Group. Bracing for the Wave: a Multi-Institutional Survey Analysis of Inpatient Workforce Adaptations in the First Phase of COVID-19. J Gen Intern Med . 2021;36(11):3456-3461.

Kumar SI, Borok Z. Filling the Bench: Faculty Surge Deployment in Response to the Covid-19 Pandemic. NEJM Catalyst. Published 10/29/2020. Accessed 1/20/2022.

Anesi GL, Lynch Y, Evans L. A Conceptual and Adaptable Approach to Hospital Preparedness for Acute Surge Events Due to Emerging Infectious Diseases. Crit Care Explor. 2020;2(4):e0110-e0110.

Vranas KC, Golden SE, Mathews KS, et al. The Influence of the COVID-19 Pandemic on ICU Organization, Care Processes, and Frontline Clinician Experiences: A Qualitative Study. Chest. 2021;160(5):1714-1728.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Coates A, Fuad AO, Hodgson A, Bourgeault IL. Health workforce strategies in response to major health events: a rapid scoping review with lessons learned for the response to the COVID-19 pandemic. Hum Resour Health. 2021;19(1):154.

Roser Max RH, Ortiz-Ospina E, Hasell J. Coronavirus Pandemic (COVID-19). https://ourworldindata.org/coronavirus . Published 2020. Accessed 2/28/2021.

Washington DC: Office of the Inspector General; April 3, 2020. Report no. OEI-06-20-00300.

Mhango M, Dzobo M, Chitungo I, Dzinamarira T. COVID-19 Risk Factors Among Health Workers: A Rapid Review. Safety Health Work. 2020;11(3):262-265.

Article   Google Scholar  

Garg M, Wray CM. Hospital Medicine Management in the Time of COVID-19: Preparing for a Sprint and a Marathon. J Hosp Med. 2020;15(5):305-307.

Biala D, Siegel EJ, Silver L, Schindel B, Smith KM. Deployed: Pediatric Residents Caring for Adults During COVID-19's First Wave in New York City. J Hosp Med. 2020;15(12):763-764.

Bloom-Feshbach K, Berger RE, Dubroff RP, McNairy ML, Kim A, Evans AT. The Virtual Hospitalist: a Critical Innovation During the COVID-19 Crisis. J Gen Intern Med. 2021:1-4.

HOMERuN. http://hospitalinnovate.org Accessed July 23, 2017.

Beebe J. Rapid qualitative inquiry : a field guide to team-based assessment / James Beebe . 2nd ed. Lanham, Maryland: Rowman & Littlefield; 2014.

Google Scholar  

Gale RC, Wu J, Erhardt T, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science. 2019;14(1):11.

Lewinski AA, Crowley MJ, Miller C, et al. Applied Rapid Qualitative Analysis to Develop a Contextually Appropriate Intervention and Increase the Likelihood of Uptake. Med Care. 2021;59.

Hamilton A. Qualitative methods in rapid turn-around health services research. Paper presented at: Veterans Affairs (VA) Health Services Research & Development Cyberseminar; 12/11/2013, 2013.

Auerbach AD, Patel MS, Metlay JP, et al. The Hospital Medicine Reengineering Network (HOMERuN): a learning organization focused on improving hospital care. Acad Med. 2014;89(3):415-420.

Averill JB. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res. 2002;12(6):855-866.

Brown DR, Hernández A, Saint-Jean G, et al. A participatory action research pilot study of urban health disparities using rapid assessment response and evaluation. Am J Public Health. 2008;98(1):28-38.

Burks DJ, Robbins R, Durtschi JP. American Indian gay, bisexual and two-spirit men: a rapid assessment of HIV/AIDS risk factors, barriers to prevention and culturally-sensitive intervention. Cult Health Sex. 2011;13(3):283-298.

Vindrola-Padros C, Chisnall G, Cooper S, et al. Carrying Out Rapid Qualitative Research During a Pandemic: Emerging Lessons From COVID-19. Qual Health Res. 2020;30(14):2192-2204.

Zuchowski JL, Chrystal JG, Hamilton AB, et al. Coordinating Care Across Health Care Systems for Veterans With Gynecologic Malignancies: A Qualitative Analysis. Med Care. 2017;55 Suppl 7 Suppl 1:S53-s60.

Birt L, Scott S, Cavers D, Campbell C, Walter F. Member Checking: A Tool to Enhance Trustworthiness or Merely a Nod to Validation? Qual Health Res. 2016;26(13):1802-1811.

Bloom-Feshbach K, Berger RE, Dubroff RP, McNairy ML, Kim A, Evans AT. The Virtual Hospitalist: a Critical Innovation During the COVID-19 Crisis. J Gen Intern Med. 2021;36(6):1771-1774.

Wachter RM, Goldman L. The emerging role of "hospitalists" in the American health care system. N Engl J Med. 1996;335(7):514-517.

Article   CAS   PubMed   Google Scholar  

Wachter RM, Goldman L. Zero to 50,000 - The 20th Anniversary of the Hospitalist. N Engl J Med. 2016;375(11):1009-1011.

The core competencies in hospital medicine: a framework for curriculum development by the Society of Hospital Medicine. J Hosp Med. 2006;1:2-95.

Arogyaswamy S VN, Keniston A, Apgar S, Bowden K, Diaz M, Kantor M, McBeth L, Burden M. Hospital-capacity strain: A qualitative analysis of solutions utilized by academic medical centers. J Gen Intern Med.

Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288(16):1987-1993.

Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and quality of care: cross-national findings. Int J Qual Health Care. 2002;14(1):5-13.

Elliott DJ, Young RS, Brice J, Aguiar R, Kolm P. Effect of hospitalist workload on the quality and efficiency of care. JAMA Intern Med. 2014;174(5):786-793.

Kamalahmadi M, Bretthauer K, Helm J, et al. Mixing It Up: Operational Impact of Hospitalist Caseload and Case-mix. Baruch College Zicklin School of Business Research Paper No 2019-10-02. 2019.

Hick JL, Einav S, Hanfling D, et al. Surge capacity principles: care of the critically ill and injured during pandemics and disasters: CHEST consensus statement. Chest. 2014;146(4 Suppl):e1S-e16S.

Einav S, Hick JL, Hanfling D, et al. Surge capacity logistics: care of the critically ill and injured during pandemics and disasters: CHEST consensus statement. Chest. 2014;146(4 Suppl):e17S-43S.

O'Leary KJ, Williams MV. The evolution and future of hospital medicine. Mt Sinai J Med. 2008;75(5):418-423.

Download references

Acknowledgements

Thanks to members of the HOMERuN COVID-19 Collaborative Group: Brigham and Women’s Hospital—Stephanie Mueller, MD MPH and Jeffrey Schnipper, MD MPH; Elsevier Publishing—Jennifer Goldstein, MD MSc; Emory University School of Medicine—Khaalisha Ajala, MD MBA; Obsinet Tadesse Merid, MD; and TaRessa Wills, MD; Johns Hopkins University School of Medicine—Shaker M. Eid, MD MBA; Ifedayo Kuye, MD MBA; and Amit Pahwa, MD; Mount Sinai Hospital—Krishna Chokshi, MD; Horatio Holzer, MD; Chris Kellner, MD; Anne S. Linker, MD; and Vinh Nguyen, MD; Northwestern University Feinberg School of Medicine—Gopi Astik, MD MSc; Oregon Health and Science University—James Anstey, MD; James Clements, MD; and Anya Solotskaya, MD; Penn State University College of Medicine—Omrana Pasha, MD; Stanford University—Samantha Wang, MD; The Ohio State University College of Medicine—Jennifer Allen, MD and Kristen Lewis, MD; Thomas Jefferson University Hospital and Mayo Clinic—Alan A. Kubey, MD FACP; University of California, San Francisco—Andrew Auerbach, MD MPH; Amy Berger, MD PhD; Sneha Daya, MD; Archna Eniasivam, MD; Armond Esmaili, MD; Margaret Fang, MD MPH; Shubhra Gupta, MD; James Harrison, PhD; Emily Insetta, MD; Kirsten Kangelaris, MD; Kristen Kipps, MD; Zhenya Krapivinsky, MD; Shradha Kulkarni, MD; Rashmi Manjunath, MD; Sirisha Narayana, MD; Nishita Nigam, MD; Anna Parks, MD; Sumant Ranji, MD; Lekshmi Santhosh, MD, MAEd; Yalda Shahram, MD; Noa Simchoni, MD PhD; Matthew Sakumoto, MD; and Charlie M. Wray, DO; University of Chicago—Elizabeth Murphy, MD SFHM; Greg Ruhnke, MD; and Andrew Schram, MD MBA; University of Colorado School of Medicine—Marisha Burden, MD; Amira del Pino-Jones, MD; Angela Keniston, MSPH; Chris King, MD; and Katie E. Raffel, MD; University of Florida College of Medicine—Nila Radhakrishnan, MD and Nick Kattan, MD; University of Iowa Carver College of Medicine—Ethan Kuperman, MD MSc; University of Kentucky—John Romond, MD; Joe Sweigart, MD FHM FACP; and Sarah Vick, MD; University of Miami Health System—Chadwick Flowers, MD; Efren Manjarrez, MD; and Magdalena Murman, MD MAEd; University of New Mexico School of Medicine—Charles Pizanis and Kendall Rogers, MD CPE FACP SFHM; University of Pennsylvania—Ryan Greysen, MD MPH; Matthew Mitchell, PhD; and Todd Hecht, MD; University of Pittsburgh—Gena M. Walker, MD FHM; University of Texas, Austin Dell Medical School—W. Michael Brode, MD; Luci K. Leykum, MD MBA MSc; Kirsten Nieto, MD; and Sherine Salib, MD FACP; University of Virginia School of Medicine—Rachel Weiss, MD; University of Washington—Dan Cabrera, MD MPH and Naomi Shike, MD MSc; University of Wisconsin School of Medicine and Public Health—Blair P. Golden, MD MS; Sean O’Neill, MD; and David Sterken, MD; Virginia Commonwealth University School of Medicine—Sarah Hartigan, MD; Weill Cornell Medicine—Devin T. Worster, MD MPH; Yale School of Medicine—Rebecca Slotkin, MD; HOMERuN PFAC—Martie Carnie Catherine Hanson and Georgiann Ziegler.

The views expressed do not represent the position of the Department of Veterans Affairs or other organizations affiliated with the authors.

Author information

Authors and affiliations.

Division of Hospital Medicine, University of Colorado School of Medicine, 12401 E. 17th Avenue, Mail Stop F782, Aurora, CO, 80045, USA

Angela Keniston MSPH

Division of General Internal Medicine, University of California, San Francisco School of Medicine, San Francisco, CA, USA

Matthew Sakumoto MD

Division of Hospital Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA

Gopi J. Astik MD/MS

University of California, San Francisco School of Medicine, San Francisco, CA, USA

Andrew Auerbach MD, MPH

Division of Hospital Medicine, Johns Hopkins Bayview Medical Center, Baltimore, MD, USA

Shaker M. Eid MD/MBA

Division of Hospital Medicine, University of California, San Francisco, San Francisco, CA, USA

Kirsten N. Kangelaris MD/MAS & Shradha A. Kulkarni MD

Tiffany Lee MA

The University of Texas at Austin, Dell Medical School, South Texas Veterans Health Care System, San Antonio, TX, USA

Luci K. Leykum MD/MBA/MSc

Division of Hospital Medicine, Mount Sinai Hospital/Icahn School of Medicine at Mount Sinai, New York, NY, USA

Anne S. Linker MD

Section of Hospital Medicine, Division of General Internal Medicine, Weill Cornell Medicine, New York, NY, USA

Devin T. Worster MD MPH

Division of Hospital Medicine, University of Colorado School of Medicine, Aurora, CO, USA

Marisha Burden MD

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Angela Keniston MSPH .

Ethics declarations

Conflict of interest.

The authors declare that they do not have a conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

(DOCX 83 kb)

Rights and permissions

Reprints and permissions

About this article

Keniston, A., Sakumoto, M., Astik, G.J. et al. Adaptability on Shifting Ground: a Rapid Qualitative Assessment of Multi-institutional Inpatient Surge Planning and Workforce Deployment During the COVID-19 Pandemic. J GEN INTERN MED 37 , 3956–3964 (2022). https://doi.org/10.1007/s11606-022-07480-x

Download citation

Received : 13 October 2021

Accepted : 03 March 2022

Published : 22 March 2022

Issue Date : November 2022

DOI : https://doi.org/10.1007/s11606-022-07480-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• hospital medicine
• workforce planning
• surge planning
• focus groups
• qualitative
• Find a journal
• Publish with us
• Track your research

• Methodology
• Open access
• Published: 02 July 2021

Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR)

• Andrea L. Nevedal   ORCID: orcid.org/0000-0003-3859-8493 1   na1 ,
• Caitlin M. Reardon 2   na1 ,
• Marilla A. Opra Widerquist 2 ,
• George L. Jackson 3 , 4 , 5 , 6 ,
• Sarah L. Cutrona 7 , 8 , 9 ,
• Brandolyn S. White 3 &
• Laura J. Damschroder 2  

Implementation Science volume  16 , Article number:  67 ( 2021 ) Cite this article

33k Accesses

140 Citations

37 Altmetric

Metrics details

Qualitative approaches, alone or in mixed methods, are prominent within implementation science. However, traditional qualitative approaches are resource intensive, which has led to the development of rapid qualitative approaches. Published rapid approaches are often inductive in nature and rely on transcripts of interviews. We describe a deductive rapid analysis approach using the Consolidated Framework for Implementation Research (CFIR) that uses notes and audio recordings. This paper compares our rapid versus traditional deductive CFIR approach.

Semi-structured interviews were conducted for two cohorts of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE). The CFIR guided data collection and analysis. In cohort A, we used our traditional CFIR-based deductive analysis approach (directed content analysis), where two analysts completed independent in-depth manual coding of interview transcripts using qualitative software. In cohort B, we used our new rapid CFIR-based deductive analysis approach (directed content analysis), where the primary analyst wrote detailed notes during interviews and immediately “coded” notes into a MS Excel CFIR construct by facility matrix; a secondary analyst then listened to audio recordings and edited the matrix. We tracked time for our traditional and rapid deductive CFIR approaches using a spreadsheet and captured transcription costs from invoices. We retrospectively compared our approaches in terms of effectiveness and rigor.

Cohorts A and B were similar in terms of the amount of data collected. However, our rapid deductive CFIR approach required 409.5 analyst hours compared to 683 h during the traditional deductive CFIR approach. The rapid deductive approach eliminated $7250 in transcription costs. The facility-level analysis phase provided the greatest savings: 14 h/facility for the traditional analysis versus 3.92 h/facility for the rapid analysis. Data interpretation required the same number of hours for both approaches.

Our rapid deductive CFIR approach was less time intensive and eliminated transcription costs, yet effective in meeting evaluation objectives and establishing rigor. Researchers should consider the following when employing our approach: (1) team expertise in the CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

Peer Review reports

Contributions to the literature

Published rapid qualitative analysis approaches often use transcripts; our approach shows how notes and verification with audio recordings can be used to ensure rigor while saving time and eliminating transcription costs.

Published rapid qualitative analysis approaches often utilize inductive approaches; our approach shows how to conduct deductive rapid analysis using the Consolidated Framework for Implementation Research (CFIR), which allows researchers to compare results more easily across studies.

CFIR users have expressed difficulty using the framework because our traditional analysis approach is resource intensive; the rapid analysis approach described here may facilitate the use of the CFIR for experienced users.

Qualitative methods are invaluable for gathering in-depth information about “how and why efforts” to implement Evidence-Based Innovations (EBIs) succeed or fail [ 1 ]. As a result, qualitative approaches (alone or within mixed methods) are foundational for implementation scientists seeking to identify and understand factors that help or hinder the implementation and use of EBIs in real-world settings [ 2 , 3 ]. Traditional qualitative approaches, however, are resource intensive, which challenges constrained study timelines and budgets. This is especially problematic in studies where scientists need real-time data to inform the process of implementation [ 4 ].

Consequently, qualitative researchers are working to develop methods that balance rigor and efficiency. The need for this balance is particularly salient in healthcare, where treatments and interventions are rapidly evolving, and evaluations of such interventions are constrained by limited timelines, funding, and staffing [ 5 ]. As a result, rapid assessment, which often involves streamlined processes for qualitative data collection and analysis, is gaining increased attention as a way to support quicker implementation and dissemination of EBIs to reduce delays in translating clinical research into practice [ 6 , 7 , 8 , 9 , 10 ].

An important element of rapid assessment is rapid qualitative analysis, which is the focus of this paper. Traditionally, qualitative analysis approaches have been resource intensive and occur over a longer timeframe; they include, but are not limited to, constant comparison, content, discourse, or thematic analysis [ 11 , 12 , 13 , 14 , 15 , 16 ]. Many traditional qualitative analysis approaches include in-depth manual coding of transcripts using software programs. In contrast, rapid qualitative analysis is deliberately streamlined and designed to be less resource intensive in order to meet a shorter timeframe [ 17 , 18 , 19 ]. Rapid qualitative analysis may involve eliminating transcription altogether or speeding up transcription processes [ 19 ] and then summarizing data into post-interview notes, templates based on the interview guide, and/or matrix summaries rather than in-depth manual coding of transcripts [ 9 , 17 , 18 , 19 , 20 , 21 ]. Though rapid analysis is not wedded to a particular approach (e.g., content or thematic), some traditional qualitative analysis approaches may be more difficult to streamline. Rapid qualitative analysis is crucial when results are needed to quickly develop or modify implementation strategies and/or inform stakeholders or operational partners [ 5 , 7 , 9 , 10 , 19 ]. Rapid qualitative analysis is also useful during longitudinal implementation research since data points can become unwieldy, and results may be needed to inform future waves of data collection [ 22 ].

Hamilton developed a rapid qualitative analysis approach that summarizes transcript data into templates using domains aligned with interview questions; summary points are then distilled into a matrix organized by domain and participant for analysis and interpretation [ 18 ]. Gale et al. adapted this rapid approach in a process evaluation of academic detailing and compared it with a traditional analysis approach [ 23 ]. Their rapid approach involved summarizing transcripts into a template and then mapping themes onto the Consolidated Framework for Implementation Research (CFIR), a determinant framework that defines constructs across five domains of potential influences on implementation [ 23 , 24 , 25 , 26 ]. Gale et al. demonstrated consistency between results from rapid qualitative analysis versus traditional qualitative analysis. The traditional approach, however, “took considerably (69 days) longer than the rapid analysis to complete” [ 23 ]. Similarly, Holdsworth et al. noted that their modified version of rapid qualitative analysis “produced contextually-rich information” and can be used to save “days and weeks of costly transcription and analysis time” [ 27 ]. Except for Taylor et al.’s [ 20 ] comparison of rapid and thematic analysis, most rapid analysis literature focuses on daily duration and does not quantify reductions in analyst hours and costs at the activity level versus the project overall.

The rapid approaches described by Hamilton and Gale et al. rely on verbatim transcripts, which means teams must wait for transcription to be completed to proceed with rapid or traditional analyses. In contrast, Neal et al. [ 28 ] developed an approach to rapidly identify themes directly from audio recordings. However, Gale et al. [ 23 ] noted that because this approach relies on general domains, rather than framework informed codes, it “limits one’s ability to compare findings across projects unless findings are [subsequently] mapped to a framework.” As implementation scientists using the CFIR to guide our evaluations, we sought to build on prior rapid analysis approaches by developing a CFIR informed deductive rapid analysis process using notes and audio recordings. The objective of this article is to compare two different qualitative analysis processes using the CFIR: a traditional deductive approach using transcripts and a rapid deductive approach using notes and audio recordings.

Evaluation background

We conducted a mixed-methods evaluation of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE), which seeks to identify and diffuse EBIs. These EBIs include innovations supported by evidence from research studies and administrative or clinical experience [ 29 , 30 ] and strive to address patient, staff, and/or facility needs. The DoE hosts an annual “Shark Tank” competition, in which VHA leaders compete to implement an EBI with 6 months of external implementation support; for additional detail, see previous publications [ 31 , 32 , 33 , 34 ]. As part of a national evaluation of the DoE, we identified barriers and facilitators to the implementation of these EBIs in VHA facilities using semi-structured interviews [ 31 ]. The qualitative interview and analysis team included CR (MPH, a senior qualitative analyst and CFIR expert user) and AN (PhD, a senior qualitative methodologist and CFIR intermediate user). Per regulations outlined in VHA Program Guide 1200.21, this evaluation has been designated a non-research quality improvement activity.

Methods for the traditional and rapid approaches

Data collection: semi-structured interviews.

Data collection methods were the same across both approaches; in effect, they will not be discussed in detail in this paper. In brief, we conducted semi-structured telephone interviews with DoE participants involved with implementing an EBI; for additional detail, see previous publications [ 31 , 35 ]. Interview guides were informed by the CFIR (see Additional File 1 ). Cohort A included 57 interviews across 17 facilities (1–4 interviews/facility) from June 2017 to September 2017; because one facility only had one interview, the need to aggregate data for that facility was eliminated. Cohort B included 72 interviews across 16 facilities (3–6 interviews/facility) from May 2019 to September 2019. Although cohort B included more interviews, the interviews were on average shorter (approximately 30 min), so both cohorts had approximately 50 audio hours total.

Data analysis: traditional and rapid approaches

The steps in our CFIR-based deductive traditional and deductive rapid qualitative analysis approaches are described in Table 1 . The traditional CFIR approach is described in detail on www.cfirguide.org and in several publications [ 31 , 36 , 37 , 38 ]. Our traditional CFIR approach is a form of directed content analysis [ 11 ] using transcripts and consisted of the following steps:

The analysts independently coded verbatim transcripts using Dedoose [ 39 ], a collaborative qualitative software program. The codebook included deductive CFIR constructs as well as inductive codes not captured in the CFIR that were relevant to the evaluation. Analysts used comments within coding software to flag sections of text for discussion or add additional notes.

The analysts met weekly to adjudicate differences in coding.

The primary analyst exported and aggregated coded data in MS Word CFIR facility memos (one for each facility). See Table 2 and Additional File 2 .

The primary analyst summarized and rated coded data and wrote high-level facility summaries in each facility memo. The secondary analyst reviewed the primary analyst’s drafts of the facility memos and edited the summaries, ratings, and high-level facility summaries. Ratings were based on two factors: (1) valence (positive or negative influence on implementation) and (2) strength (weak or strong influence on implementation). Analysts used comments and highlighting in the facility memo to flag sections of text for discussion. Completed facility memos ranged from 68 to 148 pages with an average of 108 pages.

The analysts met weekly to adjudicate differences and refine the codebook.

The primary analyst copied the summaries, ratings, and high-level facility summaries from each facility memo into the MS Excel CFIR construct by facility matrix for interpretation; the matrix included all codes from the codebook (both deductive and inductive codes) as well as a row for high-level facility summaries. See Table 3 and Additional File 3 .

In contrast, our rapid CFIR approach is a form of directed content analysis [ 11 ] using interview notes and verification with audio recordings, which consisted of the following steps:

The primary analyst took notes and captured quotations during interviews. Immediately after the interviews, the primary analyst “coded” the notes into the MS Excel CFIR construct by facility matrix and noted when additional detail or a timestamp was needed. The secondary analyst then reviewed the matrix, listened to the audio recordings, and edited and built upon the primary analyst’s notes. Analysts coded based on a codebook with deductive CFIR constructs as well as inductive codes not captured in the CFIR that were relevant to the evaluation. Analysts used comments and highlighting in the matrix to flag sections of text for discussion.

Analysts met weekly to adjudicate differences and refine the codebook.

The primary analyst reviewed notes, rated CFIR constructs, and wrote a high-level facility summary for each facility in the matrix; the secondary analyst reviewed the matrix and edited ratings and high-level facility summaries. Ratings were determined based on two factors: (1) valence (positive or negative influence on implementation) and (2) strength (weak or strong influence on implementation). See Table 3 and Additional File 3 .

Analysts met weekly to adjudicate differences.

Data interpretation: facility and construct analyses

Data interpretation methods were the same across both approaches and are discussed in detail on www.cfirguide.org . In brief, the analysts completed the following steps: (1) facility (case) analyses, to identify constructs that influenced implementation outcomes in each facility, and (2) construct analyses, to identify CFIR constructs that manifested positively or negatively across facilities or distinguished between facilities with high and low implementation success.

Methods for comparing traditional and rapid approaches

Comparing time and transcription costs.

The team tracked time for data management, data collection, data analysis, and data interpretation for both approaches using MS Excel spreadsheets. Staff time for these tasks is based on hours. We also combined both analyst’s funded effort to determine the total available analyst hours for our evaluation. Transcription costs were obtained from invoices from a centralized VHA qualitative interview transcription service.

Comparing effectiveness and rigor

The team did not plan to compare the effectiveness or rigor of our traditional versus rapid approach (see the “Limitations” section). As a result, we defined and assessed these aspects retrospectively. Effectiveness was measured by whether we met our evaluation objective in each approach. Rigor was measured primarily by assessing the credibility of each approach, i.e., if evaluation processes established confidence that the results were accurate [ 40 , 41 ].

Comparing traditional and rapid approaches

Time and transcription costs.

The traditional approach required more time than the rapid approach and included transcription costs. Cohort A, using the traditional deductive CFIR approach, required 683 total hours and $7250 in transcription costs. Cohort B, using the rapid deductive CFIR approach, required 409.5 total hours with no transcription costs. In effect, the rapid approach required 273.5 fewer total hours and saved $7250 in transcription costs. The evaluation funded two analysts with a combined total of 1305 h available for each year. Cohort A required 52.3% (683/1305 h) of the available hours while cohort B required 31.4% (409.5/1305 h) of the available hours, representing a significant reduction in time within the broader context of the evaluation. However, time savings during rapid analysis varied by phase, with the largest savings during the facility-level analysis. The following sections provide a summary of analyst hours and transcription costs for both approaches. See Table 1 , Table 4 , and Fig. 1 for additional description.

Comparison of analysis hours for the Traditional CFIR Approach (Cohort A) versus the Rapid CFIR Approach (Cohort B). This graph does not include data collection or data interpretation because both were equal across Cohort A and B

Data management

Data management in the traditional approach required 1 h to set-up the project and .5 h/interview plus .5 h/facility. In contrast, data management in the rapid approach required only .5 h to set-up the project with no other time needed. As shown in Table 1 , the rapid approach eliminated data management steps except for creating the MS Excel CFIR construct by facility template. As a result, the rapid approach reduced analys time by 33.5 h. Though not directly impacting analyst hours, transcripts were not received for 2–6 weeks following interviews, significantly delaying analysis for the traditional approach. See Table 1 , Table 4 , and Fig. 1 .

Data collection methods were the same across both approaches and the total number of audio hours was roughly equivalent between cohorts A and B; in effect, there were no significant differences in analyst hours between approaches. However, the rapid approach required blocking approximately 3 h for each interview: approximately 1 h for the interview plus 1–2 h to process the notes and “code” them into the CFIR construct by facility matrix immediately following the interview. The analyst’s immediate recall of the interview helped bolster the accuracy of the notes but intensified effort and cognitive load on interview days.

Data analysis

Data analysis in the traditional approach required 5.5 h/interview plus 14 h/facility versus 3.92 h/interview plus 3.92 h/facility in the rapid approach. In effect, the rapid approach reduced analys time by 79 h (275 versus 196 for traditional and rapid, respectively). The largest contributor to this reduction in analyst hours was in the facility-level analysis phase; where the rapid approach required 63 h, the traditional approach required 224 h. This difference was a result of how and when data were condensed and aggregated. In the traditional deductive CFIR approach, all coded data were aggregated in facility memos that were approximately 108 pages long; due to the relationships that often exist between constructs, the memos often included the same segments of text under multiple constructs. As a result, the same pieces of data were reviewed multiple times in full by each analyst independently before the data were condensed in the matrix. In contrast, the rapid deductive CFIR approach condensed data prior to aggregating by facility and was completed first by the primary analyst. Relationships between constructs were described once in the matrix, and notes in other cells referred back to this description, thus eliminating multiple references to the same data. The secondary analyst then built upon and confirmed the data in the matrix by listening to the audio recording. See Table 1 , Table 4 , and Fig. 1 .

Data interpretation

Data interpretation methods were the same across both approaches, which consisted of reviewing the CFIR construct by facility matrix. Both approaches took approximately 100 h for data interpretation. See Table 1 , Table 4 , and Fig. 1 .

Effectiveness and rigor

There were substantial differences in the number of hours and transcription costs between the traditional and rapid approaches; however, both approaches were systematic and there was concordance among many of the evaluation phases. Even when the analysis steps were different, both approaches followed the same general approach from data collection through data interpretation (see Table 1 ). Although data werecondensed earlier in the rapid approach than the traditional approach, i.e., following the interview versus following the facility memo, the depth of the data in the final matrices was similar for both approaches. For example, both matrices included brief direct quotes from participants. As a result, both approaches were effective in meeting our overall goal for the evaluation; we were able to identify and describe the factors influencing implementation in a high level of detail. However, the rapid approach also allowed us to share formal results more quickly with our operational partners (see Table 5 ).

In addition, both approaches included processes to enhance methodological rigor [ 40 , 41 ]. Credibility of results, a form of rigor, was most relevant when assessing tradeoffs between our rapid and traditional approaches [ 41 ]. We enhanced the credibility of results by having analysts with expertise in qualitative methods and the CFIR. To ensure participant responses were accurately captured in our summaries, we used two analysts per interview as a quality check and verified summaries with raw data (transcripts or audio recordings). Overall, the final summaries from both approaches were quite similar. See Table 5 for an additional description of the effectiveness and concordance of rigor between both approaches.

Our rapid deductive CFIR approach has much potential value, given the urgent need for nearly real-time results, to guide the implementation and dissemination of EBIs. The goal of this paper was to compare two qualitative approaches using deductively derived codes based on the CFIR: a traditional deductive CFIR approach using verbatim transcripts versus a rapid deductive CFIR approach using notes and audio recordings. Although we used the CFIR, this approach can be used with other frameworks. Our paper enhances the literature by describing exactly how rapid deductive CFIR analysis versus traditional deductive CFIR analysis leads to less resource use without compromising rigor.

Although our rapid deductive CFIR approach was beneficial for our evaluation team, researchers should review four considerations before using this method: (1) team expertise in CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

First, the team’s expertise in the CFIR and qualitative methods should be considered before deciding to employ a rapid approach. Prior literature suggests that traditional qualitative analysis requires more intense training than rapid analysis [ 23 , 28 ]. In-depth qualitative methods should indeed be conducted by a skilled research team. However, we argue that our rapid deductive CFIR approach may be more suited to researchers who already have a strong foundation in qualitative methods and the CFIR. Qualitative researchers familiar with the CFIR are more equipped to rapidly “code” qualitative data into CFIR constructs in real time than a novice. However, even for skilled researchers, we found that rapid analysis intensified effort and cognitive load during the initial coding phase, e.g., requiring a 3-h calendar block. Although a more experienced team may cost more in terms of salaries, the experienced team works more efficiently and likely saves money overall by reducing time spent training and overseeing project staff. For less experienced teams, we suggest linking CFIR constructs and brief definitions directly to interview questions within a notes template; this will help guide the researcher when summarizing the interview and/or listening to the audio recording. However, it is important to note that participant responses to questions will not always address the intended construct. Furthermore, while we identified a high level of fidelity between the primary analyst’s notes and the audio recordings, the secondary analyst may serve as an essential quality check for less experienced teams.

Second, researchers should consider what level of detail is needed for data analysis and the presentation of results in order to meet the project’s aims [ 28 ]. As articulated in prior research, rapid approaches using notes and audio recordings may provide a “big picture” view, yielding a lower level of detail than transcript-based approaches [ 28 ]. A project that requires a high level of detail and/or long quotations may therefore not be appropriate for our rapid approach. Our rapid CFIR approach provided less detail, but in so doing, may have allowed us to see both the overall patterns and the important details in our data more efficiently, i.e., seeing both the forest and the trees.

Third, the mode of data (transcripts or audio recordings) should be considered since it is not necessarily associated with a traditional or rapid approach. For example, audio recordings can be used for traditional analysis, i.e., many types of qualitative software allow minute-by-minute coding of audio recordings, and transcripts can be used for rapid analysis, i.e., summaries can be developed based on transcripts instead of audio recordings. For our rapid approach, we chose to use post-interview notes and audio recordings instead of transcripts to help streamline our deductive CFIR analysis process, i.e., it eliminated transcription costs and delays, and provided a point of comparison with other existing rapid approaches that use transcripts. However, if a team desires a more rapid approach while also maintaining access to the data in written form, including transcripts may be an option.

Fourth, there are advantages and disadvantages to consider when opting to use the CFIR (or another framework) regardless of the rapid or traditional qualitative analysis approach. Using the CFIR is helpful because it is a comprehensive determinant framework that includes constructs from 19 other models, including work by Greenhalgh et al. [ 42 ] that reviewed 500 published sources across 13 scientific disciplines. In effect, the CFIR helps researchers identify determinants that may be overlooked in a purely inductive approach. In addition, the use of the CFIR assists researchers with sharing and comparing results across studies, which advances implementation science. However, if researchers overly rely on the CFIR (or another framework), they may overlook constructs or miss important insights not included in the framework. To address this concern, we included questions in our interview guide beyond the scope of the CFIR, e.g., anticipated sustainment, and added codes, as needed, to capture inductively derived determinants and outcomes. Overall, even when using a more deductive approach, it is important for researchers to be open to inductive topics or domains that may arise in the data. Ultimately, researchers should consider their goals when deciding whether to adopt a deductive rapid approach (i.e., more confirmatory to compare with existing constructs or knowledge) versus an inductive approach (i.e., more exploratory to generate new constructs or knowledge).

It is important to note that our rapid deductive CFIR approach was still time intensive; it took 409.5 h to complete the analysis, including the rating process, for cohort B. However, because the analysts completed interview notes and coding in the matrix immediately after each interview, we were able to share preliminary results during regularly scheduled meetings with our operational partners on an ongoing basis. Regardless, some researchers may need additional ways to streamline our rapid CFIR analysis process. As long as a team considers both strengths and limitations, the following strategies may provide ways to streamline our rapid CFIR approach:

The team could eliminate the second analyst entirely or only use a second analyst on a subset of interviews, e.g., on the first 10 interviews or a random sample.

The team could include only the CFIR constructs expected to be most relevant to the research question in the matrix.

The team could seek to obtain project artifacts, e.g., meeting minutes, to analyze in the place of interviews.

The team could omit the rating process following coding.

Although rapid approaches are becoming more alluring to many implementation science researchers, they should not be considered a quick and easy replacement for traditional approaches or a substitute for having a skilled research team. Teams must carefully consider the best approach for their project while also exploring how to maintain scientific rigor. Qualitative expert oversight and/or training, analyst familiarity with the framework, review by a secondary analyst, and interview data quality are some important aspects of methodological rigor.

Limitations

Several limitations should be noted. First, both analysts on this project were intermediate to expert CFIR users. Our approach may be more difficult for new CFIR users, i.e., it may be difficult to translate interview notes into “coded” data in the matrix or to “code” while listening to an audio recording, unless the researchers are very familiar with the constructs. Second, the same analysts were involved in analyzing both cohorts. It is possible the analysts were more familiar with the broader findings from the study based on the traditional analysis of cohort A, which may have allowed them to progress more quickly in the rapid analysis of cohort B. However, using the same analysts improves comparability of coding between the two different cohorts of data and streamlined the process because additional analysts did not need to be trained in using the CFIR. Future research is needed to assess the extent and the conditions under which our approach works for other CFIR users. Third, we focused on differences in time and transcription costs rather than specifically testing the effectiveness or rigor of our rapid versus traditional approach, which has been discussed in prior literature [ 23 , 28 ]. While the rigor of the results was the same with both approaches, future researchers should likewise assess the rigor of this deductive rapid approach within their circumstances.

Conclusions

Our deductive rapid approach using the CFIR, involving notes and audio recordings, is an effective and rigorous approach for analyzing qualitative data that resulted in substantial reductions in time and transcription costs. We intend to use this approach for similar studies in the future. Overall, a deductive rapid approach using the CFIR (or another framework) is especially beneficial when (1) the research team has strong qualitative methods and skills using the framework, (2) the research timeline is relatively short or real-time feedback is needed, (3) the budget  does not support transcription, and (4) the research team wants to compare results across studies.

Availability of data and materials

The datasets generated and/or analyzed during the current evaluation are not available due to participant privacy but may be available from the corresponding author on reasonable request.

Abbreviations

Consolidated Framework for Implementation Research

Diffusion of Excellence

Evidence-Based Innovation

Veterans Health Administration

Hamilton AB, Finley EP. Qualitative methods in implementation research: an introduction. Psychiatry Res. 2019;280:112516. https://doi.org/10.1016/j.psychres.2019.112516 .

Article   PubMed   PubMed Central   Google Scholar  

Cohen D, Crabtree BF, Damschroder L, et al. Qualitative methods in implementation science. National cancer iInstitute. Division of cancer control and population sciences. https://cancercontrol.cancer.gov/sites/default/files/2020-09/nci-dccps-implementationsciencewhite-paper.pdf . Accessed 14 Apr 2021.

Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed method designs in implementation research. Adm Policy Ment Health Ment Health Serv Res. 2011;38(1):44–53. https://doi.org/10.1007/s10488-010-0314-z .

Article   Google Scholar  

Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. 2012;5(1):48–55. https://doi.org/10.1111/j.1752-8062.2011.00383.x .

Ash J, Sittig D, McMullen C, Guappone K, Dykstra R, Carpenter J. A rapid assessment process for clinical informatics interventions. AMIA Ann Symp Proc. 2008;2008:26–30.

Google Scholar  

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20. https://doi.org/10.1258/jrsm.2011.110180 .

Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clin Transl Med. 2013;2(1). https://doi.org/10.1186/2001-1326-2-10 .

Vindrola-Padros C, Vindrola-Padros B. Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery. BMJ Qual Saf. 2018;27(4):321–30. https://doi.org/10.1136/bmjqs-2017-007226 .

Article   PubMed   Google Scholar  

Palinkas LA, Zatzick D. Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: methods and applied case study. Adm Policy Ment Health Ment Health Serv Res. 2019;46(2):255–70. https://doi.org/10.1007/s10488-018-0909-3 .

Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40(1):423–42. https://doi.org/10.1146/annurev-publhealth-040218-044215 .

Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. https://doi.org/10.1177/1049732305276687 .

Saldana J. The coding manual for qualitative researchers. 2nd ed: Sage Publications; 2015.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa .

Kenny M, Fourie R. Contrasting classic, Straussian, and constructivist grounded theory: methodological and philosophical conflicts. Qual Rep. Published online August 17, 2015. https://doi.org/10.46743/2160-3715/2015.2251 .

Bernard HR, Wutich A, Ryan GW. Analyzing qualitative data: systematic approaches: SAGE; 2017.

Hodges BD, Kuper A, Reeves S. Discourse analysis. BMJ. 2008;337(aug07 3):a879. https://doi.org/10.1136/bmj.a879 .

Sobo EJ, Simmes DR, Landsverk JA, Kurtin PS. Rapid assessment with qualitative telephone interviews: lessons from an evaluation of California’s healthy families program & Medi-Cal for children. Am J Eval. 2003;24(3):399–408. https://doi.org/10.1177/109821400302400308 .

Hamilton AB. Qualitative methods in rapid turn-around health services research. PowerPoint Present 2013 VA HSRD Cyberseminar Spotlight Womens Health. Published online 2013. Accessed November 16, 2020. www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780 .

Vindrola-Padros C, Johnson GA. Rapid techniques in qualitative research: a critical review of the literature. Qual Health Res. 2020;30(10):1596–604. https://doi.org/10.1177/1049732320921835 .

Taylor B, Henshall C, Kenyon S, Litchfield I, Greenfield S. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open. 2018;8(10):e019993. https://doi.org/10.1136/bmjopen-2017-019993 .

Koenig CJ, Abraham T, Zamora KA, Hill C, Kelly PA, Uddo M, et al. Pre-implementation strategies to adapt and implement a Veteran Peer Coaching intervention to improve mental health treatment engagement among rural veterans: pre-implementation of a mental health intervention. J Rural Health. 2016;32(4):418–28. https://doi.org/10.1111/jrh.12201 .

Nevedal AL, Ayalon L, Briller SH. A qualitative evidence synthesis review of longitudinal qualitative research in gerontology. Heyn PC, ed. Gerontologist. 2019;59(6):e791–801. https://doi.org/10.1093/geront/gny134 .

Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci. 2019;14(1):11. https://doi.org/10.1186/s13012-019-0853-y .

The Consolidated Framework for Implementation Research (CFIR) technical assistance website. Accessed May 1, 2020. https://cfirguide.org/

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50. https://doi.org/10.1186/1748-5908-4-50 .

Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8(1):51. https://doi.org/10.1186/1748-5908-8-51 .

Holdsworth LM, Safaeinili N, Winget M, Lorenz KA, Lough M, Asch S, et al. Adapting rapid assessment procedures for implementation research using a team-based approach to analysis: a case example of patient quality and safety interventions in the ICU. Implement Sci. 2020;15(1):12. https://doi.org/10.1186/s13012-020-0972-5 .

Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: a procedure for the rapid identification of themes from audio recordings (RITA). Am J Eval. 2015;36(1):118–32. https://doi.org/10.1177/1098214014536601 .

Kilbourne AM, Goodrich DE, Miake-Lye I, Braganza M, Bowersox NW. Quality enhancement research initiative implementation (QUERI) roadmap: towards sustainability of evidence-based practices in a learning health system. Med Care. 2019;57(Suppl 3):S286–93. https://doi.org/10.1097/MLR.0000000000001144 .

Rycroft-Malone J, Harvey G, Kitson A, McCormack B, Seers K, Titchen A. Getting evidence into practice: ingredients for change. Nurs Stand. 2002;16(37):38–43. https://doi.org/10.7748/ns2002.05.16.37.38.c3201 .

Nevedal AL, Reardon CM, Jackson GL, Cutrona SL, White B, Gifford AL, et al. Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study. Implement Sci Commun. 2020;1(1):61. https://doi.org/10.1186/s43058-020-00053-1 .

Clancy CM. Creating world-class care and service for our nation’s finest: how Veterans Health Administration diffusion of excellence initiative is innovating and transforming Veterans Affairs health care. Perm J. 2019;23:18.309. https://doi.org/10.7812/TPP/18.301 .

Vega RJ, Jackson GL, Henderson B, et al. Diffusion of excellence: accelerating the spread of clinical innovation and best practices across the nation’s largest health system. Perm J. 2019;23:18.309.

Jackson GL, Cutrona SL, White B, et al. Implementation practice and science to scale-up innovative practice: VHA diffusion of excellence. Jt Comm J Qual Patient Saf. 2021;47(4):217–27.

Jackson G, Cutrona S, White B, Reardon C, Orvek E, Nevedal A, et al. Identifying, replicating, and spreading health care innovations across a nation-wide health care system: VHA diffusion of excellence. Health Serv Res. 2020;55(S1):60. https://doi.org/10.1111/1475-6773.13411 .

Damschroder LJ, Reardon CM, Sperber N, Robinson CH, Fickel JJ, Oddone EZ. Implementation evaluation of the Telephone Lifestyle Coaching (TLC) program: organizational factors associated with successful implementation. Transl Behav Med. 2017;7(2):233–41. https://doi.org/10.1007/s13142-016-0424-6 .

Damschroder LJ, Goodrich DE, Robinson CH, Fletcher CE, Lowery JC. A systematic exploration of differences in contextual factors related to implementing the MOVE! weight management program in VA: a mixed methods study. BMC Health Serv Res. 2011;11(1):248. https://doi.org/10.1186/1472-6963-11-248 .

Damschroder LJ, Reardon CM, AuYoung M, Moin T, Datta SK, Sparks JB, et al. Implementation findings from a hybrid III implementation-effectiveness trial of the Diabetes Prevention Program (DPP) in the Veterans Health Administration (VHA). Implement Sci. 2017;12(1):94. https://doi.org/10.1186/s13012-017-0619-3 .

Dedoose Version 8.0.35, Web application for managing, analyzing, and presenting qualitative and mixed method research data. SocioCultural Research Consultants, LLC; 2018. www.dedoose.com . Accessed 13 Apr 2021.

Maher C, Hadfield M, Hutchings M, de Eyto A. Ensuring rigor in qualitative data analysis: a design research approach to coding combining NVivo with traditional material methods. Int J Qual Methods. 2018;17(1):160940691878636. https://doi.org/10.1177/1609406918786362 .

Forero R, Nahidi S, De Costa J, et al. Application of four-dimension criteria to assess rigour of qualitative research in emergency medicine. BMC Health Serv Res. 2018;18(1):120. https://doi.org/10.1186/s12913-018-2915-2 .

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x .

Download references

Acknowledgements

The opinions expressed in this article are those of the authors and do not represent the views of VHA or the US Government. We are grateful to the VHA employees who participated in this evaluation and shared their experiences with us.

This evaluation was funded by the Veterans Health Administration (VHA) Quality Enhancement Research Initiative (QUERI) [PEC-17-002] with additional funding subsequently provided by the VHA Office of Rural Health through the Diffusion of Excellence.

Author information

Andrea L. Nevedal and Caitlin M. Reardon contributed equally to this work.

Authors and Affiliations

Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System (152-MPD), 795 Willow Road, Building 324, Menlo Park, CA, 94025, USA

Andrea L. Nevedal

Veterans Affairs (VA) Center for Clinical Management Research, Ann Arbor Healthcare System, 2215 Fuller Rd. (152), Ann Arbor, MI, 48105, USA

Caitlin M. Reardon, Marilla A. Opra Widerquist & Laura J. Damschroder

Center of Innovation to Accelerate Discovery and Practice Transformation (ADAPT), Durham VA Health Care System, Durham, USA

George L. Jackson & Brandolyn S. White

Department of Population Health Science, Duke University, Durham, USA

George L. Jackson

Division of General Internal Medicine, Duke University, Durham, USA

Department of Family Medicine and Community Health, Duke University, Durham, USA

Center for Healthcare Organization & Implementation Research, Bedford & Boston VA Medical Centers, Boston, USA

Sarah L. Cutrona

Department of Population and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, USA

Division of General Internal Medicine, University of Massachusetts Medical School, Worcester, USA

You can also search for this author in PubMed   Google Scholar

Contributions

LD, GJ, SC, CR, BW, AN, and MOW were engaged in the national evaluation of the Diffusion of Excellence. GJ, LD, and SC designed and supervised the overall evaluation. AN, CR, LD, and MOW led the data collection, analysis, and manuscript writing for this aspect of the evaluation. AN, CR, LD, MOW, GJ, SC, and BW were involved in the critical revision of the manuscript for intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Andrea L. Nevedal .

Ethics declarations

Ethics approval and consent to participate.

Per regulations outlined in VHA Program Guide 1200.21, this evaluation has been designated a non-research quality improvement activity.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Interview Guide.

Additional file 2.

Unabridged MS Word CFIR Facility Memo Template.

Additional file 3.

Unabridged MS Excel CFIR Construct by Facility Matrix Template.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nevedal, A.L., Reardon, C.M., Opra Widerquist, M.A. et al. Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR). Implementation Sci 16 , 67 (2021). https://doi.org/10.1186/s13012-021-01111-5

Download citation

Received : 26 December 2020

Accepted : 05 April 2021

Published : 02 July 2021

DOI : https://doi.org/10.1186/s13012-021-01111-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Consolidated Framework for Implementation Research (CFIR)
• Qualitative methods
• Rapid analysis
• Implementation science

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 8, Issue 10
• Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Beck Taylor ,
• http://orcid.org/0000-0001-5659-3296 Catherine Henshall ,
• Sara Kenyon ,
• http://orcid.org/0000-0002-1169-5392 Ian Litchfield ,
• Sheila Greenfield
• Institute of Applied Health Research, College of Medical and Dental Sciences , University of Birmingham , Birmingham , UK
• Correspondence to Dr Ian Litchfield; i.litchfield{at}bham.ac.uk

Objectives This study compares rapid and traditional analyses of a UK health service evaluation dataset to explore differences in researcher time and consistency of outputs.

Design Mixed methods study, quantitatively and qualitatively comparing qualitative methods.

Setting Data from a home birth service evaluation study in a hospital in the English National Health Service, which took place between October and December 2014. Two research teams independently analysed focus group and interview transcript data: one team used a thematic analysis approach using the framework method, and the second used rapid analysis.

Participants Home birth midwives (6), midwifery support workers (4), commissioners (4), managers (6), and community midwives (12) and a patient representative (1) participated in the original study.

Primary outcome measures Time taken to complete analysis in person hours; analysis findings and recommendations matched, partially matched or not matched across the two teams.

Results Rapid analysis data management took less time than thematic analysis (43 hours vs 116.5 hours). Rapid analysis took 100 hours, and thematic analysis took 126.5 hours in total, with interpretation and write up taking much longer in the rapid analysis (52 hours vs 8 hours). Rapid analysis findings overlapped with 79% of thematic analysis findings, and thematic analysis overlapped with 63% of the rapid analysis findings. Rapid analysis recommendations overlapped with 55% of those from the thematic analysis, and thematic analysis overlapped with 59% of the rapid analysis recommendations.

Conclusions Rapid analysis delivered a modest time saving. Excessive time to interpret data in rapid analysis in this study may be due to differences between research teams. There was overlap in outputs between approaches, more in findings than recommendations. Rapid analysis may have the potential to deliver valid, timely findings while taking less time. We recommend further comparisons using additional data sets with more similar research teams.

• qualitative research
• health services administration & management
• maternal medicine

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjopen-2017-019993

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

Our study explores a strategy to address the time-lag in reporting qualitative findings to clinicians and policymakers, which slows translation of research into practice.

This is the first comparison of qualitative analytical methods in applied health research which compares both researcher time and outputs, with a complete study dataset.

The work describes the process of comparing time and analytical outputs in detail, to inform others planning further methodological comparisons.

Due to the time lag in thematic analysis outputs, our study did not triangulate findings with the original participants.

The study uncovered important challenges in comparing analytical approaches between research teams which can inform the design future work in this area.

Background 

Applied health research frequently adopts mixed methods, often using qualitative approaches. 1 Applications of qualitative methods include: early work to identify areas for focus; throughout a study to explore processes and user experience; and following a trial or intervention implementation to explain outcomes and/or identify stakeholder experiences, to explore in more depth questions or issues identified through quantitative work and to problematise or ‘unpack’ issues or topics taken for granted. 2 Increasingly this type of research can include a broader range of contributors, for example, where members of the public, patients, clinicians and researchers are involved in analysing and interpreting data to ensure a multidisciplinary perspective or pragmatically using several researchers to code data in the interests of time. 3 4

Typically stakeholders want rapid results, 5–7 yet compared with quantitative approaches, traditional qualitative approaches often considerable time is required to manage and interpret data and deliver findings. 8 9 In a service context, delays may render the findings out of date, reducing their applicability and relevance. There are examples of apparently more rapid alternatives to traditional qualitative approaches, including specific end-to-end approaches such as Rapid Assessment Process and Rapid Ethnography. 6 9–13 There are four broad areas where time can be saved: by reducing data collection time, for example, by allowing less time between data collection episodes 6 ; by reducing data management time, for example, by relying on untranscribed audio recordings, notes, summaries and mind maps 10–12 ; by minimising the time spent analysing data by summarising as opposed to formally coding 11 13 ; and by limiting the time spent on analysis by using a ‘one sheet of paper’ summary to explore a sample of a large precoded dataset. 9 Often rapid methods describe a broad approach, including activities from entering the field through to delivery of findings and/or involve mixed methods. 6 7 This paper specifically explores whether rapid analysis (RA) of qualitative data (distinct from end-to-end rapid methods) delivers equivalent findings to traditional approaches and how much time may be saved in practice.

There are a limited number of studies that have compared different qualitative analytical techniques. 11 14–16 In some of the empirical examples identified, methodologists have predominantly compared methods of data collection (eg, interviews vs internet forums 14 ) and focused on the number and content of codes rather than interpretation. Of the three examples identified that compare analytical approaches, one used focus group data to compare thematic analysis (TA) of a partial dataset with mind-mapping of a full dataset. 11 While this paper provides minimal detail regarding the method of comparison, it reported differences in time taken to analyse the data and in the number and presentation of codes. The second example compared software-assisted and constant comparative approaches to analysis describing differences in the frequency of codes and coding levels. 15 The third example compared analysis of focus group data directly from audio recordings, with TA of transcribed data, and found that themes generated were comparable. 16

The work we present here was conducted as part of the Collaboration for Leadership in Applied Health Research and Care (CLAHRC) programme in the West Midlands of England. CLAHRC involves local teams across universities and National Health Service organisations working in partnership to deliver research to improve services for patient benefit. 17 As part of a service evaluation study of a new home birth service, we gathered interview, focus group and documentary data. We then compared the speed and outputs of rapid and traditional techniques applied to the same dataset. For the RA, we used the approach developed by Hamilton at UCLA. 13 We compared this with TA 18 and the framework method, which was selected due to the team’s existing familiarity with this approach and the fact that it is increasingly applied in multidisciplinary health services research. 8 19

This study compares rapid and traditional analyses of a UK health service evaluation dataset to explore differences in researcher time and consistency of outputs. This was a mixed methods study, quantitatively and qualitatively comparing the outputs of qualitative methods.

The data came from a home birth service evaluation study in a hospital in the English National Health Service, which took place between October and December 2014. This was a service innovation put into place by the hospital. A dedicated team of midwives was set up to provide antenatal, birth and postnatal care to women choosing to have a home birth, with the aim of providing a more reliable service and increasing the local home birth rate.

Characteristics of participants

Home birth midwives (6), midwifery support workers (4), commissioners (4), managers (6) and community midwives (12) and a patient representative (1) participated in the original study.

Description of processes, interventions and comparisons

In the original evaluation, an evaluability assessment approach was adopted, 20 and its specific objectives were to: establish the original programme design and how the service differed from this design and why; identify facilitators or barriers to implementation; establish what service data are available and how it is being/could be gathered; and identify how staff would develop/improve the service. The evaluation was a qualitative study, involving interviews and focus groups with key participants involved in the home birth service.

Twenty-three provider and commissioning staff and one patient representative were purposively sampled, with recruitment by direct email or telephone invite, with three unable to take part due to availability. Twenty-one semistructured interviews informed by the study objectives were conducted by BT at participants’ workplaces. A single focus group of 12 midwives was facilitated by BT and CH also structured according to the study objectives. A convenience sampling approach was taken for the focus group, with midwives available at the allotted time invited to take part at their workplace. Participants were not known to researchers prior to the study. Interviews and the focus group lasted approximately 1 hour, were digitally recorded and transcribed for analysis, with minimal field notes taken. Participants did not review transcripts. Eight key service documents were also used in the analysis (business case, service guidelines and commissioning policy). Local approval was obtained from the hospital research and development team. The data were analysed independently using first RA and second TA as described in detail below. All researchers work in applied health research in the same department of a UK university. BT is a public health physician, CH is a registered nurse and SK is a registered midwife. Researchers 4 and 5 are health service researchers, and researcher 6 is a medical sociologist. A summary and comparison of the process used for the two analyses is shown in table 1 . The work was undertaken using a theoretically interpretive, generic qualitative approach across both teams.

• View inline

Description of the rapid analysis and thematic analysis

Supplementary file 1

The primary ra.

RA was conducted between November and December 2014; this constituted the primary empirical work that was subsequently reported to the service. The rapid qualitative analysis approach used 13 was designed to deliver timely findings with methodological rigour. The approach includes guidance on data collection and report writing and was developed using teams of less experienced researchers. Here we have used only the analytical methodology and researchers experienced in qualitative methods. Hamilton relates how the reduced timeframe of rapid methods means that they tend to be more deductive and explanatory than inductive and exploratory. 13 It can be hypothesised that this may negatively impact on the ability of rapid methods to discover more ‘hidden’ phenomena that one associates with traditional qualitative methods, and this must be balanced with the speed at which rapid methods can deliver findings. In recognition of this, the work presented here incorporated both inductive and deductive approaches, using a deductive template to structure analysis, with explicit remit to highlight other issues that emerged inductively from the data, though the focus was on inductive analysis. The process is presented in detail in table 1 . Researchers spent approximately 1 hour with each transcript or document, as stipulated by Hamilton in her description of the approach, noting key issues in a one-sheet structured ‘summary template’, with no formal coding. The data entered into the summary templates focused on the main issues in the data, rather than every single issue that surfaced. The RA summary template was made up of a number of sections describing participant and data collection details and deductive and inductive headings. At the end of the template, there were further sections to record key documents, observations, quotations and reflections relating to the data collection episode. The deductive aspects of the initial summary template were developed from the research questions: rationale for implementing the home birth service, programme design (structured according to logic model domains), facilitators and barriers to implementation and routinely gathered data about the service. This template was tested by both RA researchers as described in table 1 . During this early testing process, it was deemed necessary to inductively develop a small number of additional subheadings for three of the template sections (rationale, barriers and facilitators) to help the researchers to organise the data. Although the use of more focused approaches has been highlighted to be of value when interpreting data for reporting in a health service context, the need to maintain a thorough and transparent process must go hand in hand with producing findings that are easily understood and relevant to stakeholders. 11 The summary template accompanies this paper (online supplementary file 1 ). Summarised data were explored with respect to the research objectives to produce a report summarising findings and recommendations.

Secondary TA using the framework method

The secondary analysis was conducted by IL between June and September 2015, after the original RA was complete, with oversight and support from [researchers 5 and 6]; all three are experienced qualitative applied health researchers from outside of the original team. Typically, the purpose of secondary analysis is to explore new research questions, 21 but in this case, secondary analysis was performed using a different method to meet the same objectives as the primary analysis to compare the outputs of the two methods. The original team ([researchers 1, 2 and 3]) provided brief contextual details about the field, the organisations and participants involved and the background to the project. No further discussion occurred to avoid revealing RA findings to the TA team. The TA was informed by the original research objectives, using an inductive approach, and following the steps set out in the framework method, an approach to TA developed by Ritchie and Lewis 8 19 : familiarisation, coding, developing a framework, applying the framework, charting data into the framework, interpreting data and writing up. Table 1 summarises the process

Notes on methods used

It is important to acknowledge that the creative and flexible nature of qualitative methods means that there is variation in the way different researchers undertake even established methods. While we refer to the methods with proper nouns, and summarise as ‘TA’ and ‘RA’ to provide clarity for the reader, it should not be assumed that these methods are ‘fixed’. In addition, while we refer to the framework method analysis as ‘TA’, we acknowledge that the framework method is one of many approaches that fall within TA. 8 We provide a full description of our approach for transparency. It should also be noted that while both methods use matrices, the approaches are quite different in that TA involves the detailed, inductive coding of data, producing a detailed coding framework and more complex matrix that accounts more completely for the dataset. RA focuses on major issues identified in the data, no full coding occurs, and matrices are deductively constructed.

The comparison

The comparative analysis was conducted between October 2015 and May 2016, comparing three aspects of the analyses: time taken, findings and recommendations. Each team recorded the time taken to perform every activity. Analytical activities were divided into two broad areas: ‘data review and management’ and ‘data interpretation and report writing’ as indicated in table 1 . Summary statistics were produced using data from the resulting time sheets. Findings were defined as individual issues identified and included in a report. Recommendations were defined as suggested actions to improve or maintain the service. Each team then independently compared RA and TA findings, allocating a ‘match’, ‘partial match’ or ‘mismatch’ category. Both teams then met to discuss and reach consensus. Any mismatches were discussed, and perceived reasons were agreed and recorded and summary statistics was produced.

Patient and public involvement (PPI)

This paper is a methodological exploration of two different means of qualitative analysis. There was no PPI involvement in establishing the criteria for comparison nor in facilitating the work. However, PPI was intrinsic to the original programme from which the data were gleaned. 17

The research teams

Table 2 presents the characteristics of the two research teams.

Characteristics of the two research teams

Comparison of time

Table 3 illustrates the time taken at each stage of the process, for the ‘management’ and ‘interpretation and report writing’ stages defined earlier in table 1 . The 4 hours of background discussions to provide IL with context were not counted in the total. The RA data review and management took around a third of the time of the TA (43 hours and 116.5 hours, respectively). The reverse was true of the report writing; RA was more than six times longer at 52 hours.

Time taken to complete analysis using rapid analysis and thematic analysis

Comparison of findings

The comparison of findings is presented in table 4 . TA elicited marginally more findings than RA (153 vs 131). There were 107 matches. There are differences in reporting style and level of detail in the matched findings, with the example below highlighting how each team provided similar findings but with a varied degree of specific information. Both teams had examples where they provided more or less detail than the other on a specific topic, but the reporting style in the RA was consistently more ‘polished’, with findings more consistently framed in a way that would be more accessible to the intended audience (explored further in the discussion).

Quantitative comparison of findings and recommendations elicited using rapid analysis and thematic analysis

There are issues around communication with ambulances/paramedics. TA finding

Some paramedics are unaware that the HBS exists and there have been delays of up to 30 min between the paramedics being informed of a BBA and this being cascaded down to midwives. RA finding

Findings from one method frequently matched two or more from the other: 71 RA and 78 TA findings delivered 107 matches. There were 43 partial matches, where findings identified similar, but not identical issues (28 RA, 37 TA, some matching more than once), for example:

There was a general consensus that useful meetings with a range of stakeholders were hard to arrange for a number of reasons including workload and shift pattern. TA finding

While support is strong in-principle, there is no formal process for strategic-level consultation and decision-making about the HBT within the provider Trust (outside of the Project Board). In addition, busy workloads make collaborative working challenging. RA finding

Eighty findings could not be matched: 46 or 37% of all RA findings and 34 (21%) of the TA findings. Exploration (see table 5 ) revealed that the most common reason for mismatches was that the other team simply did not interpret the relevant finding from the data. The TA team did not find 11%, and the RA team did not find 12% of the opposite team’s findings. The next most common reason was that findings were specific or detailed, rather than key issues with broad relevance. The RA team also reported 15 positive findings (successes and achievements), which the TA team did not include in a report to the Service: the TA team reflected that they focused on constructive feedback about challenges and areas requiring improvement, rather than positive findings (explored further in the discussion). For example, the RA team reported ‘The HBT MWs are generally supportive of the need for data collection and comply with this’, and ‘The Service has produced its first comprehensive data report for the Project Board (November 2014)’.

Suggested reason for mismatched findings and recommendations with examples

There were a small number of findings that emerged from interpretation of ‘what was not in the data’. For example, the RA team reported that staff may not gain necessary qualifications for deployment, which was a risk to service resilience, connecting data on staff training with other data concerning service staffing requirements, rather than a direct report from research participants. The TA team did not identify this finding. The RA team’s contextual knowledge meant that they perceived some TA findings to be incorrect. For example, a TA finding suggesting that regular meetings were helpful was rejected, as the RA team had been informed (outside of the formal data collection) that the meetings were not functioning as intended.

Finally, the RA team unconsciously suppressed two findings that were politically challenging: they agreed with these two findings from the TA team, which concerned relationships and performance of individuals connected to the Service (exact examples cannot be provided as they are of a sensitive nature). The RA team reflected that while they were aware of these issues, and also knew that the Service was aware of them, they did not write them up as findings in the report. This was not an actively documented, discussed decision-making process between the RA researchers; it was more implicit that they could not ‘go there’ in a report.

Some findings appeared to have no match, but cross-checking revealed that the finding aligned with the other team’s recommendations (nine RA and three TA findings). For example, the RA found that staff had requested more emergency training, and the TA recommendations included provision of more emergency training.

In terms of topics, the mismatched findings covered a range of different issues for the service.

Both teams identified findings missed by the other team, which covered operational issues and leadership and management issues for the Service. The RA team identified findings that were not elicited by the TA team relating to strategic issues, promotion of the service and performance management (which were often positive findings about ‘successes’ not reported by the TA team).

Comparison of recommendations

Quantitative comparison of recommendations is presented in table 4 . The RA generated 64 recommendations, a third less than the TA. Eighteen of the RA recommendations matched to 32 of those from the TA, and the individual RA recommendations tended to bring together multiple issues and were ‘crafted’ in such a way as to provide a smaller, number of recommendations combining multiple points. For example, the RA recommendation below encompassed three separate TA recommendations:

Working model: urgently consult regarding whether the model (shift pattern/on call volume/accrued time) is fit for purpose, and if it is, how MWs can be supported to avoid burnout. In addition, consider whether the Service can realistically attend BBAs within this model, and if not how this key objective for the Service can be achieved. RA recommendation

Collect more precise data on which BBAs did or didn’t need to attend. Then look at feasibility of HBS attending these women in the home. TA recommendation 1

Determine the capacity of current staffing levels and shift patterns. TA recommendation 2

Begin discussions with staff on preferences and flexibility in order to meet growing demand. TA recommendation 3

Some recommendations were more directly matched, for example:

Require future recruits to have achieved the minimum numeracy/literacy standard. TA recommendation

Be clear on the necessary baseline skills in numeracy and literacy that are required. RA recommendation

There were partial matches between 20 RA and 26 TA recommendations, for example.

Ensure robust lines of communication are in place between Home Birth Service and community midwives. TA recommendation

Routinely feed back to referring professionals to confirm booking with Home Birth Service, or transfer back to community midwives. RA recommendation

A further 26 (41%) of the RA recommendations and 42 (43%) of the TA recommendations had no match. Reasons are presented with examples in table 5 .

The most common reason was that the other team did not identify a particular recommendation, RA did not find 18 (35%) and TA did not find 3 (12%). Four of these TA recommendations related to training of midwives, three were about organisation of meetings and the remainder had no common theme. The researchers determined that the midwife training recommendations were important and had been an analytical blind spot for the RA team. Other mismatched recommendations were collectively determined to be of low importance by the researchers, except for the TA team’s recommendation about projected milestones for the service.

The RA team made 19 recommendations based on ‘what wasn’t in the data’, interpreting beyond the reported facts. The TA team made 15 recommendations, which the RA team did not support, as their contextual knowledge deemed them unworkable or inappropriate. Nine recommendations that were not found in the RA recommendations were from the TA team who made a detailed list of items for a future service dataset, while the RA team provided less specific recommendations regarding a future data set. Finally, four recommendations were determined to be made due to contextual knowledge of the RA researchers.

Principal findings

This study compared RA and TA methods applied to the same dataset to explore whether RA provides timely, accurate outputs for services. RA data management took around a third of the time of TA, but RA interpretation and write up took more than six times longer than TA. There was considerable overlap in the findings and recommendations between the two methods, with RA identifying marginally more findings than TA, and TA making marginally more recommendations than the RA. The comparison identified qualitative differences in the depth and detail of findings and recommendations in the two teams.

Strengths and limitations of the study

Strengths and limitations in the ra and ta processes.

The qualitative analysis processes followed by each team have been described in detail to enhance reproducibility and reliability. However, we acknowledge that work of this nature can never be reproducible due to the subjectivity of qualitative researchers and processes, 22 and the fact that research is a situated practice, where some aspects of the activity are beyond the control of the researcher. 23 In qualitative research, there is much debate regarding subjectivity, reflexivity and bias. 22 24 In the conduct of our work, we attempted to minimise ‘bias’ and described our methods in detail, though we have also retrospectively identified opportunities where others can mitigate this further in future work. The findings of research such as ours, which does reflect on and compare processes and findings in a systematic and detailed manner, can contribute to understanding the challenges faced by researchers. 25 The characteristics of the researchers are acknowledged and explored. Researchers were similar in that they were all experienced postdoctoral health services researchers, working in the same Institute for some time, arguably with similar cultures, though we acknowledge that the human, interpretive nature of qualitative research means that standardisation or researchers within and between the teams is not possible. There were differences between the researchers (see table 2 ). These factors may have conferred variation in analysis and interpretation.

The RA team had greater contextual knowledge resulting from previous clinical exposure as health professionals and working closely with the service. This appeared to impart an underlying level of understanding that was critical to the findings and particularly recommendations. It is useful to think about the concept of research as a situated practice in the context of our work. This may be particularly relevant for researchers who are ‘embedded’ in some way within the service being researched. While such embeddedness can help to provide useful insights into the meaning and relevance of research findings, it is important to be aware that this may unconsciously influence data interpretation. 23 RA in a health service setting without this background knowledge may be inappropriate. Around a third of RA findings were not accounted for by the TA: RA generated a large number of additional findings, suggesting that closeness to the field and data may have conferred an advantage. It has been recommended previously that contextual information should be provided to secondary analysts to mitigate the lack of exposure to the field. 21 The intended comparison of methods and need to avoid conferring between teams meant that the TA only received brief information, rather than the rich, iterative contextual information that may be more typically provided within secondary analysis.

The RA was conducted for a specific group of stakeholders, and the interpretation, and crafting of findings and recommendations, was done with these individuals in mind. Though not conscious of this at the time of analysis, on reflection, we believe that this focus on a specific audience, in addition to [researchers 1 and 2]’s relationship and sense of reciprocity with the service, may have resulted in a more lengthy approach. We reflected that it also resulted in more focus on reporting positive findings, or ‘good news’ in the RA team, and suppressing negative findings that concerned individuals, which the RA researchers deemed inappropriate to report in an evaluation output that would be widely shared. This contrasts with the TA that was a ‘desktop exercise’, with no commitment to the research participants, which we feel made the process more straightforward, with less need for careful presentation of data. This provides a clear example of researchers navigating the ‘politics of research’, telling stories differently as a result of the different purpose and context of the research. 26

A second factor in explaining the lengthy RA is that it is the first time that [researchers 1 and 2] have used RA. Adapting to a new method can take time, and discipline is required not to refer to more familiar, lengthier practices. However, the number and detail in the findings and recommendations in the RA (131 and 62, respectively) was similar to those in the TA (153 and 93). For qualitative researchers trained in TA, it may be difficult to wholly adopt the brevity required of RA.

The TA was predominantly conducted by one researcher IL, providing fewer opportunities for reflection in the TA development. The RA team also had the opportunity for ongoing regular reflection due to shared office space, which may have enhanced but also lengthened the process.

Our approach to this work was pragmatic, based on available researcher capacity, and there was variation in researcher characteristics, in their programmes of existing work and embeddedness in the field for this study, which may have impacted on the outputs from the work. In future comparisons, involving some or all of both teams in data collection would provide equality in exposure and embeddedness, and increasing similarity in researcher characteristics could provide further parity. The workload and capacity issues are more problematic. The time taken to undertake analysis varies from project to project, based on the available time, deadlines, funding and competing priorities. Generally, there is always scope for extended analysis of data to explore it further, and researchers must make pragmatic decisions about when analysis for a specific project is ‘finished’. It is likely that there is variation between decisions to cease analysis between research teams, particularly in our comparison, where the analysis was a ‘desk top exercise’ for the TA team and a ‘real’ project with stakeholders expecting outputs from the RA team, meaning the latter may be more inclined to spend longer on the project. To mitigate this, increased parity across the RA and TA researchers could be achieved by using two equal-sized teams, with equal division of labour, and explicit allocation of capacity to the project. However, it is still impossible to standardise decisions regarding what constitutes ‘enough’ work on a dataset.

Strengths and limitations in the comparison process

This paper has provided an opportunity to explore and reflect on approaches to comparing qualitative methods. The limited evidence base necessitated the development of the comparison methodology. The study team regularly met to review the process, emerging findings and interpretation to enhance the rigour of the exercise. A mixed methods approach was undertaken in order to explore RA, which allows for a broader exploration of a phenomenon (the analytical process) than quantitative or qualitative methods alone. 27–29 However, the qualitative aspect was restricted to evaluation of the alignment content outputs of the research and description of the researcher characteristics and activity diaries by the researchers themselves. Future comparisons of methods could be strengthened with the addition of independent qualitative evaluation of the research processes and outputs. A limitation of the quantitative approach to comparing outputs from qualitative work is that it reduces findings and recommendations, directly comparing individual outputs that display different levels of depth and detail. It is important to highlight that ‘more’ does not necessarily equal ‘better’ in qualitative research outputs.

An important consideration when undertaking comparison of methods is the variation in processes between individual researchers. For example, while TA using the framework method follows an established process described in the literature, it is acknowledged that the complex nature of qualitative analysis, and the role of the researcher in the process, means that there will always be variation between researchers in the exact physical and cognitive processes involved. It is therefore not possible to ‘standardise’ between researchers, within or between the two methods being compared. While we perceive comparisons of this nature to be worthwhile in order to develop and understand the applications of qualitative methods, they must include detailed description of and reflection on the processes and researchers.

The complexity of the process only became clear once the researchers began to compare the data. Differences in style and the degree of ‘polishing’ of the content and language with the RA team ‘crafting’ findings and recommendations deemed sensitive and appropriate to be shared with stakeholders, and the resulting impact on time taken was not apparent until analyses were complete and outputs shared. In addition, devising an approach to categorising and reporting mismatched findings and recommendations took time and was not as intuitive.

A further limitation is the fact that the comparison was conducted by the researchers themselves due to pragmatic resource constraints. While we acknowledged this and aimed to maintain objectivity, there is clearly a risk of bias in interpretation, and future projects should consider involving an independent, blinded third party to conduct the comparison.

An unexpected outcome of this study is that it has encouraged us to reflect deeply on our own research practice, resulting in a better understanding of our methods and role. Future comparisons may benefit from independent exploration of the researchers’ individual processes alongside the ‘outcomes’ of time, findings and recommendations. It is clear that there are a number of barriers that may constrain the research process in a service evaluation of the type we conducted. Greater reciprocal appreciation that these exist, and what they are, may help to facilitate discussions where there are unexpected or unpalatable research findings. 30

The initial intention was to involve participants in reviewing the importance of mismatched findings and recommendations. This was not practicable due to the unexpected length of time taken to complete the comparison, and the need for service stakeholders to determine whether mismatches would have been helpful many months in the past.

It is important to note that all researchers in this study were experienced in qualitative health research using TA, and as such this study does not explore RA and TA for novice researchers.

Possible explanations for the differences in time taken to conduct analysis

The time taken in the RA was much shorter at the data review and management stage, equating to around 2 weeks less whole time equivalent (WTE) researcher time. This suggests that managing data in this way within a short timeframe is possible. However, the interpretation and reporting phase was much longer with RA (6.5 days vs 1 day in TA). A number of factors may have contributed. Time saved in coding and data management may result in more time being required at the interpretation stage in RA. This needs further exploration; RA only took three WTE researcher days less that TA, which may be of little benefit to academic or health service stakeholders. There are further possible explanations: the researchers’ relationship with the service, the purpose of the research, the capacity of researchers and the fact that the RA team were learning a new skill. This is explored earlier in the strengths and limitations section.

Possible explanations for the difference in findings

The RA findings accounted for 78 of the 153, or 79% of the findings delivered by the TA. This considerable overlap indicates that TA, which codes all data, did not produce many additional findings. This is consistent with others’ findings comparing themes generated from different analytical approaches.

The most common reason for mismatches in findings was that the researchers had not identified the issue in error. In the RA, patterns and findings may have been missed as a result of the more deductive approach taken and the reduced time spent with primary data. However, there was a ‘did-not-find rate’ of around 1 in 10 for both methods, suggesting that this was not the case. The mismatches suggest that qualitative researchers will never elicit perfectly overlapping findings, regardless of method.

A number of mismatches were accounted for by unconscious suppression of challenging findings, higher level interpretation and differences in contextual knowledge leading to the rejection of findings. These explanations were more prevalent in the RA team, suggesting that embeddedness influences these processes. Between a quarter (RA) and a third (TA) of the mismatched findings were somewhat detailed, highlighting differences in natural reporting style, interpretation and prioritisation of what was meaningful. Again, this may arise between different researchers, regardless of method. Mays and Pope 31 relate how observations are ‘ limited by definition to the perceptions and introspection of the investigator ’, and variations in perception and introspection are inevitable between different individuals. There are different views regarding whether qualitative findings should be reproducible, 32 but we take the stance that subjectivity and individual variation make this impossible. This has been a useful exercise in reflexivity, demonstrating how experiences and unconscious processes impact on findings.

The TA team did not report positive findings, accounting for a further portion of the mismatch: this was attributed to differences in interpretation of the project scope, rather than analytical processes delivering different results. Also, the TA team were aware that they would not be presenting findings to providers, meaning that they felt more able to be critical and candid.

Possible explanations for the difference in recommendations

The recommendations also demonstrated overlap, with around three out of five being accounted for by both teams. However, RA did not pick up a third of the TA recommendations. We perceive that the majority reflected relevant but non-essential detail, and the ‘make or break’ recommendations that were key to the sustainability of the service were not missed, though we acknowledge that this is a subjective judgement. Arguably, the most important recommendation missed related to training midwives in administrative and management skills. This detail is consistent with the TA process, where the data were explored in more depth, leading to more precise recommendations. However, this pattern was not observed in the findings. A possible explanation is that the RA team, with the eventual audience in mind, were more conservative in the number and detail of recommendations. Over half of RA recommendations that the TA did not find were accounted for by higher level interpretation and contextual knowledge, and just under half of the TA mismatched recommendations were deemed inappropriate by the RA team due to contextual knowledge, suggesting that embeddedness in the field confers advantages, separate from the method used.

We found that RA was appropriate and delivered valid findings and recommendations, with reassuring but not complete overlap. Mismatches appeared to relate to minor or detailed issues. RA enabled considerable time savings in data management but may not be as rapid as assumed. Further work is needed, addressing the limitations identified to establish how much time experienced RA researchers can save, whether differences in outputs are due to the analytical method or other influences and whether these are relevant and of practical benefit for stakeholders and to services. Researcher characteristics, conduct and roles are key, and our impression is that RA requires the researchers to be embedded in the field.

We do not advocate RA for granular exploration of complex questions, for example, individuals’ experience of phenomena. It could be used to rapidly identify issues for further, in-depth qualitative exploration. RA represents one of many tools of the qualitative researcher’s trade, with particular potential for use in applied health research, when timely reporting is needed. We advocate further work to identify the practical application and use of different rapid approaches in practice.

Acknowledgments

We are grateful to Louise Bentham, who conducted thematic analysis of three of transcripts and reviewed findings with IL. We extend our thanks to the hospital trust who initiated the primary research project and granted permission for the work and to the staff who gave up their time and shared their perspectives so generously. We are grateful to the Scientific Advisory Committee of the West Midlands Collaboration for Leadership in Applied Health Research and Care, who provided feedback on initial findings and interpretation. CH acknowledges support from the NIHR Oxford cognitive health Clinical Research Facility.

• Yardley L ,
• Spiller K ,
• Daniel E , et al
• Garfield S ,
• Jacklin A , et al
• Martens PJ ,
• Vindrola-Padros C ,
• Vindrola-Padros B
• Johnson GA ,
• Vindrola-Padros C
• Cameron E , et al
• Ziebland S ,
• McPherson A
• Burgess-Allen J ,
• Owen-Smith V
• VanDyke E , et al
• 13. ↵ U.S. Department of Veterans Affairs . Qualitative methods in rapid turn-around health services research . http://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780 .
• Charteris-Black J ,
• MacFarlane A , et al
• Putten JV ,
• Greenwood M ,
• Kendrick T ,
• Davies H , et al
• 17. ↵ NHS . Collaboration for leadership in applied health research and care West Midlands . http://www2.warwick.ac.uk/fac/med/about/centres/clahrc/ .
• Ritchie J ,
• Roulston K ,
• Malthouse R ,
• Roffey-Barentsen J ,
• Richards HM ,
• Schwartz LJ
• Yiannis G ,
• Shneerson CL ,
• van der Graaf P ,
• Forrest LF ,
• Adams J , et al
• Dixon-Woods M ,
• Agarwal S , et al

Contributors The original idea for the project was conceived by SK. The study was designed by BT with intellectual input from all authors. Primary data collection was conducted by BT. Rapid analysis of data was conducted by BT and CH, with input from SK. Thematic analysis of data was conducted by IL with input from SG. Comparison of time data was conducted by BT and checked by CH and IL. Initial comparison of findings and recommendations was conducted by BT, CH, IL and SK, and all authors reviewed outputs from the comparison. The preliminary draft of the paper was written by BT. This was critically reviewed by CH, IL, SG and SK for important intellectual content and subsequent revisions to the paper were undertaken by BT as a result. Final approval of the version of the paper to be published was granted by BT, CH, IL, SG and SK, who all also agreed to be accountable for all aspects of the work in ensuring that questions relating to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding This work was supported by the National Institute for Health Research (NIHR) through the West Midlands Collaborations for Leadership in Applied Health Research and Care (CLAHRC-WM) programme.

Disclaimer The work was developed independently by the authors, and the views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Competing interests None declared.

Patient consent Not required.

Ethics approval The primary service review and secondary analysis were reviewed by the University of Birmingham Ethics Committee, ref ERN_15–0127S.

Provenance and peer review Not commissioned; externally peer reviewed.

Data sharing statement Data are securely stored at the University of Birmingham in line with our information governance and data protection policies. Due to the confidential nature of our qualitative data, which may identify individuals even following anonymisation, we have not made the data publicly available, in line with our research permissions and consent.

Read the full text or download the PDF:

• Open access
• Published: 13 May 2024

“ We might not have been in hospital, but we were frontline workers in the community ”: a qualitative study exploring unmet need and local community-based responses for marginalised groups in Greater Manchester during the COVID-19 pandemic

• Stephanie Gillibrand 1 ,
• Ruth Watkinson 2 ,
• Melissa Surgey 2 ,
• Basma Issa 3 &
• Caroline Sanders 2 , 4  

BMC Health Services Research volume  24 , Article number:  621 ( 2024 ) Cite this article

Metrics details

The response to the COVID-19 pandemic saw a significant increase in demand for the voluntary, community, faith and social enterprise (VCFSE) sector to provide support to local communities. In Greater Manchester (GM), the VCFSE sector and informal networks provided health and wellbeing support in multiple ways, culminating in its crucial supportive role in the provision of the COVID-19 vaccination rollout across the GM city region. However, the support provided by the VCFSE sector during the pandemic remains under-recognised. The aims of the study were to: understand the views and experiences of marginalised communities in GM during the COVID-19 pandemic; explore how community engagement initiatives played a role during the pandemic and vaccine rollout; assess what can be learnt from the work of key stakeholders (community members, VCFSEs, health-system stakeholders) for future health research and service delivery.

The co-designed study utilised a participatory approach throughout and was co-produced with a Community Research Advisory Group (CRAG). Focus groups and semi-structured interviews were conducted remotely between September-November 2021, with 35 participants from local marginalised communities, health and care system stakeholders and VCFSE representatives. Thematic framework analysis was used to analyse the data.

Local communities in GM were not supported sufficiently by mainstream services during the course of the COVID-19 pandemic, resulting in increased pressure onto the VCFSE sector to respond to local communities’ need. Community-based approaches were deemed crucial to the success of the vaccination drive and in providing support to local communities more generally during the pandemic, whereby such approaches were in a unique position to reach members of diverse communities to boost uptake of the vaccine. Despite this, the support delivered by the VCFSE sector remains under-recognised and under-valued by the health system and decision-makers.

Conclusions

A number of challenges associated with collaborative working were experienced by the VSCE sector and health system in delivering the vaccination programme in partnership with the VCFSE sector. There is a need to create a broader, more inclusive health system which allows and promotes inter-sectoral working. Flexibility and adaptability in ongoing and future service delivery should be championed for greater cross-sector working.

Peer Review reports

The response to the COVID-19 pandemic saw a significant increase in demand for the voluntary, community, faith and social enterprise (VCFSE) sector to provide support to local communities [ 1 , 2 ]. The role of communities was seen as crucial to supporting the pandemic response, to better mobilise public health pandemic responses and supportive health services [ 3 ]. VCFSE organisations nationally had to quickly mobilise to adapt their service offer to meet increased demand, new gaps in service provision and deliver services in different ways to address the challenges faced by local communities. These included loss of income and financial hardship, closure of schools and childcare, increased social isolation, digital exclusion, and increased mental health issues [ 4 ]. However, previous research has concluded that support provided by the voluntary sector during the pandemic has been under-recognised [ 5 ]. Some authors have explored the role that VCFSEs played at the national level, in supporting communities during the pandemic [ 4 , 5 , 6 ]. Yet, whilst it is well-known that tens of thousands of UK volunteers supported local vaccine delivery [ 7 ], no existing academic literature has explored the role of VCFSEs in supporting the vaccination rollout.

We focus on Greater Manchester (GM), where increased support from VCFSE organisations, including smaller, community-based networks, responded to increased demand from local communities and the NHS to provide key health and wellbeing-related services, including food and care packages for clinically vulnerable households, food bank services, support for people experiencing homelessness, mental health and domestic violence services and support to local community organisations [ 8 ]. This support culminated in the sector’s supportive role in the delivery of the COVID-19 vaccination rollout, in response to the need for mass immunisation across the region.

Over the last decade, the English health and care system has been evolving to integrate health and social care. A key focus is building closer working relationships between the NHS, local authorities and other providers– including the VCFSE sector– to deliver joined up care for communities [ 9 , 10 ]. To aid integration, a new model for organising health and care on different geographical footprints has been developed: Integrated Care Systems (ICSs), place-based partnerships and neighbourhood models. These collaborative partnerships bring together existing health and care organisations to coordinate health and care planning and delivery in a more integrated way and include councils, NHS provider trusts, Primary Care Networks, GP federations and health and care commissioners [ 11 ]. These new geographically-based partnerships have an emphasis on collaborative working beyond traditional health and care partners. This includes acknowledging the role that VCFSE organisations can have in supporting wider population wellbeing, particularly as part of multi-disciplinary neighbourhood teams embedded in local communities [ 12 ]. National guidance on the development of ICSs and place-based partnerships strongly encourages health and care leaders to include VCFSE organisations in partnership arrangements and embed them into service delivery [ 12 ]. In GM, the partnership working approach pre-dates the formal mandating of ICSs, with a combined authority which brings together the ten local authorities and an association of Clinical Commissioning Groups (CCGs) which represented health commissioners, and a VCFSE umbrella group which also operates as a joint venture to represent the sector’s interests at a GM level Footnote 1 . However, reorganisation to the ICS system may present new local challenges for the VCFSE sector to find a meaningful ‘seat at the table’. That withstanding, the COVID-19 pandemic coincided with the development of ICSs and place-based partnerships as arguably one of the earliest and most intense tests of partnership working across health and care organisations within the current policy landscape.

Here, we present findings from a co-designed qualitative research project, drawing on insights from 35 participants, including members of diverse communities in GM, VCFSE participants, and key decision-making health and care system stakeholders. The aims of the study were to: understand the views and experiences of marginalised communities in GM during the COVID-19 pandemic; explore how community engagement initiatives played a role during the pandemic and vaccine rollout; assess what can be learnt from the work of key stakeholders (including community members, VCFSEs, health and care system stakeholders) for future health research and service delivery. The rationale for the study developed from a related piece of work assessing inequalities in the COVID-19 vaccine uptake in GM [ 13 ]. At that time, there was little research on the experiences of under-served communities during the pandemic. As such, the public and stakeholder engagement for the related project identified a need for a qualitative workstream to explore more fully the drivers behind and context surrounding the vaccination programme in GM, centring also local communities’ experiences during the pandemic (explored in a related paper [ 14 ]).

In this paper, we examine the role the VCFSE sector played in supporting unmet needs for marginalised groups in GM during the COVID-19 pandemic and as part of the rapid rollout of the COVID-19 vaccination programme. We consider the opportunities and barriers that may influence the full integration of the VCFSE sector into health and care services in the future. This paper provides additional evidence around the role of local community-led support in the context of identified unmet needs from marginalised local communities. Whilst focused on GM, it provides an exemplar of the role of VCFSEs and community networks during the pandemic, with relevant learning for other regions and international settings with place-based partnerships.

Study design

The study utilised a participatory approach throughout and was co-designed and co-produced with a diverse Community Research Advisory Group (CRAG). The CRAG were members of local community groups who were disproportionately impacted by the COVID-19 pandemic, including one member who is a co-author on this paper. This included members of three VCFSE organisations working with specific ethnic minority communities including Caribbean and African, South Asian and Syrian communities.

CRAG members acted as champions for the research, supporting design of appropriate information and fostering connections for recruitment via their existing community networks. The strong partnerships built through our approach were crucial to enabling a sense of trust and legitimacy for the research amongst underserved communities invited to participate.

Interviews and focus groups took place between September-November 2021 and sought to explore: the context surrounding the rollout of the vaccination programme; key aspects of support delivered as part of the vaccination programme; the use of localised approaches to support vaccine delivery including engagement initiatives, as well as broader community-level responses to the COVID-19 pandemic; perceptions around barriers to vaccine uptake Footnote 2 ; experiences of local communities (including healthcare) during the pandemic Footnote 3 . During the data collection period, national pandemic restrictions were largely lifted with no restrictions on social distancing or limits to gatherings, and all public venues reopened. A self-isolation period of 10 days after a positive COVID-19 test remained a legal requirement, but self-isolation after contact with a positive case was not required if fully vaccinated [ 15 ]. By July 2021, every UK adult had been offered their first dose of the COVID-19 vaccine, with every adult offered both doses by mid-September 2021 [ 16 ]. By early September 2021, more than 92 million doses had been administered in the UK [ 15 ].

Interviews and focus groups were conducted by one member of the research team (SG) and were conducted remotely due to the pandemic, via Zoom and telephone calls. The limitations of undertaking remote qualitative research interviews are acknowledged in academic literature, including potential restrictions to expressing compassion and assessing the participant’s environment [ 17 , 18 ]. However, given the remaining prevalence of COVID-19 at the time of interview, it was judged that the ensuing risk posed by COVID-19 to both researchers and participants outweighed the potential drawbacks. Nevertheless, participants were offered face-to-face options if they were unable to participate remotely to maximise inclusion (although no participants chose to participate face-to-face).

Interviews and focus groups were audio recorded with an encrypted recorder and transcribed by a professional transcription service. Informed written consent to participate was taken prior to the interviews and focus groups. The average length of the interviews was 34 min and average length of the focus groups was 99 min. Two focus groups were co-facilitated by a CRAG member, a member of the local community who works for a mental health charity that supports local South Asian communities, who also provided translation support. In respect to authors positionality, coauthors SG, RW, MS and CS are university researchers in academic roles and had prior links to the CRAG members via a wider community forum (co-ordinated by the NIHR funded Applied Research Collaboration for Greater Manchester). The wider group met regularly to discuss and share learning regarding community experiences, community action and related research during the pandemic. BI is a member of the CRAG and a member of a local Syrian community.

Sampling & recruitment

The sampling strategy for community participants centred around groups that had been disproportionately affected by the COVID-19 pandemic in England, including ethnic minority groups, young adults, and those with long-term physical and mental health conditions. VCFSE participants included community and religious leaders, members of local community VCFSE organisations and smaller, informal community networks and groups from local communities. Health and care system stakeholders included local council workers and health and care system stakeholders (e.g. those organising the vaccination response in CCGs and GP Federations). Characteristics of the sample are provided in Table  1 . Overall, the study achieved a diverse sample of participants on the basis of gender and ethnicity.

A combination of purposive and snowballing sampling was used to recruit via pre-established links and connections to community networks and stakeholders to ensure the inclusion of specific seldom-heard groups. For example, members of African and Caribbean communities were recruited via a charity which supports the health of these groups, and members of South Asian communities were recruited via a mental health charity.

Quotes are described by respondent type (community member, VCFSE participant, health and care system stakeholder) and participant identifier number to maintain anonymity whilst providing important contextual detail.

Data analysis

We analysed the data using an adapted framework approach [ 19 ]. We adopted a framework approach to analysis as this is viewed as a helpful method when working within large multidisciplinary teams or when not all members of the team have experience of qualitative data analysis, as was the case within our team. This structured thematic approach is also considered valuable when handling large volumes of data [ 20 , 21 ] and was found to be a helpful way to present, discuss and refine the themes within the research team and CRAG meetings. We created an initial list of themes from coding four transcripts, and discussions with CRAG members: personal or family experiences/stories; work/education experiences; racism and racialised experiences; trust and mistrust; fear and anxiety; value of community/community approaches; access to services including healthcare; operational and logistical factors around vaccine rollout; communication and (mis)information. We used this set of themes and sub themes to code the remaining transcripts, including further inductively generated codes as analysis progressed, regularly discussing within the team.

We shared transcript coding amongst the study team, with one team member responsible for collating coded transcripts into a charting framework of themes/subthemes with illustrative transcript extracts. The themes were refined throughout the analysis period (November 2021-March 2022) with the research team and CRAG and were sense-checked with CRAG members and the wider study team, to synthesise a final iteration of the themes and sub-themes (see supplementary material). We present findings related to five overarching themes: (1) unmet needs of local communities during the pandemic: inaccessible care and distrust; (2) community-led approaches: social support and leadership to support services; (3) community led support to COVID-19 vaccination delivery; (4) operational and logistical barriers to community-based pandemic responses: challenges faced by the voluntary and community sector; (5) learning from the pandemic response in GM: trust building and harnessing community assets. Themes are discussed in more detail below.

Ethical approval

This study was approved by University of Manchester Ethics Committee (Proportionate University Research Ethics Committee) 24/06/21. Ref 2021-11646-19665.

Unmet needs of local communities during the pandemic: inaccessible care and distrust

The COVID-19 pandemic brought an unprecedented shift in the way NHS services could function due to social distancing and lockdown measures. Pressures included unprecedented demand on hospital capacity and infection control measures (within hospitals and across the NHS) which reduced workforce capacity. There were also staff shortages due to high levels of COVID-19 infection amongst NHS staff, and shortages in non-acute capacity due to staff re-deployment [ 22 , 23 ]. In an effort to reduce pressure on the NHS, the policy mantra “Protect the NHS” was coined as a keynote slogan from the early stages of the pandemic [ 24 ].

It is within this context that many community participants raised (spontaneously) that there was a general inability to access health services during the pandemic, including GP and specialist services.

when I tried to contact my doctor’s surgery I was on the call for over an hour, number 20, number 15. Then by the time I’m under ten I get cut off. And it happened continuously. I just couldn’t get through and I just gave up really…now it’s like a phone consultation before you can even go and see someone, and even for that you’re waiting two, three weeks. (1029, VCFSE participant)

This resulted in frustration amongst some community participants, who questioned the logic of “protecting the NHS”, seemingly at the expense of their health-related needs. This led to sentiments that other health needs were de-prioritised by decision-makers during the pandemic. It was felt that this logic was counter-productive and fell short of the principles of protecting the most vulnerable.

We were like it just didn’t matter, it could have been much more serious than just a cough or a cold, [] but the help was just not there” (1028, community participant). what about people who actually need to see a doctor so the very vulnerable ones that we’re supposed to be protecting. Yes, we’re protecting the NHS, I understand that, I said, but we’ve also got to protect all those vulnerable people that are out there that are actually isolated (1011, community participant).

Community participants described their fear of accessing healthcare service because of potential risks of catching the virus in these settings, and fear of insufficient care due to well-publicised pressures in NHS settings. Some VCFSE participants noted that the widely publicised pressures faced by the NHS, and heightened media and political attention around COVID-19 cases in health settings led to fear and anxiety Footnote 4 .

I didn’t go to the hospital because I was scared shitless whether I was going to come out alive from hospital.” (1023, community participant). …the number of people who didn’t access services when they should have done… They were either terrified they were going to go into hospital and catch COVID straightaway and die, or they were terrified that they were taking [the hospital space] away from someone else (2003, VCFSE participant).

Overall, this led to a strong sense that mainstream services were not supporting the needs of local communities. This was especially felt for those requiring specialist services (e.g. mental health or secondary services), and for those who had faced intersecting inequalities, such as health issues, language and digital/IT barriers, and newly settled refugees and immigrants.

Community-led approaches: social support and leadership to support services

As a consequence of this unmet need, VCFSE and community participants identified that local communities themselves increased activities to provide community support. Participants felt strongly that this increased support provided by the VCFSE sector and community networks remains under-recognised and under-valued by the health system and wider public.

BAME organisations were going around door to door, giving hand sanitisers, giving masks to everybody [ ]. And it was the BAME community that was the most active during COVID delivering medication, delivering food to houses, doing the shopping. [ ] Nobody gave credit to that. Nobody talks about the good work that the BAME community has done. (1020, community participant)

A number of community and VCFSE sector participants highlighted the work done at the community level, by either themselves or other networks to support local communities. This included providing support packages, running errands for vulnerable community members, cooking and food shopping services, a helpline and communication networks for local communities, and online wellbeing and support groups.

We might not have been in hospital, but we were frontline workers in the community. (1028, community participant)

Support was provided by formal VCFSE organisations and by smaller, sometimes informal, community networks and channels, in which support mechanisms included mental health support and wellbeing focused communications to combat loneliness and boost wellbeing. This was often focused around outreach and the provision of community-based support to the most marginalised and vulnerable groups that had been disproportionately impacted during the pandemic, e.g. recently settled refugees and asylum seekers, older individuals.

We have an Iranian group in Salford…And one of them spotted this young woman in the queue and she thought she looked Iranian, you know….anyway she started a conversation, and this person had been an asylum seeker at the beginning of the pandemic and had been in a detention centre during the pandemic. And then, finally got their leave to remain and then were just basically dumped in Salford. [ ] just having that friendly face and someone was trying to start that conversation, she was able to be linked into this group of women who support other refugees and asylum seekers from the Middle East. (2014, VCFSE participant)

Community led support to COVID-19 vaccination delivery

The VCFSE sector and community networks also played a crucial part in supporting the COVID-19 vaccine delivery. Community, VCFSE and system-sector participants recognised the unique role that the VCFSE sector had played in reaching diverse communities and sections of communities not reached by the mainstream vaccination programme. For example, VCFSE groups aided vaccine delivery by helping run vaccine ‘pop-up’ sites in community spaces including mosques and other religious sites, children’s centres, and local specialist charities (e.g.: refugee and sex worker charities).

The use of community ‘champions’ and community ‘connectors’ to convey messaging around the vaccination drive were deemed especially vital in this regard. Trusted members of communities (e.g. community leaders) who had crucial pre-existing communication channels were able to effectively interact with different parts of communities to advocate for the vaccine and address misinformation. Situated within communities themselves, these ‘champions’ held established trust within communities, allowing conversations surrounding the vaccine to be held on the basis of shared experiences, honesty, openness, compassion and understanding.

So, as with any ethnic minority community, unless you’re part of it, it’s almost impossible to completely dig out all its norms and its very, very fine distinctions…[ ] what is acceptable, what is not acceptable[ ]? Unless you’re part of it, or you’ve really immersed yourself in the culture for decades, it’s almost impossible to get it (2015, VCFSE participant) One of the strongest approaches that you can take to increase uptake in any community, whether it be pregnant women or a faith group or a geographical area or a cultural group, is that if you’ve got a representative from that community leading on and advocating for the vaccine, you’re going to have the best impact (2011, health and care system stakeholder participant). unless Imams or significant people in the community were coming out for them and saying, it’s absolutely fine, it’s safe, and culturally it’s the right thing to do, there was a bit of uncertainty there (2010, health and care system stakeholder participant).

Health and care system stakeholders also emphasised the importance of “community ownership” of vaccination approaches, and of system responsiveness to identified needs and priorities at the community level. Health and care system stakeholders recognised that they were able to utilise community links to have better on-the-ground knowledge, provided in real time, to supplement locally held data to inform targeted efforts to boost uptake. This included council led initiatives including door-knocking with council staff, local health improvement practitioners, and VCFSE representatives working together to provide information about vaccine clinics and register people for vaccine appointments.

if messages went out and they didn’t land right they [the VCFSE sector] could be the first people [that] would hear about that and they could feed that back to us. [ ]….we were able to regularly go to them and say, look from a geographical perspective we can see these key areas…[ ] the people aren’t coming for vaccinations, [ ] what more can you tell us. Or, we can say, from these ethnicities in this area we’re not getting the numbers, what more can you tell us. And when we’ve fed them that intelligence then they could then use that to go and gain further insight for us, so they were a kind of, key mechanism (2010, health and care system participant).

Operational and logistical barriers to community-based pandemic responses: challenges faced by the voluntary and community sector

VCFSE sector and health and care system stakeholder participants reported significant logistical barriers to partnership working to support communities during the pandemic. Barriers included red tape and bureaucracy, which delayed responses to communities’ health and wellbeing needs.

whilst we were buying masks and hand sanitisers and going door to door, [ ] the council were still getting their paperwork in order, their policies in order, it was meeting after meeting. It took them seven to eight weeks for them to say [ ] we’ve got masks, would you like to help dish them out. (1029, VCFSE participant)

VCFSE and health and care system participants also raised challenges with respect to the VCFSE sector supporting the vaccination programme. This resulted in frustration amongst both VCFSE and health and care system participants who recognised the value of these community-based approaches.

The time that trickles through to the council and the time that the council turn around and say all right, we’ll actually let you do it was weeks later, and the community is turning round to us and saying to us well, what’s going on? We don’t like being messed around like this… (2008, VCFSE participant).

Participants highlighted the numerous health-related bodies with various roles which comprise a complex system for VCFSE partners to navigate, in part due to organisational and cultural clashes. Frustration was felt by both VCFSE and health and care system stakeholder participants (from local councils) in this respect. One VCFSE participant discussing the vaccine rollout noted:

We hit dead end after dead end within the council and there was literally very little response….You’ve got so many departments within this massive organisation called the council…[ ].it’s very difficult to navigate all that and deal with all that bureaucracy… (2008, VCFSE participant).

Broader institutional and organisational barriers to VCFSE support were identified, where cultural clashes between differing values and ways of working emerged, including ethos surrounding risk aversion and the system-level commitment to privilege value-for-money during the vaccination rollout. More practical issues around information governance and training were also raised as barriers to collaborative working.

I don’t think that they understand the power of community and the way community works. I don’t think that at a governmental level they understand what it means to penetrate into a community and actually understand what needs to be done to help a community…[ ] If they did and they had better links and ties into understanding that and helping that then we likely wouldn’t have had so many hurdles to get through (2008, VCFSE participant). ….in terms of public money, this is a public programme, we need to get value for the public pound. So we’re saying to [VCFSE organisation], how much is it going to cost? And [VCFSE organisation] are like, well, we don’t really know, until we deliver it. And we’re like, well, we can’t really approve it, until we know what it’s going to cost…. (2006, health and care system stakeholder participant)

Overall, these issues surmounted to difficulties of power-sharing between public sector organisations and VCFSEs during a time of rapid response to a public health crisis, political, institutional, and other external pressures. This was echoed amongst VCFSE and health and care system stakeholder participants, where frustration towards this was felt from both sides.

the public sector [ ] need to get better at letting go of some of the control. So even still, after I said, so many times, [VCFSE organisation] are delivering this, [VCFSE organisation] are doing everything, [ ] I still got the comms team going, are we doing a leaflet? No, [VCFSE organisation] are doing it, this is a [VCFSE organisation] programme, this isn’t a Council programme. (2006, local authority participant) it is difficult sometimes working with organisations, I find myself very much stuck in the middle sometimes [ ] I engage with [community groups] and ask them how best we do it and then we put things in place that they’ve asked for, and then they’ve told us it’s not working why have you done it like that. [ ] I think it’s acknowledgement to do it right, it takes time, and it takes effort, it takes resource. (2010, local authority participant)

Health and care system stakeholders also highlighted the importance of accessibility and localised vaccination hubs to reach different parts of diverse local communities e.g. sites in local mosques and sites near local supermarkets to reach different demographics. For instance, having mobile vaccination sites to reduce accessibility barriers, alongside dialogue-based initiatives to answer questions and respond to concerns from local communities about the vaccine, with the view to building trust without explicit pressure to receive the vaccine. Describing their efforts to engage with a member of the local community over the vaccine, two local health and care system stakeholders detailed the following example of how localised, communication-based approaches were deemed successful:

She came to the clinic and there were a lot of tears. It was very emotional. She’d been through a very difficult journey and had got pregnant by IVF, so it was a big decision for her, a big risk that she thought she was taking. Whether she took the vaccine or not, it felt like a risk to her, [ ] we were able to sit down and talk to her. We had some peers there. So we had other pregnant women there who’d had the vaccine, that were able to give her some confidence. We had the specialist multicultural midwife there, [ ] And we literally just sat and drank coffee with her and let her talk and she ended up agreeing to have the vaccine [ ] (2011, system-level stakeholder). …And the feedback from that lady was amazing. A couple of weeks ago I contacted her to make sure she was going to come down for her booster and she was just so grateful. [ ] she’d had backlash from her family and people within her community for taking up the vaccine and they still thought it was a massive risk. But she had no doubts that she’d done absolutely the right thing… (2012, system-level stakeholder).

Learning from the pandemic response in GM: trust building and harnessing community assets

Taking these findings from health and care system stakeholders, community and VCFSE participants, several learning points were identified.

In terms of vaccine delivery, some health and care system stakeholder participants reflected the need for more joined-up ways of working, across existing services and amongst VCFSE partners, to ensure efficiency and maximise uptake by embedding the vaccination programmes into other health services. For example, offering vaccination through health visiting or health checks, or offering COVID-19 vaccine boosters and flu vaccinations in single visits at care homes. These settings could also provide opportunities for dialogue with local communities where there is pushback against vaccination. Another health and care system stakeholder identified the need for greater joined up delivery of services; utilising the VCFSE sector to deliver multiple services simultaneously, including the vaccine, to improve vaccine uptake and access to other healthcare services:

the sex worker clinic is a good example of that. [ ] People were coming in for another reason, to get their health check and to get their support from the advisors there at that voluntary organisation, [ ]…if there’s a multiple purpose at the site, for people to attend, you can start to engage them in the conversation and then take the opportunity and vaccinate them. So I’m really interested in looking at that a little bit more, about how that can help to increase uptake. (2011, health and care system stakeholder participant)

A VCFSE participant suggested using educational settings such as schools as a channel to disseminate public health and vaccine-related information, as trusted settings which have wide-reach to many different communities.

A number of health and care system stakeholders, VCFSE and community participants noted that long-term, continuous, meaningful engagement is crucial to build longer-term trust between institutions and communities, and to improve the efficacy of public health measures. It was felt that more concentrated efforts were required from the NHS and other statutory organisations to reach the most marginalised and minoritised communities, for example through door-knocking and welfare calls. Participants highlighted that this was required not solely at times of public health crises, but as part of continued engagement efforts, in order to adequately engage with the most marginalised groups and effectively build long-term trust. This may be done most effectively by building on existing links to marginalised communities, for example using education liaison staff to understand traveller communities’ perspectives on the vaccine.

proactive engagement with communities both locally and nationally to say, [the health system] are looking at this, what’s people’s thoughts, views, you know, is there any issues with this, what more can we do, what do you need to know to make an informed decision. This is what we were thinking of, how would this land…I think we could learn by, [ ] doing that insight work, spending more time working with communities at a kind of, national, regional, and local level (2010, health and care system stakeholder participant). [the health system] could have engaged better with communities, I think bringing them in at the beginning. So, having them sat around the table, representatives from different groups, understanding how to engage with them from the very beginning…I think they could have used the data very very early on to inform who were engaging. We didn’t quite get it right at the beginning, we didn’t link the public health data teams with the comms and engagement teams (2013, health and care system stakeholder participant).

The tone of communications was also seen to be important. One health and care system stakeholder participant noted that the strategy of pushing communications and public health messaging aimed at behavioural change did not achieve the desired effect as these did not engage effectively with the communities to alleviate or address key concerns about the vaccine. These were deemed less successful than starting from a place of understanding and openness to generate constructive dialogue which could foster trust and respect.

There was also more specific learning identified in terms of collaboration between public sector institutions, VCFSEs and community links, with this seen as vital to build strong, long-term relationships between sectors based on trust and mutual respect. This should also involve working to share knowledge between sectors in real-time.

Health and care system stakeholder and VCFSE participants both suggested a failure to further develop partnerships fostered during the pandemic would be a lost opportunity that could potentially create distrust and additional barriers between communities, VCFSEs and public organisations, perhaps further marginalising seldom-heard groups.

we need to find ways which we have ongoing engagement, and I think it needs to be more informal. People don’t want to be just constantly asked and asked and asked (2010, health and care system stakeholder participant). a network of just sharing information and insight, rather than just engaging when you’ve got something specific to engage about. (2010, health and care system stakeholder participant) We were then thinking to ourselves, well, maybe we shouldn’t be doing this. If it’s going to cause us damage, if the council can’t work with us properly maybe we just shouldn’t do it. We’ve got to weigh up. We don’t want to lose our trust within the community (2008, VCFSE participant).

In terms of dynamics and working arrangements between sectors, participants thought it important to allow community organisations and VCFSEs to lead on their areas of speciality, e.g.: community organisations leading on outreach and communications within and to communities. This relates to the identified need of pursuing adaptable and flexible approaches to vaccine delivery. Moreover, there is a need to allow more joined-up decision-making between the health system and VCFSEs to ensure better use of local intelligence and improved planning.

Discussion & policy implications

Unmet need and the role of communities during the pandemic.

Our findings clearly demonstrate that local communities were not supported sufficiently by mainstream services during the COVID-19 pandemic. This in turn led to frustration, fear and loss of faith in the healthcare system as a whole, evidenced also in responses to the COVID-19 vaccination programme in which distrust results from wider experiences of historical marginalisation and structural inequalities [ 14 ]. In the absence of mainstream service support, our findings demonstrate how VCFSE organisations and community networks mobilised to support local communities to fulfil unmet health, social care, and wellbeing needs. This supports emerging evidence from across England which finds that the VCFSE sector played a key role in supporting communities during the pandemic [ 6 , 8 , 25 ].

The importance of community-based, localised approaches, community-led and community owned initiatives, ‘community champions’ and community connectors’ were also highlighted as crucial to the success of the COVID-19 vaccination drive. Participants noted that community-led approaches were uniquely positioned to reach some communities when mainstream approaches were unsuccessful. This is echoed in existing literature, where the role of localised community responses was deemed important to reach marginalised groups, as part of the wider pandemic response [ 26 ].

Operational and logistical barriers

Operational and logistical barriers created dissonance between communities and the system. These barriers included difficulties with decision-making and power-sharing between VCFSE and commissioning or clinical organisations, organisational cultural clashes, red-tape and bureaucracy, and complex systems and power structures to navigate. This builds on existing evidence of barriers to partnership working during the pandemic, including cultural clashes and bureaucracy/red tape [ 5 , 27 ]. The VCFSE sector also suffered from the closure of services, and reduced funding and resources due to increased demand for services and needing to adapt service provision [ 8 ].

These factors hindered collaborative working and created risk for VCFSEs, including putting tension on relationships with local communities resulting from delays implementing services. In most VCFSE-health system partnerships, participants noted that power is generally held by the health system partner, but reputational risk and additional resource-based costs lie with VCFSE partners. Supporting capacity building and workforce resource within the voluntary sector will strengthen this [ 28 ].

Inadequate processes to establish collaborative working enhance distrust between the health system and VCFSE sector, which in turn enhances difficulties for collaborative working. Trust is an important factor in how the system interacts with VCFSEs, with a lack of trust leading to further bottlenecks in VCFSE activities [ 29 ]. Alongside this, is the need for greater health system appreciation for the VCFSE sector, with VSCE partners reporting they faced greater scrutiny and more arduous administrative processes than private sector partners [ 2 , 29 ].

Learning from the pandemic: service prioritisation

All sectors of the health and care system face pressures from resource shortages, internal and external targets [ 30 , 31 ]. This is often linked to drives to increase the value-for-money of services, but key questions remain as to how to assimilate the goals of achieving health equity within value-for-money objectives [ 32 ]. To this end, prioritising value-for-money may come at odds with reducing health inequities. For example, during the rollout of the vaccination programme, additional resources and innovative approaches were required to reach marginalised communities [ 33 , 34 ]. This is supported by emerging evidence from England and internationally that efforts to drive vaccination uptake and reduce inequities in uptake amongst marginalised populations require significant resources and a breadth of approaches to maximise uptake [ 34 ]. Our findings suggest that changes in vaccine uptake were smaller and slower to be realised in these populations, resulting in a “slow burn” in terms of demonstrating quantifiable outcomes. Given the NHS principles of equity [ 10 , 35 ], reaching these groups should remain a public health priority, and failure to prioritise these groups may incur greater long-term financial costs resulting from greater health service needs. Our findings support that challenging entrenched attitudes and frameworks for how success is measured and adapting structures to better incentivise targeted interventions for marginalised or high-risk groups is essential to prioritising addressing unmet needs amongst marginalised communities.

The changing commissioning landscape

The development of ICSs and place-based partnerships has changed how health and care services are commissioned. National guidance encourages health and care leaders to include VCFSE organisations in partnership arrangements and embed them into service delivery [ 12 ], with ‘alliance models’ between ICSs and the VCFSE sector [ 36 ] established in certain regions (see for example [ 37 ]. However, this rests on “a partnership of the willing” [ 37 ] between ICS partners and VCFSE sector players, and concrete guidance for achieving collaborative working in practice, is lacking. As the findings in this paper point to, evolving decision-making processes may add to resource burdens for VCFSE organisations. Traditional health and care partners such as the NHS and local authorities should consider how their ways of working may need to change to foster full VCFSE inclusion on an equal standing, otherwise only the VCFSE stakeholders with sufficient capacity and resource may be able to be meaningfully involved.

Creating a VCFSE-accessible health and care system

In terms of fostering relationships between different sectors, participants acknowledged that pre-pandemic efforts to engage communities and community networks and VCFSEs were insufficient, with more meaningful, well-resourced engagement required going forward. It was also identified by participants the importance of avoiding tokenistic involvement of the VCFSE sector, which may be counter-productive for developing meaningful long-term partnerships. More equal relationships between statutory and VCFSE sectors are needed to foster improved collaborative working [ 5 , 38 ], and this is identified already at the GM level [ 28 ]. Central to this is actioned principles of co-design, including power-sharing, community ownership and trust. In order for co-design strategies to be successful, recognition of the role of the VCFSE sector and their ownership of approaches must be championed within co-design strategies and the enactment of co-designed activities.

Relatedly, greater trust of the VCFSE sector to deliver services effectively and efficiently is needed from health and social care decision-makers to ensure that funding compliance measures and processes are proportionate and not overly burdensome, to avoid funding bottlenecks which in turn impact service delivery [ 2 ]. Currently at the national level, VCFSE applicants typically only become aware of funding through existing networks, leaving less-connected organisations to find out ‘by chance’, thereby limiting reach amongst other organisations [ 2 ]. This may be especially true for smaller or ad-hoc VCFSE networks and groups. Our findings support that bottlenecks to applying for funding should be removed, and more streamlined processes for accessing funding championed [ 2 ].

Our findings also suggest that health systems should engage with the full breadth of the VCFSE sector, creating space for the involvement of smaller scale and less formal organisations as partners. Sharing of best practice and advice for adapting to local contexts should be promoted, alongside evaluation of collaborative models.

Finally, the pandemic period saw unprecedented state-sponsored investment into the VCFSE sector [ 29 ]. Within the GM context, this funding enabled VCFSEs to develop organisational capacity and systems, develop new partnerships, and better respond to the (unmet) needs of local communities [ 39 ]. Currently there are no clear plans to maintain this investment, but sustained inter-sector partnership working will require continued investment in the VCFSE sector.

Strengths & limitations

There are two main limitations to this study. Firstly, whilst the study achieved diversity in its sample, we could not achieve representation across all marginalised communities and therefore could not cover the experiences of all marginalised communities in-depth. As such, whilst the analyses provides valuable insights, such insights may not be transferrable and do not reflect all communities in GM. Secondly, whilst other studies focused on multiple city-regions or areas, our study is limited to the city region of GM. However, this focus provides an in-depth analysis on one region, and, as we discuss in the framing of the paper, we contend that the analysis presented in this paper serves as an exemplar to explore further at the national and international level. It should also be noted that co-design approaches are inevitably time and resource-heavy, and this was challenging in the context of this study, as local stakeholders wanted timely insights to inform the vaccination programme. However, one of the key strengths of our participatory approach was that this enabled a direct connection with the experiences of communities as relevant to the research, in order to shape the research questions, as well as the design and conduct of the study.

Overall, the contribution of the VCFSE sector during the pandemic is clear, with significant support provided in respect to community health and wellbeing and vaccination delivery. Nevertheless, there remains much to learn from the pandemic period, with the potential to harness capacity to tackle inequalities and build trust through shared learning and greater collaborative working. Maintaining an environment in which VCFSE partners are under-recognised, under-valued, and seemingly face further bureaucratic barriers will only exacerbate issues to collaborative working. There are also significant questions around systemic issues and sustainability, which must be addressed to overcome existing barriers to collaborative working between sectors. For instance, our findings identify the importance of flexibility and adaptability, in ongoing and future service delivery. Where this is not pursued this may not only impact service delivery but also create roadblocks to collaboration between sectors, creating divisions between entities whilst ultimately trying to effect change on similar goals (i.e. improved population health). ICS–VCFSE Alliances and community connectors may be a mechanism to promote this, but clear, actionable guidance will be required to translate rhetoric to real-world progress.

Data availability

Data for this research data will not be made publicly available as individual privacy could be compromised. Please contact Stephanie Gillibrand ([email protected]) for further information.

10 GM is an umbrella group which seeks to represent the VCSE sector in GM. More information is available here: https://10gm.org.uk/ .

These themes are explored in a related paper by Gillibrand et al. [ 14 ].

Topic guides are provided as supplementary material.

Distrust was also raised in relation to fear and anxiety in NHS settings, and this is discussed in detail in a related paper from this study by Gillibrand et al. [ 14 ].

Abbreviations

Clinical Commissioning Groups

Community Research Advisory Group

Greater Manchester

Integrated Care Systems

Voluntary, Community and Social Enterprise

Craston MRB. Susan Mackay, Daniel Cameron, Rebecca Writer-Davies, Dylan Spielman. Impact Evaluation of the Coronavirus Community Support Fund. 2021.

NatCen Social Research. Evaluation of VCSE COVID-19 Emergency Funding Package. Department for Digital, Culture, Media & Sport (DCMS); 2022. 27 April 2022.

Marston CRA, Miles S. Community participation is crucial in a pandemic. Lancet. 2020;395(10238):1676–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Frost S, Rippon S, Gamsu M, Southby K, Bharadwa M, Chapman J. Space to Connect Keeping in Touch sessions: A summary write up (unpublished). Leeds: Leeds Beckett University; 2021 2021.

Pilkington G, Southby K, Gamsu M, Bagnall AM, Bharadwa M, Chapman J, Freeman C. Through different eyes: How different stakeholders have understood the contribution of the voluntary sector to connecting and supporting people in the pandemic.; 2021.

Dayson CaW A. Capacity through crisis: The role and contribution of the VCSE Sector in Sheffield during the COVID-19 pandemic; 2021.

Timmins B. The COVID-19 vaccination programme: trials, tribulations and successes. The Kings Fund; 2022.

Howarth M, Martin P, Hepburn P, Sheriff G, Witkam R. A Realist evaluation of the state of the Greater Manchester Voluntary, Community and Social Enterprise Sector 2021. GMCVO/University of Salford; 2021.

NHS England. Five Year Forward View. Leeds. 2014 October 2014.

NHS England. The NHS Long Term Plan. NHS England. 2019 January 2019.

Surgey M. With great power: Taking responsibility for integrated care. 2022.

NHS England. Integrating care: next steps to building strong and effective integrated care systems across England. Leeds: NHS England; 2020.

Google Scholar  

RE W. Ethnic inequalities in COVID-19 vaccine uptake and comparison to seasonal influenza vaccine uptake in Greater Manchester, UK: a cohort study. PLoS Med. 2022;19(3).

Gillibrand S, Kapadia D, Watkinson R, Issa B, Kwaku-Odoi C, Sanders C. Marginalisation and distrust in the context of the COVID-19 vaccination programme: experiences of communities in a northern UK city region. BMC Public Health. 2024;24(1):853.

Article   PubMed   PubMed Central   Google Scholar  

Cabinet Office. COVID-19 response: autumn and Winter Plan 2021. Guidance: GOV.UK; 2021.

Department of Health and Social Care. Every adult in UK offered COVID-19 vaccine [press release]. GOV.UK, 19 July 2021 2021.

Irani E. The Use of Videoconferencing for qualitative interviewing: opportunities, challenges, and considerations. Clin Nurs Res. 2019;28(1):3–8.

Article   PubMed   Google Scholar  

Seitz S. Pixilated partnerships, overcoming obstacles in qualitative interviews via Skype: a research note. Qualitative Res. 2016;16(2):229–35.

Article   Google Scholar  

Gale NK. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13(1):1–8.

Castleberry A, Nolen A. Thematic analysis of qualitative research data: is it as easy as it sounds? Curr Pharm Teach Learn. 2018;10(6):807–15.

Braun V, Clarke V. Thematic analysis. APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological2012. pp. 57–71.

Burn S, Propper C, Stoye G, Warner M, Aylin P, Bottle A. What happened to English NHS hospital activity during the COVID-19 pandemic? Brief Note IFS; 2021 13th May 2021.

NHS. COVID-19: Deploying our people safely. 2020 [updated 30th April 2020. https://www.england.nhs.uk/coronavirus/documents/COVID-19-deploying-our-people-safely/ .

Department of Health and Social Care. New TV advert urges public to stay at home to protect the NHS and save lives. [press release]. Department of Health and Social Care, 21st. January 2021 2021.

McCabe A, Wilson M, Macmillian R. Stronger than anyone thought: communities responding to COVID-19. Local Trust. Sheffieldn Hallam University. TSRC.; 2020.

McCabe A, Afridi A, Langdale E. Community responses to COVID-19: connecting communities? How relationships have mattered in community responses to COVID-19 Local Trust. TSRC, Sheffield Hallam University; 2022. January 2022.

Carpenter J. Exploring lessons from Covid-19 for the role of the voluntary sector in Integrated Care Systems. July 2021. Oxford Brookes University; 2021.

Greater Manchester Combined Authority. GM VCSE Accord Agreement. 2021 [ https://www.greatermanchester-ca.gov.uk/media/5207/gm-vcse-accord-2021-2026-final-signed-october-2021-for-publication.pdf .

Department for Digital, Culture, Media & Sport. Financial support for voluntary, community and social enterprise (VCSE) organisations to respond to coronavirus (COVID-19).: Department for Digital, Culture, Media & Sport and Office for Civil Society. 2020 [updated 20th may 2020. https://www.gov.uk/guidance/financial-support-for-voluntary-community-and-social-enterprise-vcse-organisations-to-respond-to-coronavirus-COVID-19 .

Smee C. Improving value for money in the United Kingdom National Health Service: performance measurement and improvement in a centralised system. Measuring Up: Improving Health Systems Performance in OECD Countries; 2002.

McCann L, Granter E, Hassard J, Hyde P. You can’t do both—something will give: limitations of the targets culture in managing UK health care workforces. Hum Resour Manag. 2015;54(5):773–91.

Smith P. Measuring value for money in healthcare: concepts and tools. London: Centre for Health Economics, University of York. The Health Foundation; 2009 September 2009.

Ekezie W, Awwad S, Krauchenberg A, Karara N, Dembiński Ł, Grossman Z, et al. Access to Vaccination among Disadvantaged, isolated and difficult-to-Reach communities in the WHO European Region: a systematic review. Vaccines. 2022;10(7):1038.

British Academy. Vaccine equity in Multicultural Urban settings: a comparative analysis of local government and community action, contextualised political economies and moral frameworks in Marseille and London. London: The British Academy; 2022.

England NHS. Core20PLUS5 (adults)– an approach to reducing healthcare inequalities 2023. https://www.england.nhs.uk/about/equality/equality-hub/national-healthcare-inequalities-improvement-programme/core20plus5/ .

NHS England. Building strong integrated care systems everywhere 2021. Available from here: https://www.england.nhs.uk/wp-content/uploads/2021/06/B0664-ics-clinical-and-care-professional-leadership.pdf .

Anfilogoff T, Marovitch J. Who Creates Health in Herts and West Essex? Presentation to NHS Confederation Seminar: Who Creates Health? 8 November 2022. 2022.

Bergen JWS. Pandemic pressures: how Greater Manchester equalities organisations have responded to the needs of older people during the covid-19 crisis. GMCVO; 2021.

Graham M. Learning from Covid-19 pandemic grant programmes lessons for funders and support agencies. May 2022. GMCVO; 2022.

Download references

Acknowledgements

The research team would like to thank ARC-GM PCIE team (Sue Wood, Aneela McAvoy, & Joanna Ferguson) and the Caribbean and African Health Network for their support in this study. We would also like to thank the Advisory Group members: Nasrine Akhtar, Basma Issa and Charles Kwaku-Odoi for their dedicated time, commitment, and valuable inputs into this research project and to partners who contributed to the early inception of this work, including members of the ARC-GM PCIE Panel & Forum & Nick Filer. We would also like to extend our thanks to the study participants for their participation in this research.

The project was funded by an internal University of Manchester grant and supported by the National Institute for Health and Care (NIHR) Applied Research Collaboration for Greater Manchester. Melissa Surgey’s doctoral fellowship is funded by the Applied Research Collaboration for Greater Manchester. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.

Author information

Authors and affiliations.

Centre for Primary Care and Health Services Research, University of Manchester, Greater Manchester, England, UK

Stephanie Gillibrand

NIHR Applied Research Collaboration for Greater Manchester, Greater Manchester, England, UK

Ruth Watkinson, Melissa Surgey & Caroline Sanders

Independent (public contributor), Greater Manchester, England, UK

Greater Manchester Patient Safety Research Centre, University of Manchester, Greater Manchester, England, UK

Caroline Sanders

You can also search for this author in PubMed   Google Scholar

Contributions

SG, lead writer/editor, design of the work, RW, design of the work, drafting of article, review and revise suggestionsMS, draft of the article, review and revise suggestionsBI, design of the work, review and revise suggestionsCS, design of the work, draft of the article, review and revise suggestionsAll authors read and approved the final manuscript.

Corresponding author

Correspondence to Stephanie Gillibrand .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by University of Manchester Ethics Committee (Proportionate UREC) 24/06/21. Ref 2021-11646-19665. Informed consent to participate in the research was taken from all research participants ahead of their participation in the study. Consent to participate in the study was taken from each participant by a member of the research team. All experiments were performed in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Gillibrand, S., Watkinson, R., Surgey, M. et al. “ We might not have been in hospital, but we were frontline workers in the community ”: a qualitative study exploring unmet need and local community-based responses for marginalised groups in Greater Manchester during the COVID-19 pandemic. BMC Health Serv Res 24 , 621 (2024). https://doi.org/10.1186/s12913-024-10921-4

Download citation

Received : 10 November 2023

Accepted : 28 March 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12913-024-10921-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Marginalised groups

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]