Facilitators summarize the key learnings from the morning's case studies
At the Salzburg Global Seminar program Better Health Care: How do we learn about Improvement? in the first morning session, we considered five case studies of real-life improvement projects around the world.
Patient Discharge Checklists in the UK
Antenatal Care in India
Pre-eclampsia, Malaria and Anaemia in Uganda
Water Quality in Ghana
Anaemia in Mali
Our challenge was to understand the improvement & evaluation methods, intervention details, and results in the improvement projects. The Fellows identified several themes across all of the case study discussions (see report out from each individual group). Themes are described below:
- Engaging leadership and stakeholders in the design of both the improvement and evaluation methods and activities to facilitate ownership and accountability for results
- Build capacity of QI teams to review data, identify gaps and design impactful improvement projects
- Use data in both aggregate and individual team form for optimal learning
- Key principles of changes are more generalizable than specific detailed changes (Specify roles of team member vs. detailed descriptions of exact roles of staff members)
- Qualitative assessment needed too, description of interventions, detailed
- Add methods from other disciplines i.e., anthropology, etc. - Bring patients into design of improvement interventions, include communities voice in design
- Missing from discussion: how did the team analyze their processes and systems using which tools from the toolbox (Fishbone, flow charting, FMEA, etc.)
- Measurement toolbox needs to be developed with QI and Research teams working together across the whole lifecycle of the project (before, during and after)
- Improvement projects initial plans grow and adapt over time, the evaluation approach needs to adapt with it
- Standardize reporting for the planned investigation that honors the context balanced with guidance on reporting DNA communicating easily digestible, maximally understood changes, include time series charts and changes annotated
- Several important factors in designing the improvement and evaluation scheme:
- Who is the customer?
- What are their information needs?
- Available budget
- Timeframe
- Available data
- Add these factors into building a set of credible data to answer your stakeholders needs
- Researchers shouldn’t decide in isolation - Comparison groups:
- Helpful to eliminate Hawthorne effect
- Several activities happening in parallel hard to attribute to which intervention
- By inclusion of comparison groups can tease out that competing effect
- Ethics of control group not acceptable
- Use comparison groups with head to head interventions so communities are not left out of the benefits - Systematic reviews of multiple individual studies important to create evidence around effectiveness of interventions
- Standardize taxonomy of terms for improvement to minimize confusion of audience
- Readiness of the environment for QI that a team can progress thru the 4 stages getting to performing as quickly as possible to maximize improvement results
- How can we accelerate that process? - Rushing an evaluation process can often lead to false negative outcomes as the QI teams need time to form and grow into function teams that can test and implement changes. We must allow adequate time frames for evaluating this type of approach
- Iterative evaluation process works for a dynamic improvement approach
- QI and Eval. Teams work in concert as the Improvement activities proceed the evaluation can adapt (they are often dissonant) - Iterative nature of Improvement needs to be honored in the evaluation approach
- RCTs are not an effective approach for improvement - Positive unintended consequences of improvement efforts need to be predicted and measured
- Should we adapt M&E plans to consider the side effects of changing culture? - When a change is unable to be spread beyond the initial unit, is it a warning sign of this change not being generalizable? Or is it that spread is not well executed with optimal planning?