Evaluating Health Care Collaboratives: The Experience of the Quality Assurance Project

Published by the United States Agency for International Development (USAID)'s Health Care Improvement (HCI) Project, this 89-page report summarises the findings of an evaluation of a strategy adapted and applied by the USAID-funded Quality Assurance Project (QAP) to adapt and apply quality improvement (QI) methods for health care improvement in developing and transitional countries. Specifically, the document explores QAP's work to apply what is characterised here as the collaborative approach to harness and adapt the evidence base for high-quality health care in resource-constrained settings. The particular scope of the evaluation is QAP implementation, during 2003-2007, of 35 collaboratives in 14 developing and middle-income countries.

Led by EnCompass LLC, the evaluation documents and evaluates the implementation and results of QAP-supported collaboratives using a formative, participatory methodology. The evaluation team conducted field visits to 6 countries where QAP supported one or more major collaboratives. Their core research question was: (How) can the collaborative methodology, originally developed by the United States (US)-based Institute for Healthcare Improvement (IHI), be adapted to improve the quality and sustainability of health care in developing and transitional countries?

Guided by the vision that health care can be significantly improved by applying proven QI approaches, the collaborative model addresses key challenges facing global health: expanding coverage with essential services; making services better meet the needs of underserved populations, especially women; improving efficiency and reducing the costs of poor quality; and improving health worker capacity, motivation, and retention. Specifically, the model, as originally developed by IHI in 1994, features the following elements:

1. IHI would announce to the US health care community that it was a starting a collaborative on a particular topic and invite interested health facilities to participate.
2. An evidence base, including gaps between best and current practice, would be developed.
3. Experts would teach participants during each of 3-4 "learning sessions" (2- to 3-day face-to-face meetings) about the evidence base and QI methods (e.g., how to plan, implement, and evaluate small changes in quick succession); participants would report their changes and results, share experiences, and consider how to spread their innovations to other sites.
4. Between learning sessions (during "action periods"), participants at each facility would work as multidisciplinary teams. They would implement QI methods, test changes at their sites, and share their experiences with other teams through conference calls and an extranet website where teams could post their data. Also, teams would set measurable targets and collect data to track their performance as they implemented a set of changes in care.
5. During these action periods, collaborative organisers would provide communication and coaching support, sometimes through site visits, email, conference calls, and websites.
6. The collaborative would often end with a final conference where teams would present their results, share what they had learned, and make plans to sustain and/or spread the improvements to other facilities within their organisation.

As QAP considered adapting the IHI model for collaboratives for developing countries, it had to address several challenges, which are detailed in the report; to cite only one, low levels of technology create communication challenges. QAP thus adapted the model by undertaking the following activities: ensuring ownership of the collaborative process within national structures and not as an independent (i.e., QAP) structure; building local capacity for quality data collection and effective data management; and exploring technology options for communicating and reporting data and for team communications in the absence of widespread internet capacity and technology.

The evaluation team conducted interviews in person and by telephone and email, to probe the experience of all QAP-supported collaboratives. In addition, field visits were made to the following countries: Tanzania (August–September 2006), Uganda (September–October 2006), Nicaragua (October–November 2006), Niger (November–December 2006), Ecuador (January–February 2007), and Russia (March 2007).

An excerpt from the report follows:
"The evaluation found that the collaborative approach as adapted by QAP was robust and feasible in developing country settings. QAP implemented collaboratives in countries at varying levels of development, and yet in these different contexts, collaboratives produced clear gains in compliance with standards and proved to be effective in scaling up best practices across a number of key technical areas: essential obstetric and newborn care, prevention of mother-to-child transmission of HIV, AIDS treatment and care, pediatric hospital care, and pediatric AIDS care. Evidence from Niger, Ecuador, Honduras, Nicaragua, Tanzania, Rwanda, Uganda, and Russia shows that the collaborative approach was highly effective in improving quality of care, generally attaining levels of 80% or higher compliance with standards within 8–18 months of teams working on making improvements. Moreover, the experience...demonstrated that collaboratives can be effective in spreading improvements to large areas of a country or health system. This point was underscored by the evaluation finding that new teams in spread phases of a collaborative achieved results faster than the original teams had - mostly likely because the new teams benefited from a tested change package and the cumulative learning of the initial teams.

...[In addition,] [c]ollaboratives created communities of practice where individual health workers felt empowered to improve care and connected to others and to a greater mission. The early and increasing MOH [Ministry of Health] involvement, both at the central and regional levels, was critical to the success of collaboratives and, ultimately, the sustainability of the improvements introduced.

Several essential features in the adapted QAP collaborative model emerged from this review:

• Well-defined improvement objectives or aims,
• Adequately supported quality improvement teams,
• An explicit implementation package,
• Regular analysis of measured results to guide quality improvement,
• Shared learning for accelerated improvement at greater scale,
• An explicit spread strategy, and
• Organizational structures to support the collaborative and improvement activities.

...The evaluation also suggested ways for making collaboratives more uniformly effective and efficient, including developing sustainable strategies for developing QI skills of health care providers, strengthening local capacity for data collection and analysis, gaining a better understanding of the factors that motivate individuals to participate in collaboratives, and better documenting the improvements made.

The evaluation found that the collaborative approach, as implemented by QAP, is a promising strategy for improving health care quality and strengthening health systems to address national health priority issues at scale. Several questions remain, however, on how to maximize results: How can collaboratives improve data quality and develop better data validation strategies? What strategies can be used other than spread collaboratives to accelerate spread? What additional strategies (in addition to learning sessions and coaching) can be used to strengthen human resource capacity building for supporting quality improvement? The follow-on to QAP, the USAID Health Care Improvement Project [HCI], will continue to explore these questions as it applies the lessons from QAP in support of ongoing and new collaboratives."

Editor's note: The HCI Project team referenced just above includes prime contractor University Research Co., LLC (URC) and subcontractors EnCompass LLC, Family Health International (FHI), Initiatives Inc., Johns Hopkins University Center for Communication Programs (CCP), and Management Systems International (MSI). The 5-year project supports the USAID Global Health Bureau and country missions to address significant challenges in raising the quality of health care in developing and middle-income countries.