Guide to Evaluating Behaviourally and Culturally Informed Health Interventions in Complex Settings

"Exploring the value, effect and outcomes of interventions that seek to affect people's health behaviours...is critical. Only this way can we learn and improve, avoid unintended negative effects, and replicate successful interventions." - Katrine Bach Habersaat, Regional Advisor for Behavioural and Cultural Insights at WHO/Europe

Research indicates that individual behaviour and social circumstances together account for 60% of factors determining people's health. The field of behavioural and cultural insights (BCI) seeks to investigate and understand the structural, individual, and cultural factors that affect health behaviours and to use these insights to improve the results of health-related communication, policies, and services. Developed by the Behavioural and Cultural Insights Unit of the World Health Organization (WHO) Regional Office for Europe, this framework proposes a stagewise model for evaluating the effectiveness and sustainability of BCI interventions in complex settings.

The focus on this guide is on evaluating public health interventions that are introduced in a context of continual change and high complexity, which are particularly challenging when the focus is on changing health behaviours and addressing health inequity. For several reasons - including time, complexity, and unintended impacts - evaluating interventions implemented during a public health crisis (e.g., the COVID-19 pandemic) differs substantially from evaluating those conducted during more normal times. Evaluators may need, for example, to reduce the research burden through better design of evaluation methods, including the use of novel rapid qualitative methods and of technological advances such as real-time videoconferencing transcription software.

Featuring 17 accompanying tools, the guide provides detailed information on how to evaluate the effectiveness and sustainability of BCI interventions in such contexts, particularly when the conditions for attaining conclusive proof are difficult or impossible to meet. Instead, using contribution analysis as a starting point (did the intervention play a role in bringing about the observed results?), the guide walks readers through the process of creating an evidence-informed claim for the effect of the intervention. This iterative 6-stage process is summarised by the figure below and described in detail in the guide:

The guide addresses the challenges of assessing causality during times of change and of influencing factors, and it provides a method for measuring the unintended positive and negative effects of interventions on well-being, trust, and social cohesion. These factors were selected because they are drivers of behavioural change, are sensitive enough to reflect the current situation in the population, and can be assessed both quantitatively through surveys and experiments and qualitatively through focus groups and interviews.

The guide was developed, tested, and improved in collaboration with several partners while being applied in Romania, Portugal, and Greece. In that vein, as WHO Europe argues, "The best evaluations are created through an interplay between those who commission an evaluation and those who conduct it and produce the results. Therefore, this document is intended as a tool and reference for both parties."