Something Old, Something New, Something Borrowed & Something Blue: A Review of Social Norms Change Monitoring & Evaluation for the Anti-Corruption M&E Professional

"It is a not inconsequential challenge to develop processes to assess the unique dynamics of social norms that fit within the financial constraints and existing competencies of integrity and anti-corruption agencies."

International and national anti-corruption non-governmental organisations (NGOs) are increasingly paying attention to social norms as a driver of corrupt patterns of behaviour. The Corruption, Justice and Legitimacy Program (CJL) is a research-to-practice initiative committed to improving the effectiveness of anti-corruption programming in contexts of endemic corruption. One of CJL's priorities is the nexus of corruption and social norms. This CJL paper offers evaluators and monitoring and evaluation (M&E) specialists a review of existing social norms change assessment practices from other fields filtered through the lens of utility to anti-corruption programming. It reviews common signs of shifts in norms, adapts them to the anti-corruption context, provides tailored examples to illustrate the concepts, and explores additional elements of concern.

As the authors explain, social norms' transient, multi-dimensional nature poses a challenge to traditional M&E, which typically seeks to assess changes that are singular in dimension. For instance, a knowledge change assesses retention of new information, an attitude change looks for differences in a belief, and a behaviour change assesses new actions. Social norms, on the other hand, are an amalgamation of factors. Such factors include beliefs about what others typically do and think is the right thing to do, which are supplemented by beliefs about what others you care about will do if you behave differently. Further, social norms are situational, as they do not transcend contexts like morals or values but, rather, are triggered by situations and people.

To look for guidance, CJL began with a literature review that led to the identification of 55 publications from a range of sectors, most notably gender empowerment and public health, including female genital mutilation. (The full list of sources can be found in the report's Bibliography.) The authors lay out the insights they find most useful to apply to anti-corruption work and articulate where they think modifications are necessary or limitations should be considered.

For example, from the Social Norms Learning Collaborative's work (e.g., "Monitoring Shifts in Social Norms", at Related Summaries, below), CJL suggests five signs one could track to assess social norm shifts that have potential application for anti-corruption programming. As a hypothetical example, they consider a programme that sought to change the following social norm within a government ministry: Civil servants are expected by their peers to avoid taking any action (regardless of the rules or standard operating procedures) that could get another civil servant into trouble. If a civil servant contributes to difficulties for a colleague, their peers may view them as untrustworthy and exclude them from professional processes (e.g., meetings). CJF suggests that one would select signs to be tracked based on how the programme seeks to change the social norm. If the strategy is to negate the consequences or likelihood of a social sanction, then tracking any changes in the amount of backlash would be essential. Conversely, if diffusion beyond the initial programmatic participant to the wider group is central to the strategy, then monitoring active promotion would be a minimum threshold.

While limited, programme monitoring literature describes qualitative data as the go-to data collection means for monitoring. CJL notes that individual interviews, staff journaling, participant feedback forms, and observation were cited in the literature as useful data collection methods. With the appropriate modifications taken into account to protect participants and ensure honesty of information shared, these methods could be valuable in an anti-corruption context, CJL asserts. Staff could be provided with training on identifying social norms and distinguishing them from related concepts (e.g., attitudes), as well as given structured documentation mechanisms for their observations (e.g., template). Negative verbal or physical reactions that suggested anger, outrage, shock, or incredulity or positive ones such as affirmation or contentment could be part of the observation checklist.

CJL stresses the need to monitor for harm in the form of backlash. A "successful" social norm change programme encourages individuals to behave differently than what is expected, meaning they will possibly experience "punishments" by the group in return. It is important to monitor for the presence of backlash to make sure mitigation strategies are working. For instance, in CJL's Kuleta Haki anti-corruption programme with the justice sector in the Democratic Republic of the Congo, participants stated that the more they spoke out against corruption or in favour of following the rules with their colleagues, the more their peers actively mocked them. Given the power and resources that may be involved in the corruption being targeted, sensitivity may be needed to be given to confidentiality and individuals' willingness to "report" the behaviour of others.

Substantially more work has been done on the measurement of social norms change than programmatic monitoring; Section VI of the report outlines insights from that literature that are applicable to the anti-corruption context, including:

• Conducting a broad assessment followed by a diagnostic can double as a baseline.
• Assessing social norms change requires data on all social norm components; CARE's Social Norms Analysis Plot (SNAP) framework is one relevant tool (see Table 4 on page 16).
• Determining changes in the strength of the norm is a promising approach to ascertain progress.
• Methods-centric evaluation does measure social norms change but misses out on valuable programme evaluation contributions.
• Different types of survey questions are useful to assess corruption norms.
• Vignettes hold promise for qualitative data collection amongst citizens.

The report also outlines M&E that is unsuitable for social norms that drive corrupt practices. To cite only one example, the majority of tools reviewed were dedicated to assessing shifts in social or gender norms, rather than assessing a social norm change as one change pathway within a multi-faceted theory of action. CJL contends that "social norms are not the magic bullet to stopping corruption, but rather a key component that will need to be programmed into a multi-faceted theory of change....As such, any evaluation effort would need to reflect the different change pathways sought and ideally seek to explore how they relate to each other."

Finally, the report puts forth questions and research areas that CJL hopes will advance the M&E of social norms change agenda in the anti-corruption space. One question would be: Are there proxies that can be used to assess social norm change related to corrupt practices? One potential research area would be: the common intersections between gender and corruption-driving norms to offer some insights for where M&E should focus its attention.