Evaluating Communication, Development and Change Programmes

Communications for social change is a process of public and private dialogue and linkages through which people define who they are, what they want, and how they can get it.

Social change is positive change in people's lives - as they themselves define such change. This work seeks, in particular, to improve the lives of the politically and economically marginalised, and is informed by principles of tolerance, self-determination, equity, social justice, and active participation for all.

Behaviour change communication is a process of understanding people's situations and influences, developing messages that respond to the concerns within those situations and using communication processes and media to persuade people to increase their knowledge and change the behaviours and practices which place them at risk.

The above summarises where we are at present - the very positive base from which to move forward.

The Rockefeller group's definition attempts to re-balance strategic approaches to communication and change by taking the overriding emphasis:

• ...away from people as the objects for change...and on to people and communities as the essential components of their own change;
• ...away from designing, testing, and delivering messages...and on to supporting dialogue and debate on the key issues of concern;
• ...away from the didactic conveying of information from technical experts...and on to sensitively placing that information into the dialogue and debate;
• ...away from a focus on individual behaviours...and on to social norms, policies, culture, and a supportive environment;
• away from persuading people to do something...and on to negotiating the best way forward in a partnership process; and
• ...away from technical experts in "outside" agencies dominating and guiding the process...and on to the people most affected by the issues of concern playing a central role.

How Come?

• The scale of the development challenge
• The nature of the development challenge
• Principles for development
• Programming experience
• Evidence
• Strategic observation
• Communication opportunities and trends

Why Measure?

• Accountability to the people engaged in the communication intervention through to funders.
• Progress - impact on the priority issues.
• Improvement - information for strategic and fine-tuning programme decisions.
• Motivation: A sense of achievement is crucial to all involved.
• Credibility - enhance the acknowledged credibility of the work.

Measure what?
The long-term goal: positive change in the issues of concern:

• less poverty;
• higher employment levels;
• better gender equality;
• less HIV/AIDS;
• more girls in school;
• higher educational achievement;
• less child and maternal mortality;
• better nutrition;
• fewer accidents and no violence;
• etc...etc...

These are the crucial measures of success.

Evaluation Challenge [1]:

Time Scale - these changes can take a long time. They will be the subject of overall study. Programme and policy people need more immediate data that indicates the contribution being made. Therefore, the changes require agreement on indicators which:

Evaluation Challenge [2]:

When measured in the short term, we need indicators that:

• indicate a strong likelihood of long-term change in the issue being addressed; and
• are applicable across issues.

Evaluation Challenge [3]:

This happens in many fields:

• increased immunisation levels predict decreased child mortality; and
• increasing numbers of girls in school is often cited as a predictor of economic progress.

In communication, intent to change has been used as predictor of actual change.

Evaluation Challenge [4]:

Communication and social change need a similar set of agreed indicators - both to measure and indicate progress and to drive the nature of the programming. These indicators need to be derived from reliable information and analysis.

1. Indicator 1: Expanded public and private dialogue and debate. Key measurement questions [in relation to the issues of concern]: What increase has there been in:
   
   • Family discussion?
   • Discussion among friends?
   • Discussion in community gatherings?
   • Coverage and discussion in news media?
   • Problem-solving dialogue?
   • Focus and discussion in entertainment media?
   • Debate and dialogue in the political process?

   Example: The approach to HIV/AIDS in Uganda.
   

2. Indicator 2: Increased accuracy of the information that people share in the dialogue/debate. Key measurement questions [in relation to the issues being addressed]
   
   • 5 pieces of data over which there is general consensus; and
   • 4 different perspectives on the issue.

   
   
   

   Test the extent to which these are accurately reflected in the locations for dialogue and debate mentioned above among friends, within the family, etc.
   
   

   Example: The increasingly informed views of the general public on the implications of tobacco use and the strategies of tobacco companies.

3. Indicator 3: Supported the people centrally affected by an issue[s] voicing their perspective in the debate and dialogue. Key measurement questions:
   
   • Which groups in relation to the issue of concern are most disadvantaged?
   • How were they supported to give voice to their perspective?
   • What happened?
4. Indicator 4: Increased leadership role by people disadvantaged by the issues of concern. Key measurement questions:
   
   • Who makes the major decisions concerning the priorities and activities of the communication intervention?
   • How are the people centrally affected by those issues engaged in the decision making process?
   • What are some specific examples where the involvement of that group has influenced strategic or fine-tuning decisions?

   Example: The role of people with HIV/AIDS in significantly affecting the nature of some HIV/AIDS communication initiatives and the absence of that perspective from many other HIV/AIDS communication initiatives.
   
   

5. Indicator 5: Resonates with the major issues of interest to people's everyday interests. Key measurement questions:
   
   • Which were the issues that provided the focus?
   • To what extent were people energised by these issues?
   • What actions followed?

   Example: In the civil rights struggle in the United States:

   
   
   • Buses: Getting on a bus, including to go to school, was an activity to which everyone could relate.
   • The Dream - captured the essence of the struggle.
6. Indicator 6: Linked people and groups with similar interests who might otherwise not be in contact. Key measurement questions:
   
   • Which groups are involved?
   • What are their interests?
   • Have they been linked together?
   • How does that linking take place?
   • Is there an alliance?
   • How does the alliance work?

   Example: The Child Survival and Development Revolution of the 1980s and early 1990s was essentially based on a communication strategy to draw a number of disparate partners into one focused effort.

Additional supportive material:

• Some of the "Route Map" Data: Westoff concluded: There is a persistent and frequently strong association between exposure to the mass media and reproductive behaviour in Africa in the expected direction; greater knowledge and use of contraception, intention to use contraception in the future, preferences for fewer children and intention to stop child bearing. In addition, there is evidence that media exposure is also associated with later age at marriage. These conclusions are generalisable to women and men, both married and single. The research controlled for other factors that would make such a difference: age, rural/urban, socio-economic status, educational achievement, etc. It was the linkage that made the difference.
• Some of the "Route Map" Data [3]: The Jiggasha programme in Bangladesh is a community-level family planning programme based on community, small group, women-to-women, and family dialogue and discussion. From the Johns Hopkins University (JHU) Center for Communications Program (CCP) book on health communication: Over 50% of the participants said that, due to the meetings, they discussed family planning methods with their husbands and other women. Women who attended the meetings were 3.1 times more likely to discuss family planning with other women than non-participants and 1.7 times more likely to discuss family planning with their husbands than non-attendants. About 30% of the women adopted a new contraceptive method, 22% continued using their current method, 17% visited a family planning clinic, and 7% said they switched to a new contraceptive method as a result of the Jiggasha meetings.
• Some of the "Route Map" Data [4]: Twende na Wakati is a radio drama in Tanzania on the broad issues related to sexual health, including HIV/AIDS and sexual health. From the evaluation: 55% of listeners talked to a friend about issues raised in the radio programme, and 23% of the listeners said they adopted family planning as the result of listening to "Twende na Wakati". Most adopted oral contraceptive pills and condoms.
• Some of the "Route Map" Data [2]: In the evaluations of the national-scale Soul City programme in South Africa, a correlation can be drawn between the promotion and support for interpersonal dialogue on an issue and a resulting positive change. From the first evaluation report: 56% of the sample discussed issues raised in Soul City with friends or family. And, as an example of positive change: People accessing two components of the Soul City programme are almost twice as likely to disagree with the statement "AIDS sufferers should be moved away as people not watching the programme" (64% versus 36%).

Communication opportunities and trends: The technological, social and political developments of the 1990s create real possibilities for development communication interventions that meet good development principles and imperatives:


From community radio...

...to more participative political systems...

...to the new technologies...

...to rapid urbanisation...

...to [so many].

The opportunities are immense.