Author Archives: Sarah Osman

31 Oct

How to make a literature review useful for your team

  • 2:13 pm

For most projects, I recommend conducting or commissioning a literature review before anything else. It’s an efficient and cost-effective way to help you focus your intervention design, or plan further formative research if it turns out that that is necessary.

The challenge though is what to do once that 50-page report is ready!

In social and behaviour change (SBC) programming, we want to use evidence to help us guide product, tool or service design. This, in turn, means that we need to bring all our colleagues, who have multi-disciplinary backgrounds, together to develop the programme. A crucial part is getting everyone on the same page about what the literature is telling us.

Here are some tips on how to make the process as fruitful as possible.

Tip 1: Make sure that, as the behavioural science expert, you are involved in writing the literature review terms of reference.

While the team responsible for monitoring and evaluation should take the lead in writing the terms of reference (ToRs) it is imperative that the SBC manager is involved too. The SBC or behavioural science expert’s role is to formulate the questions that should be answered during the literature review. They should be targeted behavioural questions that will help the team understand the drivers and barriers across the socio-ecological model. No vague questions please! A nebulous question would be something like ‘Are contraceptives widely available in Nairobi?’, while a useful literature review question would be ‘What obstacles do 15–19-year-old girls experience when attempting to access a method of contraception in lower income neighbourhoods in Nairobi?’.

Tip 2: Give clear instructions on how to present the literature review findings.

If I see another slide deck that looks like this, my head will explode!

Give the reviewer clear instructions on how you expect the literature review to be communicated. It is best to have a complete report with all references, as well as a slide deck and presentation to be shared with the team. Do yourself a favour and give them examples of what you think a good slide deck should look like.

Tip 3: Use a tool that your team is comfortable with to collate the findings.

If you are lucky, the literature review slide deck will be organised in a way that is ready to use for programme design. Most of the time though, you will still need to undertake another exercise of collating the literature review findings to make them implementable in project design.

The key consideration here is using a tool that your whole team is comfortable with. Think about all the team members who will have to use the tool and consider choosing a tool that is both user-friendly and visual enough to communicate key points to the entire team.

Tip 4: Make the literature review findings compelling for intervention design.

As the SBC manager, it is key to make the research findings directly make sense for intervention design. Once you’ve identified the right tool to communicate with your team, think about what the team needs to get to the next stage. The next steps are usually:

  • Categorising the barriers identified in a way that will help teams assess what is possible to address and what is not.
  • Shortlisting the categorised barriers.
  • Identifying the correct behaviour change tactics.

Assuming that there were no major knowledge gaps found and that there is sufficient data to go from the literature review, the communication tool you choose should be very effective in helping teams move through the process of categorisation, shortlisting and tactic selection.

Tip 5: Make the connection between the literature review and intervention design clear.

I think it is always worthwhile to help team members who do not specialise in behavioural science understand how all three or five stages (depending on the framework you use) of the behavioural intervention design process fit together. As such, it is usually a plus if you can find a tool that helps to tell the whole story:

  • What is the high-level impact you want to achieve?
  • What are the behavioural targets?
  • What does the behavioural journey look like and when are barriers encountered?
  • What are some of the characteristics of the target audience?
  • What type of product, tool or service would be most appropriate?
  • How are we going to assess success?

As SBC experts, to make ourselves useful to various team members, it is important for us to continuously explain how all the pieces fit together and why none of the steps are redundant. Just trust the process!

30 Sep

How does behavioural theory improve intervention design?

Recently, I was confronted with an interesting question: what evidence do we have that programme intervention design is best served by using a theory? Do we have concrete data and evidence that using theory leads to more effective interventions?

In my quest for a decent response, I realised that the question isn’t so much why interventions should be informed by theory, but rather, how they should be informed by theory.

According to Dalgetty et al, (2019) “theory provides a coherent and explicit framework for designing, evaluating, and optimising interventions, a common language to aide communication, a means for accumulating evidence over time, and allows predictions in uncertain or new contexts”. But when assessing the role of theory in intervention design, what criteria should we focus on?

A socio-ecological lens is critical

Dalgetty et al (2019) looked at whether there is evidence for the widely accepted view that theory should be the basis for intervention design. Their meta-analysis found that there was no difference in effectiveness between interventions designed using theory and those that did not. Some of the reasons they offer for this lack of evidence include:

  • The wrong theory is selected
  • The theory is not applied as intended and with fidelity
  • The theory does not specify behaviour change techniques, or those that are given are not applied correctly
  • Theory does not specify Forms of Delivery
  • Theory has limited generalizability
  • Theory has limited (intended) range of application

These reasons show that it is critical for practitioners to have a good grasp of how to interpret and apply theory. But there is another important mediating factor, which is that we may not be analysing the full scope of behavioural determinants for a given behaviour. 

In his argument for why behavioural theory has not delivered the desired results (that is, wide-scale, population level behaviour change), Weed (Hagger & Weed, 2019) states that the main reason for this is the sole focus on individual-level behavioural determinants. Using health behaviour change interventions as the example, he argues that “it is known that poor health outcomes, particularly non-communicable diseases, correlate with social deprivation, low employment, poverty, poor housing, and other indices of multiple deprivation. Behavioural theory provides neither the explanation nor, through interventions targeting individuals, the solution to such problems, which must focus on wider causal systems that underpin the social practice and economy of behaviours such as low physical activity and poor diet.”

This argument is very much in line with the frameworks that I tend to gravitate towards. Two favourites are the Behavioural Drivers Model (Petit, 2019) and the SPACE Framework (Schmidt, 2022). Their multi-theory approach also makes them translatable across thematic areas, as their focus is on uncovering the underlying behavioural drivers at various levels of the social ecology, rather than solely focusing on individual-level behavioural drivers. All good behavioural intervention design frameworks take this same multi-theory, socio-ecological approach (see Intervention Mapping and COM-B as well).

“Theory-inspired” versus “theory-based”

One of the long-standing challenges we have in applying behavioural science in global development is the low level of thoroughly reporting the intervention design process. Some of the questions we should be answering in every project report include:

  1. What was the process of identifying key behaviours?
  2. How was formative research conducted to identify the barriers and facilitators?
  3. How were relevant theories identified?
  4. How were barriers prioritised?
  5. How were behaviour change tactics selected?

A well-documented, theory-based intervention, would, at a minimum, be able to answer these questions. In other words, a lack of implementation fidelity will lead to what Hagger (Hagger & Weed, 2019) describes as “theory-inspired interventions [that] pay ‘lip service’ to behavioural theory but fail to link intervention components with relevant theoretical determinants.”

This tells us that the issue is not so much our inability to prove the value of behavioural theory in intervention design, but rather our inability to sufficiently document how we have applied behavioural theory to begin with when we design development-related interventions.

Theory has a role beyond effectiveness

Those who are wary of using behavioural science to design interventions usually look at the role of theory purely from an effectiveness lens. But theory has at least two other roles to play that are vital to intervention success. First, using theory helps to identify the right change methods that have been shown to achieve results (Bartholomew & Mullen, 2011). Without being able to have a reference point of which tactics work for which drivers (whether they are individual, social or policy related), we will find it increasingly challenging to design more innovative and targeted interventions.  Second, using theory helps to measure and describe the pathways through which change is happening (Bartholomew & Mullen, 2011). For us to be able to replicate successes, we need to be able to detail how we got there.

For a list of global development relevant intervention design frameworks, check out this blog post.

Would you say that your interventions have been theory-inspired or theory-based? Share your experiences with me here!

Bartholomew, L. K., & Mullen, P. D. (2011). Five roles for using theory and evidence in the design and testing of behavior change interventions. Journal of Public Health Dentistry, 71(SUPPL. 1). https://doi.org/10.1111/j.1752-7325.2011.00223.x

Dalgetty, R., Miller, C. B., & Dombrowski, S. U. (2019). Examining the theory-effectiveness hypothesis: A systematic review of systematic reviews. British Journal of Health Psychology, 24(2), 334–356. https://doi.org/10.1111/bjhp.12356

Hagger, M. S., & Weed, M. (2019). DEBATE: Do interventions based on behavioral theory work in the real world? International Journal of Behavioral Nutrition and Physical Activity, 16(1), 1–10. https://doi.org/10.1186/s12966-019-0795-4

Petit, V. (2019). The Behavioural Drivers Model: A Conceptual Framework for Social and Behaviour Change Programming. UNICEF.

Schmidt, R. (2022). A model for choice infrastructure: looking beyond choice architecture in Behavioral Public Policy. Behavioural Public Policy, 1–26. https://doi.org/10.1017/bpp.2021.44

29 Aug

Modelling: theoretical underpinnings and parameters for correct application

Modelling is one of the most popular behaviour change methods used in global development. It is especially popular in programming for young people. The typical application of modelling involves showcasing a relatable role model being reinforced for a desired behaviour. It is especially popular amongst NGOs because of its potential reach and impact. For example, a recent study by Riley (2022) showed that having girls watch the film Queen of Katwe, a 2016 film based on the true story of Phiona Mutesi who took up chess aged nine and went on to compete in international tournaments, led to better performance in exams.

Modelling is an interesting behaviour change method to understand because it seems that the scale of interventions that can be designed using it can make it very cost-effective, for example, due to the number of potential viewers of a film whose behaviour could be influenced. To help NGO practitioners take full advantage of modelling as a behaviour change method, this post provides an overview of its theoretical underpinnings and the conditions that make it work. 

It starts with ‘conditioning’

To understand the underlying mechanisms through which modelling works, we first need to understand operant conditioning. Operant conditioning is “a form of associative learning in which the behavioural act becomes associated with its outcome: either reward or punishment, which will result in more or less frequent occurrence of the behaviour” (Bartholomew et al., 2016).

At its core, behaviour change is about learning. Through operant conditioning we learn which behaviours we should engage in (the ones being positively or negative reinforced) and which ones we should stop doing (the ones being positively or negatively punished).

Vicarious learning

Modelling as a behaviour change method has its theoretical basis in Social Learning Theory (Bandura & Walters, 1977). It works through our ability to learn vicariously, that is, we are able to learn by observing the actions and outcomes of other people. This remarkable human ability was famously demonstrated in Albert Bandura’s Bobo Doll experiments. In this experiment, children in the experimental group saw a role model (an adult that resembled a teacher) who repeatedly hit, kicked and was generally unpleasant to the Bobo Doll. When the children were alone with the Bobo Doll in a room, they behaved just as unpleasantly with it, and re-enacted the same behaviours: kicking and punching the doll. For behaviours to be modelled, four key elements must be in place:

  • Attention: we see the behaviour being conducted and reinforced or punished.
  • Retention: we remember the sequence under which the behaviour must be performed.
  • Production: we can independently replicate the behaviour.
  • Motivation: we associate some positive outcome with the behaviour and want to do it again.

We learn best vicariously when (1) we like the social model being presented to us and (2) when we see that their behaviour is being positively reinforced. In the case of the film Queen of Katwe¸ we see the protagonist Phiona being positively reinforced for her accomplishment in chess. We could then hypothesise that because Phiona’s character is likeable and relatable to Ugandan girls, and that she is being positively reinforced, this will lead Ugandan girls wanting to model this behaviour and taking her example as a potential of what they could achieve.

Types of reinforcement

One of the key reasons that modelling works is that we are re-enacting an experience that the observer is not going through in real life. To encourage retention of how the behaviour is to be performed, we must induce an emotional state in the observer. In the case of Queen of Katwe, the reinforcement is associated with positive emotions. There are various types of positive reinforcement that can be used when designing behaviour change products:

  1. Social reinforcement: the model can get praise from people in their immediate social network.
  2. Vicarious reinforcement: we can see the model being reinforced after completing a specific behaviour.
  3. Self-reinforcement: we can see the model rewarding themselves with a gift or something else that is regarded as pleasant. 

Effective application of modelling

There are some parameters that are essential for the correct application of modelling in behaviour change products (Bartholomew et al., 2016). There are at least five key conditions that need to be fulfilled:

  • The learner has to be able to relate to the model. Although this is an obvious parameter, it can still be easy to get this wrong if the communication material or the intervention is not pretested. It is best to not rest on assumptions but rather make sure that the audience can identify with the model.
  • The model has to present a scenario where they are coping with a situation that did not start off well. It is important that the model exemplifies that there has been a struggle or some level of a learning curve that eventually led to them succeeding in whatever they want to do. The model or the storyline of the model must show that they tried multiple solutions before they finally reached the one that worked. This is important because it makes the model relatable to the person we are trying to influence by exemplifying that it is normal not to do the behaviour correctly right away and that some degree of trial and error can be expected.
  • The observer has to watch the model being reinforced. Reinforcement, as discussed earlier is a key ingredient of why modelling works. Therefore, an intervention that includes a model that is not being reinforced in some way is not going to be effective.
  • The social and physical environment have to be conducive to doing the behaviour. It is well and good to model a behaviour that programme developers think is desirable, but if the physical or social environments are not conductive to doing the behaviour (e.g. a key service is not available) then it is not good practice to be promoting the specific behaviour.
  • The person we are communicating to needs to know how to create a facilitative environment. For modelling to work, the learner or the persons we are designing the intervention for must have a minimum of some of the subskills that are needed to conduct the behaviour correctly.

Have you used modelling recently? Share your experience with me here!

Bandura, A., & Walters, R. H. (1977). Social learning theory (Vol. 1). Prentice Hall: Englewood cliffs.

Bartholomew, L. K., Markham, C., Ruiter, R., Fernandez, M., Kok, G., & G.S., P. (2016). Planning Health Promotion Programs: An Intervention Mapping Approach.

Riley, E. (2022). Role Models in Movies: The Impact of Queen of Katwe on Students’ Educational Attainment. The Review of Economics and Statistics, 1–48. https://doi.org/10.1162/rest_a_01153

30 Jun

Policy reform for social change: lessons from abortion law reform in sub-Saharan Africa

Since 2000, 21 sub-Saharan African countries have reformed their penal codes or passed laws to expand the legal grounds for abortion (Bankole et al, 2020). This is a tremendous achievement which has had and will continue to have an immensely positive impact on African women’s sexual and reproductive health and rights (SRHR).

A conducive policy environment is often a prerequisite for social change. One of the areas where this is highly evident is in SRHR and especially when it comes to access to abortion.

If we apply the socio-ecological model (SEM) to access to abortion services, we find that in several geographies, the most prominent barrier is a restrictive policy environment. When we look closer, we can see that in order for an individual’s access to services to increase, we will also need to work at additional levels (individual, interpersonal, community, health-facility) to enable service access and uptake.

In this piece, I’d like to zoom into how change in abortion laws in sub-Saharan Africa was achieved over the past two decades. There are some important lessons to be learnt on effective strategies that can transform health equity. Here are a few policy change tactics that have contributed to the recent policy reforms (Bankole et al, 2020).

Tactic 1: Using an international policy that countries have ratified.

Undoubtedly, one of the key success factors in abortion law reform in Africa is the Maputo Protocol, which is the African Union’s main legal instrument for protecting women’s rights. It clearly states that Member States shall “protect the reproductive rights of women by authorising medical [here, ‘medical’ means ‘safe’] abortion in cases of sexual assault, rape, incest, and where the continued pregnancy endangers the mental and physical health of the mother or the life of the mother or the foetus” (Bankole et al, 2020). The Maputo Protocol provides a solid legal foundation to demand legal access to abortion, to preserve a woman’s health including her mental health. In recent years, the Maputo Protocol has been used as the basis to expand as well as achieve law reform in several countries, most recently the Democratic Republic of Congo and in Rwanda.

Tactic 2: Timing to coincide with policy windows.

One of the first African countries to have progressive abortion laws was South Africa, which decriminalised abortion in 1996. The tactic employed by advocates at the time was to build coalitions around the formulation of the post-Apartheid constitution to highlight how unsafe abortion impacted Black South African women. This proved to be a very effective tactic and the Termination of Pregnancy Act, with one of the rarest provisions of abortion upon request, is still in force.

Other examples of seizing opportune timing come from Kenya and Somalia. In both countries, reforms to expand abortion laws happened when constitutional amendments were being made. Although the legal scope is not as large as it is in South Africa, it still enabled moving the needle from a total ban on abortion for it to at least be permitted to save a woman’s life.

Tactic 3: Advocacy by government officials.

Government officials taking up advocacy on polarising issues such as abortion is an important tactic in policy reform. This is because their support for issues both legitimises the work being done by civil society actors as well as offers access to support from colleagues who can continue intragovernmental advocacy. In the case of abortion law reform in Mauritius, one of the most influential moments was the speech by the then Minister of Gender Equality, Child Development and Family Welfare, on the health care challenges faced by young women in Mauritius including unsafe abortion.  

Tactic 4: Expanding the interpretation of the law.

Rather than creating new laws, some African countries have reformed interpretations of the current laws (Ethiopia, Ghana and Zambia). In these instances, it comes down to who, for example, should assert that a woman was raped when she seeks as abortion. In many countries, a woman will have to prove that she was raped, which is in and of itself an enormous access barrier. However, countries like Ethiopia have reformed the law to state that a woman’s claim that she was raped is sufficient grounds to grant her an abortion.  This, along with clear and expanded clinical guidelines, decreased unsafe abortion-related morbidity by 10% in just six years.

Social and behaviour change is as important as the existence of legal frameworks.

Bankole et al (2020) note that “law reform alone far from guarantees that women will get the safe procedures that they are legally entitled to. Implementation of a law matters far more than its wording, and many obstacles prevent new laws from changing entrenched behaviours. Failure to inform the public, health care providers and government officials about what a new law entails is a common barrier to implementation.”

It is important to further understand how countries with restrictive abortion laws have managed to change the tide, because while it is not the end all of service access, it is crucial to have a legal framework to ensure that services provided are of high quality and for personnel to be trained to provide the best service. However, if we do not work across the socio-ecological model when it comes to safe abortion access, we will not succeed. The painful reality is that even in countries with progressive laws such as South Africa, women still opt for unsafe providers due to psychological, social and structural barriers.  

Please read the excellent Guttmacher report From Unsafe to Safe Abortion in Sub-Saharan Africa: Slow but Steady Progress.

31 May

Evaluating projects with a behavioural perspective

According to the leading evaluation resource, Better Evaluation, “evaluation, in its broadest sense, refers to any systematic process to judge merit, worth or significance by combining evidence and values”. Just as evaluations are a key part of the process of any behaviour change intervention, they are also key in traditional international development projects and programmes.  Virtually all international development organisations commission (external) evaluations. In theory, these project or programme evaluations are supposed to offer insights into what worked during implementation, for who and under what circumstances. However, often, they are delivered to the commissioning organisation, passed on to the donor, and rarely looked at again.

My suspicion is that one of the main reasons that evaluation reports are rarely used, or not used efficiently, is because they do not ask the right questions. In my (very biased) opinion, evaluations should at a minimum include behaviour change questions. Ideally however, they should start with a behavioural perspective.

The OECD Development Assistance Committee (DAC) evaluation criteria are now standard in almost every evaluation. In 2020, the OECD updated their evaluation criteria to include an additional sixth criterion, coherence. Coherence focuses on how well a project fits in with or adds to other ongoing projects, as well as local policies.

The OECD calls for the ‘thoughtful application’ of the criteria, but if you ask anyone in the sector, they will admit that most of the time, the evaluation questions are copy/pasted without much thought. Part of the reason is that evaluations are seen as a tick-box exercise rather than a valuable part of intervention implementation and future intervention design.

If we look at criteria such as relevance, effectiveness and sustainability, there are often missed opportunities in how these criteria are interpreted. For example, while assessing relevance, evaluators will often look at whether an intervention is relevant to the needs of a community from a technical standpoint. In other words, if community members have a problem generating income, offering them goods to sell as a means of income-generation would be a relevant solution. However, if one was to return some time after the evaluation was completed, they may find that the community members have used the majority of the goods for their own sustenance because that was their main priority at the time. If, on the other hand, a behavioural perspective was first considered, an evaluation would assess the extent to which the intervention was co-designed with the intended audience for a nuanced solution that helps to address their current needs as well as future needs.

Effectiveness is another tricky criterion to evaluate because NGO outcomes are notoriously vague and unspecific, and they leave a lot of room for interpretation. The conclusions on the effectiveness of an intervention end up being vague because the defined objectives themselves are vague. One of the most innovative solutions I came across that addresses the problem of vagueness in the evaluation process comes from E. Jane Davidson and her innovative technique of evaluation rubrics.

A rubric is a framework that sets out criteria and standards for different levels of performance and describes what performance would look like at each level.  Rubrics have often been used in education for grading student work, and in recent years have been applied in evaluation to make transparent the process of synthesising evidence into an overall evaluative judgement (Oakden, 2013).

This adapted approach helps us to dig down on, for example, what has really changed in people’s behaviour as a result of what we introduced. For example, our outcome might read ‘partner organisation builds capacity’, but the question is, what in their behaviour will tell us the extent to which their capacity was built and what values will we put to the varying degrees of these changes.

Sustainability questions also often leave a lot to be desired. A quick Google search shows just how unsustainable many development programmes are. Again, if we ask the right questions, we can do better in the next iteration of an intervention. For example, a hallmark of sustainability should be the extent to which adopting new tools, services, trainings, stipends, or whatever else is offered under the intervention, at a minimum, follows the principles of the EAST framework: Easy, Attractive, Social and Timely.

One way in which many evaluations can be made more informative and user-friendly would be to incorporate behavioural intervention design frameworks as a way of assessing the extent to which an intervention truly made a difference in a community. For the novice in behavioural science, using the socio-ecological model as an entry point to understanding social change will offer a lot of depth and nuance, and move us away from sole descriptive facts to evaluative conclusions that effectively guide better programme design.

On that note, I leave you with evaluation questions that Davidson (2012) recommends for better quality, to-the-point and usable evaluation reports:

  1. Was the programme needed and is it still needed? How well does it address the most important root causes? Is it still the right solution?
  2. How well designed and implemented is the programme?
  3. How valuable are the outcomes for the intended groups?
  4. What works best for whom, under what conditions, why and how?
  5. How worthwhile was it overall? Which parts or aspects of the programme generated the most valuable outcomes for the time, money and effort invested?

Do you have other ideas about how we can make evaluations more useful? Share your thoughts with me here!

 Davidson, E.J. (2012). Actionable evaluation basics: Getting succinct answers to the most important questions. Real Evaluation.  

 Oakden, J. (2013). Why rubrics are useful in evaluations.

28 Apr

High fidelity project implementation: Why it matters and how to monitor it

Imagine one of your partner organisations just launched a financial literacy programme and wants to work with you to get the word out to young people to attend the trainings. You design a Facebook and Instagram campaign that runs for 3 months. Let’s say that at baseline, 50 people were attending the trainings, and after the campaign the same number of people were attending the trainings: 50.

It would be very tempting to conclude that the campaign didn’t work because the number of attendees didn’t increase. But this is not accurate because we don’t know whether the campaign was implemented with fidelity.

Fidelity of implementation is the degree to which an intervention is carried out the way it was designed. Monitoring implementation fidelity, though a crucial part of an intervention, is often ignored. Without monitoring fidelity, we can’t really say with any certainty whether interventions work or not. In the Behavioural Drivers Model, this is referred to as ‘the black box of behaviour change programming’ (Unicef, 2019, pg. 3).

Why monitoring intervention fidelity matters

Continuing with our example above of the social media campaign we designed, consider the four scenarios below. In all four cases, let’s assume that we monitored fidelity.

In the first scenario, fidelity was high and the outcome was also positive. In this case, we can be confident that our intervention, as it is designed, works for the purpose we designed it for.

In the second scenario, we can’t draw conclusions about the effectiveness of our intervention, because we didn’t implement it the way it was designed. For example, maybe we planned to post three times per day on Instagram but did not do that throughout the implementation period.

In the third scenario, we have good reason to believe that our campaign is not effective in getting young people to attend the financial literacy training. It may be because there are factors we did not design for or the modality is not correct. In any case, we have great information that, the campaign, as it is currently designed, does not work and we can draw important lessons as we move forward.

In the final scenario, although the outcome was positive, we don’t know whether we can take any credit for our campaign, because it wasn’t implemented as designed. Perhaps there was another parallel intervention that caused this positive outcome.

As we can see, scenarios 2 and 4 would be particularly troubling. In scenario 2, we may incorrectly draw the conclusion that our campaign doesn’t work at all, and throw it out, meanwhile, it may not have worked in that instance because we didn’t implement it with fidelity. Scenario 4 would also be unfortunate because without measuring fidelity, we would think that our campaign is fantastic and continue to implement it piecemeal, and never really know whether it works or not, or what exact aspects work.

Monitoring fidelity

According to Finney et al (2021), there are five criteria that one should look at when assessing implementation fidelity.

As a starting point, we want to have a theory of change that explains why we think the intervention will work in the way that we designed it: the programme differentiation. Second, we want to monitor how well we adhered to implementing the different features of the intervention. Third, we need to check the standards to which we implemented the intervention and ensure that the quality was high. Fourth, we need to check whether the full dosages of the intervention were administered. And finally, we need to note how responsive our target audience was to the project.

Who should monitor implementation fidelity?

To get an objective perspective on fidelity, it would be good to have multiple check-in points of project implementation. For example, the programmes department responsible for roll-out should have a robust monitoring system that would allow them to track project implementation. It would also be good to have regular feedback loops with the intended audiences on how much of the dosage is reaching them, by, for example asking them how many posts they see in their feed or how often they interact with a component of the project. Finally, it would also be important to verify the five implementation fidelity components by an external evaluator.

In conclusion, implementation fidelity is essentially just as important as implementing the project at all. Without it, we would be left with little information on whether what we are doing works and to what extent.

Has implementation fidelity impacted a project you worked on recently? Share your thoughts with me here!

References

Finney, S. J., Wells, J.B. & Henning, G. W. (2021) The Need for Program Theory and Implementation Fidelity in Assessment Practice and Standards. National Institute for Learning Outcomes Assessment

Kok, G. (2014). Practical Guideline to Effective Behavioral Change. The European Health Psychologist, 16(5), 156–170.

UNICEF. (2019). The Behavioural Drivers Model.

31 Mar

Debunking and Pre-bunking: A brief overview of the application of Inoculation Theory to combat misinformation

In 1964, social psychologist William McGuire wondered how resistance to persuasive messages can be built. Motivated by the idea of recovering brainwashed American prisoners of war, he came up with the analogy of looking at a persuasive message as an infectious disease. Inoculation Theory postulates that attitudes can be inoculated against persuasive attacks by building the ‘cognitive immune system’, the same way that one can build biological immunity. Similarly to a biological vaccination, this psychological vaccination can be brought about by exposing individuals to a persuasive message that contains weakened versions of the argument.

Recently, work by Compton et al (2016) and van der Linden (2022) has highlighted the mounting evidence for the design of messages to counter misinformation using Inoculation Theory. One of the many exciting features of Inoculation Theory is that it can be used to both treat misinformed individuals (debunk) as well as psychologically vaccinate against future misinformation (pre-bunk).

This piece summarises the key tenets of the application of Inoculation Theory to treat against as well as prevent infection with misinformation.

Designing debunking messages

Research by van der Linden (2022) has shown that one of the major determining factors to whether a debunk message is effective or not is the quality of its design. To this end, he offers four steps to designing a successful debunk message.

Lead with the facts

Misinformation spreads because it is repeated. The repetition of information is effective for two reasons. First, people think that information that they encounter repeatedly is true, also known as the illusory truth effect. Secondly, information that is repeated becomes more available in our short-term memory, also known as processing fluency. Therefore, the first step towards designing an effective debunk message is to lead with the facts. The information needs to be simple and sticky, while maintaining accuracy. What will make the information sticky will of course depend on the target audience. The key thing is to lead with the correct information, and not the false information.  For example, a factual message could be “the IUD is a safe and convenient contraceptive method preferred by hundreds of thousands of African women.”

Warn about the myth

Following the inoculation analogy, for people to build cognitive immunity against false information, they need to be exposed to a safe dose of false information. At the second stage, one needs to present the target audience with a warning of the type of false information that they may encounter, along with a low dose of the message that should only be presented once. For example, the low-dose false message could be “the IUD is not made for African women.”

Expose the manipulation technique

It is then important to explain to people why this myth is not true and the way in which the information has been designed to mislead them. The motivation of the communicator should be clearly explained as well as how they used their words to try to trick the audience into believing the false information. In our example, the culprit could be exposed as trying to elicit in-group/out-group emotions amongst African women by making them falsely believe that IUDs are not for them, while there is ample scientific evidence on its perfect safety for African women and that in fact hundreds of thousands of African women already use one.

Present the fact again

The message should end with presenting the fact again to reinforce it. It should also include a credible alternative explanation. For our IUD example, we could include a video by an African WHO-affiliated doctor who explains the safety of the IUD along with an interview with an African woman who uses one and vouches for how happy she is with it.

Designing pre-bunking messages

The pathway to prophylactic messages is slightly different in that one starts with variations of the false message. One type of messaging triggers a response to a threat, while another offers a low dose of falsehood, or a pre-bunk.

Forewarning of impending misinformation

Psychological inoculation can be triggered through a motivation threat response, where one attempts to defend themselves against the threat of the false information. Going back to our IUD example, we could present women with a message such as “there is an intentional false message being spread in your social media network about IUDs” and adding that this is purely fictional and a source of proof that it is incorrect information.

Refutational pre-emption

The second way is to present the pre-bunk message in the form of a low dose of the false information. This could be same type of message used in debunking where we warn about the myth, e.g. “the IUD is not made for African women”. This is then followed by truthful refutations.

The application of Inoculation Theory to message design is not only highly effective, but it also has some remarkable additional features (Compton et al., 2016) that make it feasible and efficient for use in international development:

  • Psychological vaccination against one type of misinformation has been shown to provide umbrella protection from other types of misinformation. It is as though cognitive pathways are trained to resist misinformation in general.
  • Inoculated individuals engage in post-inoculation talk, which leads to the spread of the correct information, and when enough people are psychologically vaccinated, it can lead to a kind of herd immunity against misinformation.
  • Treatment messages can be delivered quickly and early to prevent known threats. For example, this could be exceptionally useful for areas such as promoting information for soon-to-be parents to improve child protection.
  • Interventions can be designed to promote the active generation of cognitive antibodies to misinformation through games that give players the chance to get into the mind of a misinformation designer.
  • Messages can be delivered with brief or little interpersonal interaction, which means that they can effectively be used in mass media campaigns sustainably and cost-effectively.

Have you used Inoculation Theory in your work? Share your insights with me here!

Compton, J., Jackson, B., & Dimmock, J. A. (2016). Persuading others to avoid persuasion: Inoculation theory and resistant health attitudes. Frontiers in Psychology, 7(February), 1–9. 

van der Linden, S. (2022). Misinformation: susceptibility, spread, and interventions to immunize the public. Nature Medicine, 28(3), 460–467. 

28 Feb

Characteristics of impactful information campaigns

I recently came across an incredible article from Abdul Latif Jameel Poverty Action Lab (J-PAL) that summarised their review of what designing an effective information campaign looks like. In social and behaviour change, we are tempted to design complicated, multi-layered interventions. But can information campaigns work too?

Information is a fundamental part of behaviour change. However, the way that this information is conveyed is crucial. The format, dosage, framing, structure, and delivery are amongst many factors that influence the impact of the piece of information being delivered. Often, we think that campaigns have been effective because they increased knowledge, but most of the time, they have changed other important behavioural determinants as well and that is why they were successful. To illustrate this, I will share two examples from the J-PAL review that used information as a medium and that were successful in changing behaviour.

Sugar Daddies

According to the Health Belief Model, risk perception (perceived susceptibility) is one of the key determinants of whether one takes on a protective behaviour or not. Risk information therefore needs to be carefully shared to increase perceived susceptibility. A study by Dupas (2011) showed how an HIV risk information campaign reduced unsafe sex with older men amongst Kenyan teen girls by up to 61%. Given that unsafe sex with older men is one of the key drivers of HIV infection rates amongst adolescent girls in Africa, this result is remarkable. So, what characteristics of this information campaign made it so successful?

  1. “Relative Risk Information Campaign”. Although the campaign was designed to disseminate information (increase knowledge) it also had a specific focus on ‘relative risk’. The information was presented using examples from a nearby city that teens could relate to.
  2. Presenting data in a familiar scenario. Because the information was presented using statistics of people that the target audiences could relate to, they managed to build a mental risk model of how this could play out in their own lives and how they could protect themselves.
  3. Use of videos. A 10-minute video was shared with the target audiences that contributed to grabbing attention through modelling.
  4. Group discussions. Group discussions after watching the videos likely contributed to reinforcing knowledge as well as impacting attitudes towards older men who have sex with younger women.

Increasing insurance coverage

According to the Extended Parallel Process Model, people will go through a number of stages when processing a preventative health message. First, they will assess how big the threat is and how susceptible they are to it. Second, they will assess whether there is something that they can do about it and whether that thing is available or accessible. If either of these steps fails, the message will be rejected.

In France, low-income youth are unlikely to use primary health care and have a higher prevalence of physical and mental health conditions than wealthier youth. One of the barriers to their access of the health system is the fact that they are not registered in the universal health insurance system. Crépon and Pernaudet (2019) designed and tested an information intervention that not only got low-income youth to enrol in the health insurance scheme, but one of the intervention arms doubled the number of those who saw a psychologist afterwards. What worked?

  1. Making things easy. The intervention was offered at the same place where young people came for job seeking support. They could see a social worker (one arm of the intervention) or a social a worker and a doctor (second and more successful arm of the intervention) at the same place.
  2. Tailored information dissemination. The information sessions were offered one-on-one, which meant that clients could get information tailored to their specific situation and needs.
  3. Personalisation. The social workers and doctors were able to stay in touch with each client over a one-year period, keeping the opportunity for personal communication open.
  4. Credibility. The fact that the information about seeking health services came from a credible source (a doctor) probably increased the persuasiveness of the message.

What is interesting about these two examples is how impactful a well-designed information campaign can be. At the surface, it may seem like all that was done was, well, information dissemination, but in fact, both interventions had elements that targeted several behavioural determinants and that is most likely why they were effective.

J-PAL (2021) conclude with some key characteristics of successful information campaigns.

  • Pairing specific information with actionable steps encourages take-up of healthy behaviours.
  • Choosing effective messengers for the local context is critical for building confidence and trust to encourage behaviour change.
  • Peers and other members of a social network are often powerful means of spreading information about health products and behaviours.
  • Using technology, including entertainment media, SMS messages and social media such as Twitter and WhatsApp, can amplify the reach of messages.
  • When the perceived cost of behaviour change is high due either to financial or nonfinancial costs, information alone is often ineffective at changing behaviour.

Have you come across some well-designed information campaigns? Share them with me here!

References

Abdul Latif Jameel Poverty Action Lab (J-PAL). (2021). “Designing information campaigns to increase adoption of healthy behaviors.” J-PAL Policy Insights. Last modified August 2021. https://doi.org/10.31485/pi.0805.2021.

Crépon, B. and Pernaudet, J. (2019). “Correcting Beliefs to Increase Health Investments: A Field Experiment Among Disadvantaged Youths.” Working Paper, October 2019

Dupas, P. (2011). “Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya.” American Economic Journal: Applied Economics 3: 1–34.

Maloney, E.K., Lapinski, M.K., Witte K. (2011). “Fear Appeals and Persuasion: A Review and Update of the Extended Parallel Process Model: Fear Appeals and Persuasion”. Social and Personality Psychology Compass. 5 (4): 206–219.

Skinner, C. S., Tiro, J., & Champion, V. L. (2015). The Health Belief Model. In K. Glanz, B. Rimer & K. Viswanath (Eds.), Health behavior: Theory, research, and practice (5th ed., pp. 131–167). San Francisco, CA, US: Jossey-Bass.

30 Jan

Useful behavioural mindsets

Success in applying behavioural methods to projects requires a bit more than following specific guidelines prescribed in books. At the heart of an impactful social and behaviour change project is the way in which the project team and the organisation view and think about what they are doing. With a new year ahead of us, here are five behavioural mindsets to consider.

Be open to considering another discipline’s approach to the issue.

To start, it is useful to step aside from your area of expertise and consider the problem you are trying to address from the perspective of other disciplines such as social psychology, behavioural economics or anthropology, some of the disciplines that inform behavioural science. At the centre of all projects is people needing to change their behaviour, and training in your field of expertise may not offer sufficient tools to deal with human behaviour. For example, if you are managing an agricultural project to improve farm yields, distributing improved seed varieties alone may not be enough to get farmers to use them. Looking at the issue from a behavioural lens will help to identify factors that may impact how the project runs.

Be open to the fact that scientific rigour can lead to creative solutions.

It may sound counterintuitive, but you will discover that the process of scientifically analysing an issue will lead to creative solutions. The behavioural design process of identifying key behaviours, sub-behaviours and behavioural barriers can seem tedious and over-the-top. But if you see the process through and match the correct behaviour change tactics to the barriers you identified, often, you will find completely new approaches that you haven’t considered before. For example, we may discover that we have been approaching an issue from a very high-level place, e.g. “let’s change how a community views contraception” rather than what is actually happening in the experience of contraception use, e.g. “how can we give young women the skills to negotiate condom use when they need to?”. These two approaches will naturally lead to completely different interventions.

Be open to failure because it is great evidence.

In development work, the stakes are very high because we are dealing with people’s lives. The heightened stakes make us very fearful of making any kind of mistake. We translate this by designing large-scale projects with enormous outcome statements, and the ostrich effect comes into action causing us to avoid the possibility of getting information on whether the intervention is actually working. A behavioural approach to programme design would allow us to prototype a small number of potential solutions at a small scale first and scale those that seem promising. Some of the solutions will fail, but that’s great because we got some very useful information. Fostering an experimental mindset in an organisation is a great investment.

Be ready to overhaul and rethink your strategic plan.

You know that five-year strategic plan your team gave input on two years ago? How many times have you revisited it since it was published? The interventions that you are designing will have ripples and effects, some negative and some positive. Life is happening and things are constantly changing, so your strategic plan needs to be malleable and adaptable. The Asia Foundation has a fabulous resource on how to revisit your strategy, called Strategy Testing.

Be communicative about your projects, your approach and your results.

As we all know, one of the best ways to learn is exchanging ideas with others about what we did and what happened. One of the things that often surprises me is how little organisations know about what their colleagues are doing in other countries, but even in other departments. The best way to further test an innovation is to try it somewhere else in the organisation that has similar conditions. Fostering an environment of knowledge exchange internally is vital. Often, there is also latent knowledge and expertise across teams that can help you further design your project. Sharing project processes and outcomes externally is just as crucial. A great example of this is how the global Johns Hopkins projects Breakthrough RESEARCH and Breakthrough ACTION have created a mega resource for learning across disciplines and countries on how to apply behavioural science in global development.

For more, I highly recommend the article by Naru & Laffan (2021) on their experience fostering a behavioural mindset in the OECD, which can be found in the 2021 Behavioural Economics Guide.

Have you recently adopted any of these mindsets? What has your experience been? Share your thoughts with me here!

22 Dec

Behaviour change tactics in action: real world examples

Amongst the most critical steps in the behaviour change intervention design process are (1) choosing a behaviour change tactic that has evidence of changing the behavioural determinant that you identified and (2) that you use that behaviour change tactic bearing in mind the implementation parameters for that tactic, that is, understanding the conditions under which the behaviour change tactic will be effective (Kok et al., 2016).

In this article, I share five examples of the operationalisation or practical implementation of some behaviour change tactics (aka ‘methods of behaviour change’, ‘behaviour change techniques’ or ‘behaviour change principles’) in development projects.

Behavioural journalism

Behavioural journalism is an effective technique in changing social norms about topics that may be regarded as taboo. It uses mass media to share stories of behaviour change based on authentic interviews with the target group.

A great example of this is the youth sexual and reproductive health project Merci Mon Héros. The format centres around young people sharing stories of times they faced difficulties related to sex, contraception or relationships. They share stories of how they got help from an older person in the community: a parent, extended family member or community member. Both tell their side of the story and detail how they made the change they needed to make. The videos end with a call to action to use contraception and learn more about sexual and reproductive health. You can read about the preliminary impact of the project here.

Scenario-based risk information

A key factor in people’s decision to take up a preventive or curative heath service is that their risk perception, or the perception that they are at risk of experiencing negative consequences due to inaction, must be high. However, risk perception must be high in conjunction with high self-efficacy and response efficacy. That is, it needs to be clear what people can do to avoid the harmful outcome and hold a belief that the solution presented will indeed be useful in mitigating the harmful outcome.

A good example of how risk perception can be carefully increased is the use of scenario-based risk information. This tactic allows people to create a mental picture of how a future harmful incident can occur by allowing them to create their own mental scenarios. This can be done verbally or in written form, through the description of specific scenarios, or through vignettes. It can also be done using a combination of visual  cues and text, as in the example below of a visual aide created to help people gauge the level of risk they are taking in contracting Covid-19 when they engage in specific activities.

Normative feedback

A tactic that can work well in changing behaviour at the individual or group level by increasing motivation is normative feedback. It works by presenting information about what others who are similar to the target audience are doing. In this sense, it also works by influencing perceptions about descriptive norms (what others like us do).

Below is an example of how MSI Reproductive Choices, tested normative feedback in their clinics in Nepal. Depending on the current clinic status, the posters would suggest ways to improve or commend high-performing centres. The aim was to increase nurse’s counselling and offering of post-abortion family planning (PAFP) and inform women of the choice to get a long-acting reversible contraceptive (LARC) such as an intra-uterine device or implant. Every month, clinics got feedback on how they performed in comparison with three other similar clinics. Green indicated good performance and red indicated poor performance. You can read about the impact of the intervention here.

Self-revaluation

Attitudes have strong underlying belief components influenced by perceptions of the self as well as the community that one lives in. Programmes that want to change attitudes towards products or services must address attitudes at the individual level as well as a person’s experiences externally.

An effective approach to changing attitudes is self-revaluation. It works by allowing individuals to reflect rationally and emotionally on their self-image and their lives with or without the service or product being promoted. Since it often leads to awareness-raising as well, it is important that the process is guided by a person that can offer direct access to the service or product.

Another great example comes from MSI Reproductive Choices in Pakistan. The Improving Reproductive Health Through Awareness, Decision and Action (IRADA) project was built around three steps, the first of which used self-revaluation. It included facilitated “photo discussions to promote self-reflection on critical life-choices, using a timeline to help women understand their life-course and daily routines; drawing body maps to understand their own bodies and role of contraception; and social maps to identify environmental barriers to women’s mobility.” Full details are available here.

Developing new social network linkages

Social networks are often critical to the uptake of services and products. When the usual tactics are not working or when a new product or service needs to be introduced, creating new social networks is often crucial.

A fantastic example of how this can be done is a tuberculosis (TB) case detection project in Nigeria. It developed new networks to increase TB testing through:

  • Chemists or medicine-dispensing stores, which gave TB testing referral cards to clients who they suspected of having TB.
  • Religious leaders who disseminated messages on how to spot TB symptoms to get tested.
  • Community members, using social support messaging, to recognise symptoms and get tested.

You can watch the full video here.

Do you have examples of behaviour change tactics? Share them with me here!

Breakthrough RESEARCH (2021). Findings from the Merci Mon Héros social listening endline report. Population Council, Washington DC.

Kok, G., Gottlieb, N. H., Peters, G. J. Y., Mullen, P. D., Parcel, G. S., Ruiter, R. A. C., Fernández, M. E., Markham, C., & Bartholomew, L. K. (2016). A taxonomy of behaviour change methods: an Intervention Mapping approach. Health Psychology Review, 10(3), 297–312. https://doi.org/10.1080/17437199.2015.1077155

MSI Reproductive Choices (2018). Evidence and Insights.