A clear framework is essential to guide monitoring and evaluation. A framework should explain how the programme is supposed to work by laying out the components of the initiative and the order or the steps needed to achieve the desired results. A framework increases understanding of the programme’s goals and objectives, defines the relationships between factors key to implementation, and articulates the internal and external elements that could affect the programme’s success.
Why are monitoring and evaluation frameworks important?
A well thought out monitoring and evaluation framework can assist greatly with thinking through programmatic strategies, objectives and planned activities, and whether they are indeed the most appropriate ones to implement.
Monitoring and evaluation frameworks:
(PATH Monitoring and Evaluation Initiative)
Considerations when developing a monitoring and evaluation framework
Developing a corresponding monitoring and evaluation plan that acts as a monitoring tool by defining how information from the programme will be tracked.
Developing the framework and plan before activities are implemented.
Determining which framework is best to use. A number of different frameworks are used or requested by organizations and donors. Some donors combine aspects of frameworks in a customized approach. Others do not include explicit guidance for programmes around the selection of a framework. Programmes should select the type of framework that best suits their strategies and activities and responds to institutional requirements.
It is also important to keep in mind that:
All programmes should at a minimum conduct monitoring activities that allow them to ensure they are not putting women at greater risk.
(Bott, Guedes and Claramunt, 2004)
Steps in Developing Monitoring and Evaluation Frameworks
Programme implementers (often from diverse sectors) should jointly take steps in developing the monitoring and evaluation framework (UNHCR Guidelines, 2003).
1. They should determine the purposes of the monitoring and evaluation mechanisms and assess the information needs of each actor .
2. Ensure prevention and response interventions have clearly defined objectives, outputs and indicators;
3. Establish coordinated and common reporting tools;
4. Determine methods for obtaining information on indicators;
5. Assign responsibilities for information gathering, determine time frame and frequency of data collection, and allocate resources; and
6. Establish mechanisms for sharing information and incorporating results into prevention and response planning.
(UNHCR Guidelines, 2003)
Though there is no ideal framework and different frameworks are used for different situations, three of the most common are conceptual frameworks, results frameworks and logical frameworks/logic models. (Frankel and Gage, 2007)
i. Conceptual Frameworks
Conceptual frameworks are diagrams that identify and illustrate relationships among relevant organizational, individual and other factors that may influence a programme and the successful achievement of goals and objectives. They Help determine which factors will influence the programme and outline how each of these factors (underlying, cultural, economic socio-political etc.) might relate to and affect the outcomes. They do not form the basis for monitoring and evaluation activities, but can help explain programme results.
Key issues and questions addressed in conceptual frameworks include:
1) What is the theory of change framing the intervention?
The theory of change reflects the underlying process and pathways through which the hoped for change (in knowledge, behaviour, attitudes or practices, at the individual, institutional, community or other level) is expected to occur.
Mobilising Communities to Prevent Domestic Violence: A Resource Guide for Organisations in East and Southern Africa, (Raising Voices, 2003)
Theory of Change: Preventing domestic violence in homes and communities requires individuals to identify the problem of domestic violence, consider its importance, evaluate their own behaviour, and then begin making changes in their lives. Behaviour is a result of our experiences, attitudes, and beliefs, and thus it is deeply linked to the prevailing belief system in the community. The attitudes and actions of neighbours, friends, co-workers, religious leaders, police, health care providers, etc. greatly influence an individual’s behavioural choices and collectively create the climate in the community. Although each individual is unique and will come to the issue of domestic violence differently, the process of how individuals change often follows a similar pattern.
Individual Behaviour Change: The Stages of Change Theory provides a way of understanding the process of how individuals can change their behaviour.
Stage 1 Pre-contemplation: An individual is unaware of the issue and its consequences for her/his life.
Stage 2 Contemplation: An individual begins to wonder if the issue relates to her/his life.
Stage 3 Preparation for Action: An individual gets more information and develops intent to act.
Stage 4 Action: An individual begins to try new and different ways of thinking and behaving.
Stage 5 Maintenance: An individual recognizes the benefits of the behaviour change and maintains it.Facilitating Social Change: The Resource Guide adapts this theory of individual behaviour change and scales it up to the community level. It proposes that a community also goes through a process of change before any given value system is adopted, and projects that recognize this process and operate in harmony with it are more likely to facilitate an enduring change.
(Michau and Naker, 2003)
2) What is the range of potential exposures people may have to the intervention?
• First, who is going to be exposed directly to the intervention? For example, who will be receiving services? Who will be receiving training? Who will be receiving materials? Who will be hearing/seeing/participating in the campaign?
• Second, who might be exposed indirectly to the intervention? For example, who will probably be hearing about the campaign even if they are not the intended primary audience? Who will probably learn that services are being provided in a neighboring community? Who will probably be exposed to some of the ideas disseminated in the training even if they did not directly participate?
3) What is a realistic timeframe for behaviour change to occur?
• Keep in mind that while changes in knowledge may be easy to effect, changes in attitudes and behaviours, especially with community level norms, demand a much longer process.
• Justice sector personnel receiving training about domestic violence legislation or protocols for handling sexual violence cases may be reasonably expected to improve knowledge of the law or the protocol in one session. However, stigmatizing attitudes that serve as barriers to their meaningful implementation cannot be expected to change overnight.
4) How will this change be measured?
• The conceptual framework should identify appropriate measurements for the kind of change that is expected.
• Following on the example above, it would be appropriate to assess a change in the knowledge and attitudes of the justice sector participants by comparing them before and after the training. However, to determine whether the changes in knowledge and attitudes translated into changes in practice over time, a different method would have to be employed, for example, by reviewing court records or by interviewing complainants on their experiences with the handling of their cases.
Illustrative Example from the the Rural AIDS Development Action Research (RADAR) Programme Intervention with Microfinance for AIDS and Gender Equity (IMAGE) in South Africa
IMAGE seeks to influence factors that predispose individuals to HIV infection and gender-based violence through targeting the environment in which they occur. Individual agency, household well-being, communication and power relations, and the norms, networks, relationships and responses of communities constitute the environment in the IMAGE framework. The framework attempts to conceptualise the complexity of factors and relationships that constitute the environment in which sexual behaviour and gender-based violence occurs. The framework was developed to guide both the intervention and evaluation components of the IMAGE programme.
Results frameworks sometimes called strategic frameworks illustrate the direct relationships between the intermediate results of activities all the way to the overall objectives and goals. They show the causal relationship between programme objectives and outline how each of the intermediate results/ outputs and outcomes relates to and facilitate the achievement of each objective, and how objectives relate to each other and the ultimate goal. Results frameworks do form the basis for monitoring and evaluation activities at the objective level.
Logical frameworks or logic models provide a linear, “logical” interpretation of the relationship between inputs, activities, outputs, outcomes and impacts with respect to objectives and goals. They show the causal relationship between inputs, activities, outputs, outcomes and impact vis-à-vis the goals and objectives. Logical frameworks outline the specific inputs needed to carry out the activities/processes to produce specific outputs which will result in specific outcomes and impacts. Logical frameworks do form the basis for monitoring and evaluation activities for all stages of the programme.
Logic models are valuable tools for:
Logical frameworks are presented as diagrams connecting programme inputs to processes, outputs, outcome and impact as they relate to a specific problem or situation. Logic models show what resources the programme will need to accomplish its goals; what the programme will do; and what it hopes to achieve, emphasizing links between these aspects.
A series of “if-then” relationships connect the components of the logic model: if resources are available to the programme, then programme activities can be implemented; if programme activities are implemented successfully, then certain outputs and outcomes can be expected.
The logical framework does not try to account for all of the factors that may influence a programme’s operation and results like a conceptual framework. Instead, the logic framework focuses on the programme’s inputs, activities, and results. This narrow focus assists programme managers and monitoring and evaluation planners as they clarify the direct relationships among elements of particular interest within a specific programme.
(Adapted from Gage and Dunn, 2009 and PATH M&E Initiative)
Example Logical Framework for a Health Provider Training Programme
This logical framework presents a straightforward view of a project designed to improve health providers’ knowledge, attitudes and practices and to increase providers’ awareness of violence against women as a public health problem and a violation of human rights.
[Gage and Dunn, 2009]
Example logical framework framework for understanding the impact of communication, advocacy and training activities on violence against women and HIV (PATH, 2007)
Next Topic Indicators
Previous Topic Situation analyses/Needs assessments (Formative research)