M&E Sourcebook: Case Studies

Appropriate indicators

1. CARE International, Risk Management for Local Sustainable Development 2001-2003

   Indicators were identified at the beginning of the project to measure performance in terms of three objectives:

   Overall objective: Increase in documented vulnerability-reduction measures

   • Indicator: Municipalities have identifiable vulnerability-reduction measures among the eleven focal municipalities.
   • Indicator: 50% of participating key organizations have adopted the risk management approach.

   Objective 1: Key organisations have improved emergency response and risk reduction capabilities and adopted an integrated risk management approach.

   • Indicator: 4 staff from each regional CARE mission trained in risk management and disaster response as part of regional and national technical units.
   • Indicator: 1800 people sensitized and trained in workshop process; 24 trainers trained; 59 workshops carried out.
   • Indicator: 11 local risk management systems consolidated.

   Objective 2: Risk management concepts and practices incorporated into local development plans, projects and programs.

   • Indicator: 11 municipalities to incorporate risk management concepts and practices.
   • Indicator: Communities in 11 municipalities carried out significant, socially prioritized risk management strategies, risk reduction activities.
   • Indicator: 11 municipalities incorporated improvements in disaster response preparations.
   • Indicator: # of events to disseminate experiences & best practices from project.
   • Indicator: At least 2 schools per target municipality incorporated risk management elements into formal or non-formal programs.

2. BRCS/IFRC/DfID Disaster Reduction Programme, 2001-2003

   Three areas of focus were identified in the TOR for evaluation (impact, efficiency and sustainability) but no indicators were developed to measure progress towards these. Although the log-frames designed at the beginning of the programme identified quantitative indicators, outputs and purposes were often not that clear so this framework was largely ignored in the evaluation. The evaluation team decided to group results within the Well-Prepared National Society (WPNS), an IFRC framework for assessing disaster preparedness and reduction within the Red Cross. This framework has six key features by which the programme’s success was measured: relevance/ assessments; DP policy and planning; structures and organisations; human resources; material and financial resources; and advocacy and effectiveness.

   Overall, the programme lacked baseline data. For example, information on the historical disaster record, including dates and indicators of impact on past disasters in the areas assisted was either not available or not collected. Suitable indicators which could have been collected from secondary sources and through primary data collection include:

   • Impact on lives: numbers killed, wounded and displaced during disaster
   • Impact on livelihoods: e.g. acres of crops destroyed, numbers of livestock killed, days before key livelihood activities recommence
   • Red Cross response time: time before initiating assessment and response, appropriateness, value for money etc.

3. CAMI/ARC Mitigation Grant for Risk Management and Community Preparedness, 2001-2003

   The goal of the project was to decrease excess morbidity and mortality caused by natural disasters through increasing the capacity of four Red Cross National Societies to monitor and respond to such disasters. Evaluation indicators were identified in the log-frame in order to measure progress towards three main objectives (outcomes of the project) and related outputs.

   See table of objectives and indicators.

   The overall design of the baseline and final evaluation of project outputs and outcomes was a "before and after" study within the CAMI Project Area. Data was collected at the start of the project through household surveys, simulations (drills) and direct observation. Baseline results enabled the CAMI project to establish a) what was already working and b) what could be improved in Red Cross, school and household preparedness and response. Monitoring and evaluation data were collected on a quarterly basis (Q) and at the end of the project (B).