M&E Sourcebook: Case Studies - Examples

Joint Evaluation of the BRCS/IFRC/DfID Disaster Reduction Programme 2001-2003:

See the project evaluation summary report , March 2004, [pdf, 0.6MB]

1. Type of project/programme evaluated

   The programme's main activities were non-structural: at global and regional levels the focus was on organisational learning and development; at the national and local level main activities included assessments (VCA), planning and training. In one local district the programme also supported a food security project with structural DRR components. The programme focused on enhancing local disaster preparedness capacities in seven disaster-prone National Societies in South Asia (India, Nepal, Bangladesh and Pakistan) and East Africa (Rwanda, Sudan and Ethiopia). Activities were implemented through Red Cross and Red Crescent National Society structures at the Headquarters and Branch level, the aim being to leave behind sustainable national and community-based structures.

2. Evaluation method(s) applied

   The evaluation was asked to look at the successes of the programme in meeting its stated objectives, the technical quality of programme interventions, the effectiveness of implementation arrangements and management and the continuity or "sustainability" of interventions. It was carried out one year after the programme ended thus providing an opportunity to ascertain longer-term impact/sustainability of the programme. In general, a qualitative approach was adopted, although the log-frames designed at the beginning of the programme were mostly based upon quantitative indicators. Most of the secondary sources reviewed contained qualitative information, with the exception of the questionnaires which were designed to collect quantitative information on National Society capacity. The findings of the evaluation were based on a desk-top review of existing documentation including planning documents, progress reports, workshop reports, policies and strategy documents and some of the written outputs of the programme (case studies, plans and policies).

   In the absence of clear indicators to measure against logframe objectives the evaluation team grouped programme activities and outputs within the Well-Prepared National Society (WPNS), an IFRC framework for assessing disaster preparedness and reduction within the Red Cross/Red Crescent. This framework has six key features: relevance/ assessments; DP policy and planning; structures and organisations; human resources; material and financial resources; and advocacy and effectiveness. Programme outputs were reviewed against the original plans and objectives. However, evaluators conceded that these objectives were over ambitious and the timeframe unrealistic. As a result, the overall success of the programme appears to be only moderate.

   In addition, an attempt to measure programme impact on disaster response was made. Monitoring of the response efforts of the Assam State branch in India to floods and a tornado during and after the end of the programme showed response capacity had increased. However, this assessment suffers from a lack of baseline data. As the evaluators point out: "Information on the historical disaster record, including such information as dates and indicators of impact on past disasters in the areas assisted was either not available or not collected."