M&E Sourcebook: Case Studies - Approaches

Methodological approaches

1. CARE International, Risk Management for Local Sustainable Development 2001-2003

   The overall approach was mixed but with a strong qualitative and, in particular, participatory component, which was well suited to the type of project being reviewed – a capacity building project focusing on improvement of local capabilities and strengthening links between local communities and national disaster response systems. The evaluation of impact was conducted in a participatory manner where possible in not only the methodological design, but also in implementation and, most importantly, in the analysis. The participatory assessment was designed primarily to “gauge the amount and nature of knowledge beneficiaries gained through participation in the project and the degree to which they were able to share that knowledge with other members of target communities.” A quantitative approach was adopted to evaluate outputs such as the number of direct beneficiaries who received training. This analysis was useful as it revealed problems in project processes: not all offices used the same methodology to count beneficiaries and reporting was not very punctual. A qualitative analysis of institutional development and learning was carried out, based on semi-structured interviews with personnel and consultants of CARE as well an analysis of project documentation. This activity also produced an increased awareness at the country management office level and in some CARE programs of the need to incorporate risk management in CARE programming.

2. BRCS/IFRC/DfID Disaster Reduction Programme, 2001-2003

   The evaluation adopted a mixed methodology involving the collection and analysis of both quantitative and qualitative output data, an approach which was well suited to the programme's objectives - to enhance disaster preparedness capacity in seven Red Cross/Red Crescent National Societies that would result in sustainable systems and more effective institutional structures.

   On balance, the approach was more qualitative with a strong emphasis on reviewing programme documentation and interviewing Red Cross/Red Crescent staff and government officials in each country. Analysis focused mainly on measuring progress towards achieving programme activities and outputs, such as carrying out VCAs and producing hazard risk maps, disaster preparedness policies and plans, training materials and operational guidelines. In addition, a questionnaire was applied in all participating National Society's to measure capacity in disaster preparedness. These self-assessments could be used as a benchmark for monitoring progress in disaster preparedness.

   A participatory SWOT analysis of the programme’s processes and outputs was carried out at the end (following a workshop session) which summarised and presented programme successes and challenges in terms of design, management, establishing links with external stakeholders, implementation at global and regional level and implementation at country level.

3. DIPECHO Action Plan for South East Asia, 1998

   The evaluation methodology was purely qualitative and non-participatory, based almost entirely on interviews with local NGO staff and some government officials. The evaluators also reviewed project documentation and directly observed a sample of structural mitigation projects. It is not clear from the report what method was used to assess different respondent’s views i.e. how they were cross-checked or weighted.

4. CAMI/ARC Mitigation Grant for Risk Management and Community Preparedness, 2001-2003

   The overall design of the evaluation of project outcomes is a before and after study within the CAMI Project Area. Initial baseline results enabled the CAMI project to establish a) what is already working and b) what can be improved in Red Cross, school and household preparedness and response. Performance was then measured against the baseline. This was the first disaster preparedness baseline assessment prepared by the American Red Cross, International Services. Significant effort went into researching viable methods for utilizing and adapting existing measurement methods from OFDA, FEMA and Red Cross Domestic Disaster Services.