Somalia: End of Project Evaluation - Emergency Food Security Program

Mar 19, 2019 | World Vision
View Original

World Vision is a global Christian relief, development and advocacy organization dedicated to working with children, families and communities to overcome poverty and injustice. World Vision serves all people, regardless of religion, race, ethnicity, or gender.

1. Introduction

The purpose of this Terms of Reference is to provide a framework for planning and conducting the Final Evaluation (FE) for the Emergency Food Security Program in Somalia (EFSP). The Final Evaluation will be in the form of an external review and will be conducted by an independent third-party consultant. The methodology of the evaluation will be based on the collection and analysis of quantitative and qualitative data. This document describes the objectives and goals of this evaluation. It also explains the logistical details to be considered during the data collection process and steps, technical procedures and tools to be used.

2. Background & Program description

The collapse of the state of Somalia in 1991 has had an enormous impact on the country’s human development. While Somalia has made modest development gains, high levels of humanitarian needs persist due to cyclical climatic impacts, armed conflict, clan violence, widespread human rights violations, political instability and insecurity. Food security challenges in many parts of Somalia has been compounded by a combination of factors that include recurrent droughts, flooding and insecurity. To prevent further deterioration of productive assets and mitigate the need for households to engage in negative coping strategies to meet basic food needs, WV implemented the Emergency Food Security Program (EFSP) in Somalia. The EFSP is a USAID-funded emergency food assistance project implemented by World Vision, Inc. in Berbera, Burco, Odweyne and Gabiley districts of Somaliland and Wajid, Hudur, Luuq and Qansaxdheere districts in the South of Somalia. The EFSP commenced in May 1, 2018 with plans to support 5,300 vulnerable households (approximately 31,800 individuals) access food through the local market over a seven-month period (May 1, 2018 – November 30, 2018) but was later modified in September 2018 to extend the implementation period to May 2019 and cover additional 5,700 households. The Program is implemented directly by World Vision in Berbera, Burco, Odweyne and Gabiley districts and in partnership with African Relief and Development (ARD) as the sub-recipient in Wajid and Hudur districts in Bakool region and Centre for Research and International Development (CeRID) in Luuq and Qansaxdheere districts.

The overall goal of the Emergency Food Security Program is to improve food security by addressing the immediate, life-saving food needs of 11,000-targeted households affected by natural disasters such as drought, floods and cyclones[1] in addition to setting the foundation for early recovery through rehabilitation and creation of community assets. This has been accomplished through provision of conditional and unconditional cash based food vouchers to enhance access to diverse and quality foods that meet the nutritional requirements of vulnerable households through the local market, developing the capacities of local vendors to progressively meet food needs and promotion of household nutrition and hygiene. Furthermore, early recovery and resilience building for target communities has been facilitated by creating and rehabilitating community assets and restoration of infrastructure promoting access to markets, farmland, pasture and other services. To achieve greater impact, the EFSP has been complemented by the OFDA funded project in Wajid, Hudur, Qansaxdheere and Luuq district. Target households have been linked to other services provided by the OFDA project such as Health and Nutrition, WASH and Livelihood support. In other locations, the EFSP was deliberate in ensuring that integration and layering with other existing and upcoming projects took place to maximise impact.

Target households under the EFSP were determined using the Somalia Food Security Cluster’s Community Based Targeting Guidelines adapted from best practices in consultation with the communities; the selected households were registered using WV’s LMMS system[2].

Program Goal: Improved food security for vulnerable households and restoration of community assets for early recovery in Burco, Odweyne, Berbera, Hudur, Qansadheere, Wajid, Gabiley and Luuq districts in Somalia.

Purpose 1: Increased access to diverse and quality foods that meet the nutritional requirements of vulnerable households through the local market.

Purpose 2: Early recovery and resilience-building for target communities facilitated

3. Evaluation Purpose and Objectives

The EFSP has planned for a Final Evaluation to be undertaken as part of a culture of learning and accountability. A team of independent, external consultants will be contracted to assess the performance and results of the EFSP against the mandate that was set in the project’s strategic frameworks, and to determine the reasons for success or lack thereof, draw lessons and recommendations for improved performance in future food security responses. The overall purpose of this final evaluation is to analyze the level of achievement of the EFSP project goal, objectives and results and how these have been achieved.

Specifically, the final evaluation will serve the following purposes:

  • Evaluate the achievements of the activity in relation to the goal, objectives, results and targets.
  • Evaluate the activity’s effects on local markets, and how it affected certain groups of interest (women and men; the youth population; boys and girls, etc.)
  • Evaluate the effectiveness and relevance of the modality, transfers, and complementary interventions to achieve activity outcomes.
  • Identify best practices, lessons learned, strengths, and challenges in the activity design, including the Log frame, and implementation for achieving project achievements.

4. Evaluation questions

The final evaluation will raise and analyze the key learnings and challenges as well as the strengths, weaknesses, opportunities and threats, the project and its’ implementation have passed through and how well the project has managed them. It will investigate potential effects or impacts of the project on participants and their community knowledge, attitudes and practices.

Illustrative evaluation questions include:

a. Project Relevance – Did the project address priority problems faced by the target areas and communities and was the project consistent with recipient governments or agencies

  • What are the stakeholders’ opinions about the nature and quality of project implementation?
  • Under what circumstances and/or in what contexts would the program be replicable or could it be scaled-up?
  • Were the appropriate Government Department officials involved?
  • Was the project implementation strategy adjusted to accommodate field realities? If yes, in what way?
  • What lessons were learnt?
  • Recommendations for future project design.

b. Project Efficiency – Were inputs (staff, time, money, equipment) used in the best possible way to achieve outputs; could implementation been improved/was there a better way of doing things

  • Did the project have adequate and the appropriate resources (human, financial and capital) for implementation?
  • If there were any lack/problem in resources/capacity, how was this addressed?
  • Assess the communication structure in place and its effectiveness in supporting the implementation of the program.
  • Were the quality control and accountability measures in place and consistently applied during the review, approval, fund disbursement, monitoring and reporting phases?
  • What lessons were learnt?
  • Recommendations for future project design.

c. Project Effectiveness – Whether activities, outputs and outcomes have been achieved

  • How successful was the project in accomplishing each of its objectives?
  • Did the Monitoring and Evaluation system provide quality information that was appropriate and reliable in measuring the intended indicators?
  • To what extent was the project model effective in terms of its design, appropriateness, management and accountability?
  • What are the major factors influencing the achievement or non-achievement of the objectives?
  • How effective was the project in terms of program delivery (coordination, cooperation, efficiency, standardization)?
  • How appropriate were the strategies used to accomplish the planned activities?
  • Were humanitarian standards met and humanitarian principles followed? (Sphere, HAP, Codes of conduct)?
  • What lessons were learnt?
  • Recommendations for future project design.

d. Project Impact – what changes are observed in the lives of the target group as a result of the implementation of the project?

  • What are the unintended positive and negative impacts of the implementation of the project?
  • Did the response reduce future vulnerabilities?
  • Which measures were taken to identify and reduce the negative effects of the project?
  • To what extent are the interventions improving the condition of affected communities?
  • How satisfied are the communities with the response?
  • Did the program demand more time from women?
  • To what extent was the project model effective in terms of its design, appropriateness, management and accountability?
  • What do the men and other household members (in-laws etc.) think about women’s participation in the program?
  • What gender specific issues have been addressed?
  • What lessons were learnt?
  • Recommendations for future project design

e. Linkages, Layering, and Exit Strategies

  • To what extent did the project take advantage of other USG and on-USG investments in the same space to facilitate linkages with complementary services, layering with earlier investments, and implementing exit strategies to minimize the dependency on external support.
  • To what extend did the project align and integrate with host government social protection strategy/policy/service delivery?

f. Sustainability - Continuation, maintenance and replication of the project outcomes by communities, local authorities and central government after funding assistance has ceased.

  • To what extent will the benefits of the program or project continue after donor funding ceases? Are the positive effects sustainable?
  • What were the major factors that influenced the achievement or non-achievement of sustainability of the program or project?
  • What sustainability drivers are evident? (Local Ownership, Partnering, Transformed Relationships, Local and National Advocacy, Household and Family Resilience)?
  • To what extent does the intervention reflect on and take into account factors that, by experience, have a major influence on sustainability like economic, ecological, social and cultural aspects?
  • Did the conditional nature/aspect of the project address any aspects of sustainability within the targeted communities?

5. Evaluation Methodology

The methodology of the evaluation will be based on the collection and analysis of quantitative and qualitative data and information in the intervention area. Results will be analysed against the baseline study findings. The purpose of the quantitative review will be to collect and analyse relevant data that will facilitate comparison of key indicators of success to determine the impact against objectives. The qualitative review will focus on gathering appropriate data that will facilitate a deeper understanding of processes and approaches, perceptions and behaviours and other factors that have contributed to the achievement and/or non-achievement of targets.

6. Deliverables & reporting requirements

Inception Report: The consultant will prepare and submit an inception report describing detailed evaluation work plan, methodology, data collection tools and a detailed timeline. Draft questionnaires and other data collection tools will be submitted to WV for review and approval before data collection starts. As part of the inception report, the consultant must provide a data analysis plan showing the questions and analysis for each of the project indicators to be investigated.

Preliminary Report: The consultant will submit a draft evaluation report to the Emergency Food Security Program team through supply chain. The draft report will be reviewed and comments provided on the report within a week of submission.

Meeting/presentation to disseminate key (draft) findings to the EFSP team. Reactions to the preliminary report/presentation will be accommodated in the final report.

Final Report: The consultant will submit detailed final report outlining the evaluation methodology, findings, lessons learned and recommendations. The report shall incorporate specific simple and achievable recommendations, including the most appropriate strategies that can be undertaken and/or incorporated by WVS and partners to attempt to address the issues identified. The final report should address the issues and questions raised in this ToR and correspond to the evaluation objectives set out above. The final report in both hard and electronic copies shall be made available to WV Somalia not later than May 28, 2019.

7. Consultant/Team Expertise

Survey Team Composition, Qualification and Roles

Team leader qualifications:

  • Must possess a post-graduate degree in Humanitarian Studies, Disaster Management, Development Studies, Anthropology, applied research and/or relevant Social Sciences discipline.
  • Must possess extensive evaluation experience using mixed methods in developing countries
  • Must possess prior experience/knowledge in both quantitative and qualitative approach and survey design
  • At least 7 years of proven working experience in the context of Somalia, knowledge of the regional context and languages will be an advantage.
  • Must be experienced in evaluation of food security programs, with strong preference toward USAID FFP programs.
  • Proven experience with natural resource management planning processes, policy development or reform, multi-stakeholder platforms or expert networking in environment -related context, in relation to sustainable development / poverty issues in developing economy context.
  • Experience in the use of participatory methodologies and developing equality and gender sensitive evaluation methodologies.
  • Familiarity with international quality and accountability standards applied in emergencies.
  • Strong analytical and conceptual skills.
  • Excellent facilitation skills, co-ordination, negotiation skills and oral and written communication skills in English (particularly report writing).
  • Excellent written and spoken communications skills in English.
  • Experience in assessing organizational capacity and gaps and ability to recommend the corrective measures.