Mozambique: Disaster Risk Resilient Livelihoods Recovery in Gaza Province - Conduct Final Evaluation

Dec 6, 2015 | UNDP
View Original

In response to the devastating floods that hit Mozambique from January to February 2013, UNDP Mozambique launched a two year project in support of disaster risk resilient livelihoods recovery of flood-affected communities in Gaza province. In close partnership with INGC at national, provincial and district levels, local government institutions, affected communities, the project attempted to restore an enabling environment for flood-affected communities in Gaza province to recover from the impact of the floods and “build back better” through three interlinked outputs:

  • Disaster resilient livelihoods and economic opportunities for the flood-affected people in Gaza Province restored and improved;
  • Capacity of local authorities and communities’ livelihoods recovery strengthened;
  • Coordination of livelihoods recovery by Early Recovery Cluster strengthened;
  • Toward this end, UNDP applied a community-based approach in Gaza province through restoring and developing capacity of local authorities and communities to lead the disaster risk resilient livelihoods recovery process. This project forms a part of the ongoing support that UNDP is providing to the government through INGC for disaster risk reduction and climate change adaptation.  The project was implemented in two phases in line with Early Recovery framework: 1) to respond to the urgent needs with the interventions to help stabilize livelihoods and build the foundation for sustainable development pathways; 2) to further enhance and sustain medium-long term local economic recovery and development, building upon the foundation made through the phase I (first year);
  • The UNDP Evaluation Policy states that: Project evaluations assess the efficiency and effectiveness of a project in achieving its intended results. They also assess the relevance and sustainability of outputs as contributions to medium-term and longer-term outcomes;
  • Terminal evaluations (TE) provide a comprehensive and systematic accounting of performance at the end of the project cycle, considering the totality of the effort from project design, through implementation to wrap up, also considering the likelihood of sustainability and possible impacts;
  • The target audience for a terminal evaluation is project partners and beneficiaries, UNDP at country, regional and HQ levels, BCPR, UNDP Evaluation Office. The project in question is funded by BCPR and the project document was signed on June 2013;
  • The duration of the project was meant to be until June 2015 as stipulated in the signed project document.  The project requested for a no cost extension until December 2015 due to delays the project experienced with field implementation of certain activities;
  • The project is now reaching its final stages hence a terminal evaluation is necessary in order to assess the project’s overall performance and intended outcomes.
Duties and Responsibilities

Scope of work:

  • Provide a comprehensive and systematic accounting of performance;
  • Assess project design, implementation, likelihood of sustainability and possible impacts;
  • Conduct an assessment/ stock taking of the achievements and impacts of the project with regards to the program objectives;
  • Indicate project drivers and constraints to achieve the intended objectives;
  • Assess the sustainability of current achievements and identify measures and recommendations to ensure long term sustainability in support of the national development process and poverty reduction priorities;
  • Present lessons learned and selected best practices / core strengths. Give recommendations to build on and/or scale up the best practices for future projects to effectively support sustainable development;
  • Conduct a needs-assessment with relevant institutions and sectors in order to identify future project possibilities.

Tasks to be conducted by the consultant and/or consultant team:

  • Desk review of relevant documents (project reports, sector plans and strategies, Mozambique key DRR documents and policies and other relevant documentation ) –  4 working days;
  • Interviews with strategic partners  + focus groups + review of documents on site  - 3 working days;
  • Interviews and field work -7 working days;
  • Writing up the reports (draft reports and final report), including incorporating comments from stakeholders – 7 working days (5 days for a draft; 2 working days for final report).

The following questions should be covered by the evaluation within the scope define in the current consultancy work:

Project formulation:

  • Were the project’s objectives and components clear, practicable and feasible within its time frame?
  • Were the capacities of the executing institution(s) and its counterparts properly considered when the project was designed?
  • Were lessons from other relevant projects properly incorporated in the project design?
  • Were the partnership arrangements properly identified and roles and responsibilities negotiated prior to project approval?
  • Were counterpart resources (funding, staff, and facilities), enabling legislation, and adequate project management arrangements in place at project entry?
  • Were the project assumptions and risks well articulated in the PIF and project document?

Assumptions and risks:

  • An assessment of the stated assumptions and risks, whether they are logical and robust, and have helped to determine activities and planned outputs;
  • Externalities (i.e. effects of climate change, global economic crisis, etc.) which are relevant to the findings.

Project implementation:

  • The logical framework used during implementation as a management and M&E tool;
  • Effective partnerships arrangements established for implementation of the project with relevant stakeholders involved in the country/region;
  • Lessons from other relevant projects (e.g., same focal area) incorporated into project implementation Feedback from M&E activities used for adaptive management.

Finance/co-finance:

  • A forensic audit of the project accounts should be undertaken. This should assess the quality of budget management of the project. Budget analysis should be undertaken to assess expenditures against planned budget lines in the Total Budget and Work Plan (TB&WP). Variances between planned and actual expenditures for each project Outcome should be assessed as to whether they are commensurate with project outputs delivered and results, and reasons for any inconsistencies should be identified;
  • Observations from financial audits as available should be considered. The consultant should prepare recommendations for improving control mechanisms if necessary.

 Expected Outputs:

  • An inception report should be prepared by the consultant and/or consultant team prior to the main evaluation mission. It should detail the consultants’ understanding of the project being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods, proposed sources of data and data collection procedures.  The inception report should include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product.  The inception report should annex the signed code of conduct agreement form;
  • A draft evaluation report (see Annex 2 for outline), which includes the evaluation scope and method, findings, conclusions and recommendations.  The report should cover the following five major criteria:  relevance, efficiency, effectiveness, results and sustainability, applied to a) project formulation b) project implementation and c) project results. Draft evaluation report is expected to be submitted after 3 weeks from the start of consultancy work;
  • A final evaluation report, the draft final report is considered complete, in contractual terms, only when it has achieved acceptable standards. Final evaluation report is expected to be submitted after four weeks of entire period of the consultancy work;
  • The evaluation will properly examine and assess the perspectives of the various stakeholders.  Interviews should include a wide array of interested persons including beneficiaries, local district authorities and technical officers involved, NGOs and INGC at provincial level. Field visits to the project sites in Gaza (Chókwè, Chibuto, Guijá, Xai-Xai) are expected.   Decision regarding which sites to visit should be done jointly with the Country Office (CO) and project team. Data analysis should be conducted in a systematic manner to ensure that all the findings, conclusions and recommendations are substantiated by evidence.  Appropriate tools should be used to ensure proper analysis (e.g. including a data analysis matrix that records, for each evaluation question/criteria, information and data collected from different sources and with different methodology). By the end of the evaluation mission and prior to submitting a first draft evaluation report, a wrap up discussion should be organized with the CO and project team to present initial findings and request additional information as needed.

Institutional Arrangement:

  • The consultant will be under supervision of the main IP team (INGC) team through project focal point and will also be supervised by UNDP project team and focal points in order to ensure that expected results are achieved and key project information is provided in a proper format. The consultant will work directly with the project assistant, project focal points and district authorities in Gaza, including local communities where interviews will be conducted;
  • As described above, the consultant will be requested to submit an inception report, draft of the field finding report and a final report to be approved by UNDP and INGC;
  • Main partners and institutions to be interviewed by the consultant include: National Institute for Disaster Management (INGC), National Meteorology Institute (INAM), District Economic Activities Office (SDAE), Provincial Department for Agriculture (DPA), District Infrastructures office (SDPI); government authorities within targeted districts;
  • The IP/ INGC is expected to provide basic facilities for the consultant and/or consultant team, which will include facilitate setting up meetings with stakeholders and partners; provide all necessary documentation of the project and necessary back up on information and guide on project objectives and main findings within the project life time;
  • Other facilities, such as transport, accommodation, meals and extra trips within the field and in the main city will be covered by consultant fees.

Duration of the Work:

  • An 'evaluation mission' should be scheduled, providing an intensive 21 days for the evaluation team to hold interviews and visit project sites, field data collection and analysis, and draft final report.  The evaluation mission should be planned far enough in advance to enable interviews to be properly set up, especially to request meetings. A detailed plan for the mission should be included in the TE inception report, which should be revised based on CO, project team and IP inputs.

Working days are expected as follows:   

  • Desk review of relevant documents (Coping with Drought project reports, sector plans and strategies, Mozambique key Climate change and environmental documents and policies and other relevant documentation ) –  4 working days;
  • Interviews with strategic partners  + focus groups + review of documents on site  (3 working days);
  • Interviews and field work (7 working days).

Deliverables/ Writing up the reports:

  • The inception report is expected 4 days after the formal assignment of the TE;
  • The draft evaluation report, (5 days to write the draft) is expected to be submitted after 3 weeks from the start of consultancy work;
  •  Final evaluation report, (2 working days for final report), is considered complete, in contractual terms, only when it has achieved acceptable standards. Final evaluation report is expected to be submitted after four weeks of entire period of the consultancy work;
  • Following the review of the draft evaluation report, the evaluation team should indicate how comments have been addressed in the revised evaluation report. The consultancy work is expected to start in December 1, 2015 and final evaluation report to be submitted by December 21, 2015;
  • Estimated lead time for UNDP and project Implementing Partner to review outputs, give comments, approve/accept outputs to the draft TE report is 10 days;
  • The time frame should be strictly followed as subsequent actions are dependent on this report, and serious consequence/impact of any form of delay in the completion of the work.