Kenya: Terms of Reference: Endline Evaluation for UKAid Match Project in Kitui, Tharaka-Nithi and Embu Counties

Jul 30, 2019 | Trócaire
View Original

Trócaire is an Irish development agency that was established in 1973 by the Catholic Bishops of Ireland. Trócaire works in over 20 countries in partnership with local civil society organizations implementing both long-term development and emergency response programmes. Trócaire has been working in Kenya for over 30 years and established its Nairobi office in 1994. Working with over 25 local partner organisations, Trócaire in Kenya aims to contribute to the development of a “just and peaceful Kenya in which poor women, youth and other vulnerable groups realise their rights and improve their quality of life in dignity and safety”. Trócaire seeks to achieve this goal by implementing programmes in three thematic areas: Accountable Governance, Resilient Livelihoods and Women’s Empowerment.

2.0 UKAID Match Programme Overview

Since October 2016, Trócaire has been implementing a DFID funded resilience-building project named Community Resilience and Climate Change Adaptation Program; herein referred to as UKAid Match (UKAM) in drought affected areas of Tharaka Nithi, Ishiara and Kitui. The project whose period is October 1, 2016 – September 30, 2019 was aimed at strengthening the capacities of 12,000 vulnerable households; 7800 female headed households (FHH) and 4200 male headed households (MHH) in the drought-affected semi-arid areas of 3 counties (Meru, Kitui and Embu) to be more resilient to shocks and stresses they face as a result of climate change.

2.1 Project Context

This project specifically targets the most marginalised communities of the semi-arid lands of Kitui (Ngomeni and Nguni Wards), Tharaka-Nithi (Igamba-ng’ombe and Gatunga Wards) and Embu Counties (Evurore Ward). These areas, all of which are extremely poor, are in danger of becoming arid due to irregular climate patterns and unsustainable resource use. The high county poverty rates of 63.5% (Kitui County), 48.7% (Tharaka-Nithi County) and 42.0% (Embu County) mask even higher pockets of extreme poverty and inequality caused by displacement and migration, intensified environmental degradation and increasing competition over natural. Over the past decade droughts have become more severe and frequent, having a negative effect on all rural households (HHs). Among the Semi-arid Marginal Mixed Farming areas, the short rains start later and the long rains have become more unreliable, causing low harvests and food insecurity. By the end of 2014, 40% of Marginal Mixed Farmers (MMF) in project counties were either severely or moderately food insecure and families faced a 3-month hunger gap each year. As a result, HHs resorted to an additional 3 negative coping strategies in 2014. Coping strategies, such as missing meals or charcoal making only deepen poverty and undermine health. HH dependence on biomass (hard wood) for fuel also undermines the sustainable management of the forests and tree cover.

2.2 Project impact

The project’s intended impact is that - 12000 vulnerable households in three drought-affected semi-arid Counties in Eastern Kenya are more resilient to the impact of shocks and stresses related to climate change. This is aimed at contributing to the reduction of poverty and vulnerability for drought-affected households living in three semi-arid counties in Eastern Kenya: Kitui, Embu and Tharaka Nithi. The project aimed to achieve this impact by supporting vulnerable rural households to become more resilient to drought and other climatic shocks.

The approaches set to achieve this were:

a) Supporting target households to become both more economically secure and more food secure.

b) Supporting vulnerable communities to benefit from improved ecosystem services through the adoption and application of ecologically sound and climate sensitive natural resource management and agricultural production techniques, and

c) Supporting communities and local government actors to develop the capacities that they need to effectively plan for, adapt to, cope with, respond to and recover from climate-related shocks and stresses.

This multi-layered approach would support vulnerable households and their communities to become more prosperous and more resilient to shocks in a sustainable manner that is ecologically appropriate and adaptive to climate change. Ultimately, this project would contribute to a reduction in both poverty and chronic malnutrition within the target households and communities, and to an improvement in their well-being and productive capacity. The project was set to contribute to the achievement of the following Sustainable Development Goals:

  • SDG 1: End poverty in all its forms everywhere by reducing the number of people in target communities living below $1.25 per day
  • SDG 2: End hunger, achieve food security and improve nutrition and promote sustainable agriculture
  • SDG 13: Climate action
  • SDG 15: Life on land

3.0 Purpose of the End line and Programme Evaluation

The main purpose of the End Line Study and Programme Evaluation is to measure the extent to which the intended impact of the project has been achieved. The evaluation will be both qualitative; looking at the changes observed in the lives if the target beneficiaries over the three consecutive years, and quantitative - collect outcome data collection alongside each intended outcome as per the logframe and in relation to the baseline data collected at the beginning of the project. The secondary aim of the evaluation is to gain insight into key elements of the implementation process, which either explain why reported changes occurred and shed light on how such changes were achieved. We expect that an assessment of impact and understanding of key processes will support the generation of programme recommendations that could be considered for the design of future programmes.

We intend to contract an independent consultant to carry out this work. It is expected that both Trócaire and partners will play a key role in designing and carrying out the evaluation such as confirming the suitability of tools, introducing the consultant to the communities and assisting the consultant in data collection, as well as commenting on the analysis of all findings.

3.1 The Specific Objectives of the End Line Evaluation(ELE)

On the results framework, the ELE will:

  • Assess achievements and progress made against planned results, intended and unintended, positive and negative as well as assess challenges and lessons learnt;
  • Assess, to the extent possible, how the emerging issues not reflected in the project document such as SDGs may have impacted on outcomes;
  • Assess effectiveness towards attainment of results and reflect on how Trócaire and Partners have contributed to the results achieved;
  • Assess if broader development, social and gender aspects of the project were achieved; and
  • Assess quantitative and qualitative achievements against each of the project indicators.

3.1.1 Impact level

In reference to the baseline study as the benchmark, the evaluation will identify:

  • i. Proportion of targeted FHHs and MHHs that are able meet their basic needs during the lean season
  • ii. Proportion of target MHH and FHH resorting to negative Livelihood coping strategies during dry season.

3.1.2 Outcome level

1) The evaluation will seek to identify, how many of the 4800 (3120 FHH 1680 MHH) Vulnerable households targeted on economic security improvement in three drought affected semi-arid Counties have improved levels of economic security. This will specifically measure:

  • a. Proportion of targeted FHHs and MHHs with more than two sources of income
  • b. Total Mean value of HH Assets held by targeted FHH and MHH: a) Mean value of productive assets held by FHH and MHH; b) Mean value of non-productive assets held by FHH.
  • c. Extent of economic change of targeted HHs as a result of project intervention.

2) The evaluation will show, how many of the 7200 (4680 FHH 2520 MHH) Vulnerable households targeted on food security improvement in three drought affected semi-arid Counties have improved levels of food security. The evaluation will specifically measure:

  • a. The proportion of targeted MHH and FHH that are within the acceptable food consumption score range
  • b. The proportion of target FHHs and MHHs that are food secure at the end of the long dry season

3.2 End Term Evaluation Criteria and Review Questions

Using the OECD criteria, the evaluation will measure:

Strategy

The extent of contribution to higher level change in line with national priorities, as evidenced through the theory of change, alignment with Trócaire’s Global and Country Strategic Plan

  • To what extent did the project pro-actively taking advantage of new opportunities, adapting its theory of change to respond to changes in the development context, including changing national priorities?
  • Was the project aligned with the thematic focus of Trócaire’s Global and Country Strategic Plan?

Effectiveness

The extent to which programme results are being achieved:

  • To what extent has the project contributed to improving the resilience of HH in the target counties?
  • To what extent has the project interventions contributed to improving the economic status of HH in the target counties?
  • What is the degree of achievement of the planned results of the project?
  • To what extent has the programme outcome and outputs been achieved (assess outcome and output indicators against targets)?
  • To what extent have effective partnerships and strategic alliances (e.g. national partners, county partners etc.) been promoted around the programme?
  • What are the indirect results (externalities) of the project, if any?
  • Are there any unintended programme results either positive or negative?
  • What are some of the emerging successful programming or cases especially from county programming and how would they be scaled up in the next programme?

Efficiency

  • Is the implementation mechanism the most cost effective way of delivering this programme?
  • Were the financial resources mobilized used in the most efficient way to reach the results?
  • Are there any apparent cost-minimizing strategies that were encouraged, and not compromised the social dimension of gender, youth and PwDs?
  • How efficiently have resources/inputs (funds, expertise, time, etc.) been converted to results, including value for money?
  • Relevance
  • Responsiveness of implementation mechanisms to the needs of IPs including national and county institutions
  • To what extent were the interventions consistent with the needs of the IPs the project was designed to serve in line with the priorities set by Trocaire’s Global and Country Strategy?
  • Does the programme design promote ownership and participation by the partners?
  • To what extent has the project been able to respond to changes in the needs and priorities of the IPs?
  • Are the stated project objectives consistent with the requirements of Trocaire’s programming principles, in particular, the requirements of most vulnerable populations?
  • How relevant and appropriate is the project to the communities?
  • Are all the target groups appropriately covered by the stated project results?

Sustainability and Ownership

Assess the extent to which the project interventions took into consideration longer term needs of the target population and to what extent programme results or benefits will be sustainable after programme closure. The programme has to be environmentally as well as financially sustainable. The extent to which these implementation mechanisms can be sustained over time:

  • Assessment of extent of sustainability of the program thus far.
  • Did the project incorporate adequate exit strategies and capacity development measures to ensure sustainability of results over time?
  • Are conditions and mechanisms in place so that the benefits of the project interventions are sustained and owned by IPs at the national and county levels after the programme has ended?
  • Have strong partnerships been built with key stakeholders throughout the project cycle that would enhance sustainability?
  • Have institutional capacity development and strengthening been built to enhance sustainability?

Management and Monitoring

The quality of the formulation of results at different levels, i.e. the results chain:

  • To what extent are the indicators and targets relevant, realistic and measurable?
  • Were the expected outcomes realistic given the project timeframe and resources?
  • Were the indicators in line with the SDGs and what changes need to be done in the next programme?
  • To what extent and in what ways were risks and assumptions addressed in the project design?
  • How were such risks dealt with during the programme implementation phase?
  • Is the distribution of roles and responsibilities among the different partners well defined, facilitated and have the arrangements been respected in the course of implementation?
  • To what extent and in what ways are the concepts of cross-cutting issues reflected in programming? Were specific goals and targets set? Was there effort to produce sex disaggregated data and indicators to assess progress in gender equity and equality? To what extent and how is special attention given to women empowerment?
  • To the extent possible, look at the UKAM programme in relation to the other resilience programmes (synergies, complementarities, overlaps/duplication etc).
  • Social and Environmental Standards
  • Does the project seek to further the realization of human rights using a human rights based approach?
  • Are social and environmental impacts and risks (including those related to human rights, gender and environment) being successfully managed and monitored in accordance with project document and relevant action plans?
  • Were unanticipated social and environmental issues or grievances that arose during implementation assessed and adequately managed, with relevant management plans updated? f.Impact
    Give evidence of impact of the programme with reference to approaches and methodologies used to reduce poverty and builds resilience – knowledge, attitude and practices. This includes intended, unintended, positive or negative changes. It involves the main impacts and effects resulting from activities on the local social, economic, environmental and other development indicators.
    To the extent possible, assess the impact of the project on resilience i.e.
  • Determine whether there is any major change in the indicators that can reasonably be attributed to or associated with the project, including impact of the project on communities in regard to empowerment, management, effectiveness, accountable, transparent and efficiency in use of resources;
  • Assess any impacts that the project may have contributed to.

Relevance

Analyse the appropriateness of the project design, activities, strategies and approaches in the light of the operational context, the timeliness of the response and its adaptation to the livelihoods security situations. It involves assessing the extent to which the activities are suited to the priorities and policies of the target groups, target HH and donors.

Innovation and technology

Identify, review and recommend promotion and scale up of innovations and appropriate technology that enhance livestock and agricultural productivity and build resilience to drought and other shocks through use of technology and other means.

4.0 Proposed Methodology

The evaluation exercise will be consultative and participatory, entailing a combination of desk review, key informant interviews, face to face household surveys, and FGDs. While interviews are a key instrument, a range of data sources will be triangulated to ensure that the evaluation is sound and objective. A random sample of the required sample size will be extracted from a beneficiary database of 12,000 households targeted by the project. On the basis of the foregoing, the consultants will further elaborate on the method and approach in a manner commensurate with the assignment at hand and reflect this in the inception report, which will subsequently be approved by the programme team in consultation with key stakeholders. Each of the project indicators must be assessed hence the evaluation firm must ensure that the proposed methodology is responsive to this. The evaluation firm in their proposal will provide specific approaches/methodology to achieve the planned evaluation, including assessing the project indicators

The key inputs to the evaluation should be as follows:

  • Interviews with key staff such Directors, Project Managers, Project Coordinators, Project Officers, M&E Officers, community resource persons facilitators (CRPs), ToTs and others deemed necessary by the evaluation team.
  • Interviews with stakeholders including respective county government departments, other NGOs working in the various counties and community leaders
  • Field visits, household surveys and FGDs in project sites in the project locations; Kitui (Ngomeni and Nguni Wards), Tharaka-Nithi (Igamba-ng’ombe and Gatunga Wards) and Embu Counties (Evurore Ward).
  • Desk review of project documents:

a) At Trócaire level: UKAM project proposal and budgets, Logframe, baseline report, risk register, annual and quarterly reports, study reports among other as shall be deemed necessary by the evaluation team.

b) At partners’ level: Monitoring reports, budgets, financial reports, narrative reports and project review reports.

5.0 Scope of Work

This being an external evaluation, it is expected to comply with the international evaluation standards (OECD), Trócaire guidelines and provide a space for learning around best practices for supporting target communities. Trócaire is seeking to identify a consultancy team, which will undertake this end line evaluation within the agreed terms of reference. This will be done in Kitui (Ngomeni and Nguni Wards), Tharaka-Nithi (Igamba-ng’ombe and Gatunga Wards) and Embu Counties (Evurore Ward).

5.1 Proposed Stages of Review & Reporting

5.1.1 Develop an inception report with a review framework. This will be done after discussion of the TOR and will include; detailed plan on the evaluation process, plan of respondents with the inquiry method, timelines and budget.

5.1.2 Collection and review of partner information and identifying information gaps through a desk review. This should include review of necessary government policy documents.

5.1.3 Develop and agree on set tools to use and field data collection process, based on tools used in Baseline and Mid Term Review process; data collection must be based specifically on the output and outcome indicators contained in the RF

5.1.4 Train enumerators and data management clerks on the use of tools – digital data would be preferred. This should include pre testing of tools.

5.1.5 Conduct fieldwork to gather information and fill identified gaps from the desk review.

5.1.6 Prepare and share statistical tables and coding lists (as appropriate). This should be disaggregated by gender and location/ or partner for comparison with the baseline.

5.1.7 Provide full final transcripts in English (They should be well labelled);

5.1.8 Preliminary analysis, draft report and feedback to wider group (including decision makers at partner level)

5.1.9 Final analysis and report of findings and recommendations (25 – 30 pages excluding annexes). This should contain the following:

  • a) Executive summary (5 pages). This should cover: background to the programme, brief overview of aims of the evaluation, brief summary of the methodology, key findings per evaluation criteria and on the achievement of indicators, conclusions, recommendations and summary of management response;
  • b) Introduction (1 page).
  • c) Description of evaluation methodology (3 - 4 pages)
  • d) Situation analysis with regards to the outcomes, outputs and partnership strategies (4-5 pages)
  • e) Key findings, including best practices and lessons learnt (3 – 7 pages); This section of the report should be clearly structured to show levels of achievement against each indicator contained in the Results Framework, providing quantitative and qualitative evidence for achievement of impact for each indicator
  • f) Analysis of opportunities to provide guidance for future programming (3 – 4 pages)
  • g) Conclusions and recommendations (2-3 pages)
  • h) Appendices: charts, TOR, field visits, people interviewed, documents reviewed etc.

5.1.10 Dissemination of findings. After a validation and stakeholder workshop, the final report will be submitted to Trócaire headquarters for final review and endorsement by the Strategic Impact Unit. This process will declare the report to the donors and the wider public.

6.0 Evaluation Team Composition and Competencies

The suggested team for this assignment includes:

  • i. Resilience, Disaster Management, Disaster Risk Reduction or Natural Resource Management (essential)
  • ii. MEAL Expert with experience in qualitative and quantitative research methods and data analysis (essential)
  • iii. A gender and inclusion expert with experience in rural livelihoods / development (essential)
  • iv. An expert in advocacy and governance (is desirable but not essential)
  • v. Enumerators will be recruited from respective sites and should have a post-secondary level of education;
  • vi. At least 3 partner staff will be fully involved in the facilitation and mobilisation where necessary

7.0 Evaluation Quality and Ethical Standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. The evaluation team will also commit to adhering to Trócaire’s Safeguarding Programme Participant Policy and Code of Conduct.

8.0 Duration and Schedule

The End of Programme End line and Evaluation should take 35 days preferably beginning from September 2nd 2019.

9.0 Anticipated Outputs/ Deliverables

The consultant will be responsible for the following deliverables:

  • 9.1 Inception report. The inception report will be a scoping exercise for the evaluation and will include a detailed description of the proposed methodologies, data collection and reporting plans. The report will also come with draft data collection tools such as interview guides and survey questionnaires, the allocation of roles and responsibilities within the team, and a timeframe with firm dates for deliverables. The inception report shall be submitted after 5 days of commencing the consultancy.
  • 9.2 Debriefings/feedback to management at all levels: The team will report its preliminary findings to Trócaire 2 days after carrying out fieldwork, in Nairobi. The team leader will incorporate any comments relating to factual inaccuracies etc., and present the full draft report to the evaluation management team within one week of the debriefing meeting.
  • 9.3 Draft report: A draft report, identifying key findings, conclusions, recommendations and lessons for the current and future projects, and taking into consideration the outputs of the debriefing session. The report should be clearly structured to provide specific evidence of achievement and effectiveness against each indicator in the RF.
  • 9.4 Statistical tables, coding frameworks and transcripts: This should be shared with the draft report
  • 9.5 Final End Line and Evaluation Report: A final evaluation report of a maximum of 30 pages excluding appendices, clearly setting out recommendations arising from the evaluation will be submitted 5 days after receiving comments from the evaluation management team. The content and structure of the final analytical report with findings (evidence of achievement and effectiveness of programme against each indicator providing both quantitative and qualitative data), recommendations and lessons learnt covering the scope of the evaluation should meet the requirements of the Trócaire Field Programme Monitoring and Evaluation Policy.
  • 9.6 Updated Programme Results Framework: This gives a summary of performance of the programme against all indicators and outputs.
  • 9.7 PowerPoint presentation of key findings: A summary of context, finding and recommendation for high level/ management discussion of the programme performance.

Notes:

1)Intellectual Property

All the materials, information and reports, the output of the evaluation exercise shall be the property of Trócaire and the consultant is bound by Trócaire’s confidentiality requirements. All data sets and transcripts must be provided to Trócaire Kenya office and Head Office in electronic copies and on a CD (2). The evaluators will not be allowed, without prior authorization in writing, to present any of the analytical results as his or her own or to make use of the evaluation results for private publication purposes.

How to apply:

Submission of Proposals

Based on the above, Trócaire is inviting interested parties to submit expressions of interest entailing technical and financial proposals. Individuals or firms applying shall detail the following:
Consultants’ profile and Capability Statement describing the technical capacity and experience of the firm or group of individuals;

  • Names and resume of individuals or team members proposed and their roles in the achievement of the assignment. This should also entail the proposed team structure for the evaluation;
  • 3 professional referees (preferably previous clients) and sample reports of similar assignments taken in the recent past;
  • Understanding of the Terms of Reference (TORs);
  • Detailed evaluation design with implementation plan and timeframe;
  • Quality Control and Quality Assessment for the Evaluation
  • Interested candidates should submit their application to the following email address: infoNairobi@trocaire.org Copying hrkenya@trocaire.org with “Trócaire UKAM 2016 - 2019 External Evaluation Consultancy” as the subject of the e-mail. Deadline for submission is July 31, 2019.
    Kindly don’t forget to submit samples of your previous work (reference calls shall be made to validate your work)