Terms of Reference to Conduct Process Evaluation at the End of Project.
Process Evaluation for the “Increasing Protection for Vulnerable Refugees and Host Communities in Nairobi” Project.
HIAS is a global Jewish not-for-profit organization that stands for a world in which displaced persons attain full potential and contribute to society through a sustained legal, social, and systematic change, attaining legal status, exercising social and economic rights, and rebuilding their lives. HIAS’ operation in Kenya was established in 2002 with the mission to serve the most vulnerable refugees through community-based protection, mental health and psychosocial support (MHPSS), economic inclusion, gender-based violence, child protection, and provision of durable solutions.
With funding from the Bureau of Population, Refugees, and Migration (BPRM), this project, “Increasing Protection for Vulnerable Refugees and Host Communities in Nairobi”, will be implemented by HIAS Kenya over a one-year period. The project’s goal is to improve the safety and well-being of vulnerable refugees and host communities living in Nairobi through community-based protection and capacity building. The project aims to achieve the following outcomes:
- Improved safety and well-being of survivors of GBV and vulnerable women and girls at risk of GBV through risk reduction and response services.
- Strengthened safety of unaccompanied and separated children (UASC) and other children at risk by facilitating access to community-based safe living arrangements.
and appropriate support services.
- Improve mental health and psychosocial wellbeing of vulnerable refugees through expanding the provision of integrated MHPSS interventions.
PURPOSE AND OBJECTIVES OF THE EVALUATION
The evaluation will focus on assessing the process and implementation of the project in order to inform future program design. While this will include some discussion of project outcomes, the evaluation will focus primarily on understanding how project activities were implemented and adapted in the ongoing COVID-19 context in Kenya, as well as how these adapted activities contributed to enhancing protection among the project target population.
The purpose of the evaluation is three-fold, as detailed below:
- Assess the implementation of the project activities in each of the three objectives, focusing especially on the relevance, quality, and sustainability of both planned activities and COVID-19 adaptations to these activities.
- Document the end-of-project protection outlook for the target population through the collection of end-line data for a cross-cutting key performance indicator.
- Identify and articulate lessons learned that can inform future similar activities, as well as inform the development of future programming and COVID-19 adaptations for upcoming project cycles.
The lessons learned and key recommendations derived from the evaluation will inform future project design and implementation. In particular, the evaluation will be shaped by the following specific study aspects:
- Project implementation process and practices, including how planned activities and services were adapted over the course of the project to respond to the changing COVID-19 context; project management processes and allocation of resources to the project; and skills-building and training for staff in order to deliver the intended activities.
- Relevance and alignment of both planned and adapted project activities to identified community needs in the changing COVID-19 context, including the process of adapting activity design, implementation, and service modalities(s); and whether and how the activities met the changing/changed needs of the target population.
- Accountability to stakeholders, including project acceptance at the community level, whether and how local communities and the intended target population were consulted, whether and how the project adapted to ensure inclusion of the target population in the changing COVID-19 context, and how the project is/was perceived by local organizations and actors.
- Overall effects of COVID-19 on achievement of the program goal and outcomes, including whether and how HIAS’ programmatic response cushioned COVID-19 shocks, supported coping strategies in intervention areas and affected absorptive, adaptive, and transformative resilience.
- To document the end-of-project value for the following cross-cutting key performance indicator, which measures the protection outlook for people served under each of the project objectives: “Percentage of beneficiaries who report an improved sense of safety and well-being at the end of the program, disaggregated by age and gender.”
- To assess the extent to which Age, Gender, and Diversity were mainstreamed; how gender equality, social inclusion, and equity issues have been considered during implementation.
SCOPE OF THE EVALUATION
The geographical scope of the study will be within Nairobi County. The design and implementation of the end-line will ensure ethical considerations are put in place, e.g., principles of gender equality, inclusion, and non-discrimination are considered and acted upon throughout, and that the meaningful participation of the most vulnerable groups and other key stakeholders is promoted in the design and implementation of the end-line.
METHODOLOGY OF THE STUDY
The consultant will develop a detailed evaluation methodology in consultation with the HIAS program and M&E team. The following methodologies will be utilized but should not be limited to.:
- Desk review.
- Qualitative data collection on implementation and process—interviews with key HIAS staff, partners, and people served who can speak to the focus areas we have identified above.
- Survey data collection for the safety and well-being indicator.
The process evaluation will be organized in a participatory manner i.e., in line with SPHERE standards, including HIAS staff, Partner agency staff, refugee community leaders, and project beneficiaries. All data will be collected in a COVID-19-safe manner as per HIAS’ COVID-19 safety policies and as aligned with all Government of Kenya directives and guidelines.
EXPECTED OUTPUTS AND DELIVERABLES:
The expected key outputs and deliverables are:
- Work plan and Inception Report (with key parameters and indicators to be considered for the baseline survey including the methodologies for data collection and dissemination).
- Information collection tools/instruments (such as interview guides and protocols) that address the focus areas outlined above.
(Note: the survey questionnaire that will guide data collection has already been developed and will be shared with the consultant)
- Draft evaluation report, including key findings, analysis, methodology description, and additional gaps identified, for stakeholder review.
- Cleaned final dataset(s) containing all data collected for the evaluation, including (1) qualitative data collection transcriptions, coding, and analysis, and (2) updated end-of-project values for the cross-cutting key performance indicator on protection, to be delivered in the form of a cleaned SPSS/Stata dataset with key descriptive statistics and basic comparative analysis.
- Final Report (incorporating inputs from the review) will include:
A comprehensive and well-organized final report complete with standard reporting formats (main body of the report should be a maximum of 20 pages in length, excluding Table of Content, tables, and annexes). The report must be in English and must address both the overall and specific objectives of the survey. The report must contain:
- An executive summary. It should include the major findings of the evaluation and summarize conclusions and recommendations.
- The evaluation objectives, evaluation questions, and methodology.
This must include information on relevant underlying values and assumptions, theories; desk review process; a detailed description of data collection strategy and selection criteria for interviews and survey; and any data collection limitations observed.
- Analysis and findings
- Lessons learned
- Appendices with TOR; list of documents reviewed; list of persons interviewed; survey sampling frame; survey and interview tools used.
- The findings will be presented in a meeting organized for key HIAS staff:
The probable date for the evaluation to commence will be the first week of August 2021 and is expected to take a maximum of 18 working days, which includes desk review, preparation, and implementation, report-writing.
HIAS will establish an evaluation team to oversee all the related tasks. The HIAS Kenya M&E Officer will be responsible for the overall coordination of all the evaluation tasks with the Consultant, and HIAS’ HQ M&E team will provide additional technical guidance and advisory via the Regional M&E Officer. In addition, the Program Managers, Head of Programmes and Country Director will provide all the necessary technical and operational support required throughout the evaluation process.
HIAS will provide:
- Relevant documentation and background information.
- Contacts of relevant stakeholders and support with setting up the meeting(s).
- Survey data collection tool for key performance indicators (as specified above).
- Consolidated feedback/guidance on draft reports and strategy.
REQUIRED EXPERIENCE AND QUALIFICATIONS
We are looking for a consultant/team with the following skills and qualifications:
- Strong understanding of the Kenya contexts, particularly around issues refugee operations, conflict, resilience, the humanitarian principles and system, protection risks
- An individual with demonstrable experience in qualitative & quantitative research, data analysis, and reporting with a focus on social research and evaluation, preferably in the region.
- The team leader should preferably possess a post-graduate degree in research-oriented social science or related discipline with extensive knowledge of and experience in leading (designing and undertaking) qualitative research initiatives, as well as survey methods.
- Experience in managing and coordinating evaluation/research exercises, delivering agreed outputs on time and on budget.
- Experience in data collection and analysis using participatory methodologies;
- Experience conducting and facilitating interviews and group discussions in cross-cultural contexts.
- Experience transcribing, coding, and analyzing qualitative data, including use of relevant analysis platforms such as Nvivo, Dedoose, or others
- Excellent and demonstrated understanding of ethical issues in research, including child protection.
- Ability to respond to comments and questions in a timely, appropriate manner.
- Capacity to use mobile data collection systems such as KoBo Collect, and analysis of survey results.
- Excellent organizing, facilitating, presentation, and communication skills, including good report writing in English.
APPLICATION PROCESS AND REQUIREMENTS
Qualified and interested parties are asked to submit the following:
- Letter of interest in submission of a proposal
- A detailed technical proposal clearly demonstrating a thorough understanding of this ToR and including but not limited to the following:
- Proposed methodology including areas of piloting, Sample Size Determination, and a proposed consultancy work plan.
- Demonstrated previous experience in similar assignments and qualifications outlined in this ToR (with submission of the most recent report)
- Proposed data management plan (collection, processing, and analysis).
- Team composition and level of effort of each proposed team member (include CVs of each team member).
- A financial proposal with a detailed breakdown of costs for the study quoted in Kenya Shillings.
Method of Application
Interested persons should submit their applications by Friday 23rd July, 2021 at 4.00pm to:
THE BOM SECRETARY
ST. MARY’S LWAK GIRLS’ HIGH SCHOOL
P.O.BOX 4 — 40611
Shortlisted candidates will be contacted on Tuesday 27th July, 2021. Yours faithfully,
SR. ANASTACIA F. AMOLLO, HSC
BOM SECRETARY/CHIEF PRINCIPAL.