Background
Following the Russian large-scale invasion of Ukraine in February 2022, Swiss Solidarity (SwS) launched an appeal for donations and collected more than CHF 134 million. The Ukraine fund supports projects that respond to the needs of the people affected by the armed conflict in Ukraine. By January 2024, 40 projects in Ukraine were approved for a total contribution of CHF 72.9 million. With a few exceptions, the projects are implemented by Swiss partner NGOs of SwS, either directly, in collaboration with local partners or with mixed modalities. They are active in 21 of the countryโs 27 oblasts. Multipurpose cash assistance accounts for approximatively 44% of total programming in terms of funding volume, followed by shelter/housing repair projects aiming to contribute to acceptable housing conditions that account for 38%. Overall, SwS partners have however been providing a multi-sectoral response trying to tackle the broad spectrum of existing needs. Cash assistance and shelter support have been complemented by in-kind assistance, health assistance, mental health and psychosocial support and protection interventions as well as demining and other activities. A scoping mission focusing on cash and voucher assistance (CVA) and shelter/housing repairs was carried out in Ukraine in May 2023. The mission recommended issues and challenges that could be assessed in an upcoming evaluation. The report of the mission (available upon request) informed the elaboration of these Terms of Reference. In the second half of 2023, the funding guidelines have been adapted, aiming at more targeted integration of CVA interventions into sectoral programming and a better tailoring of shelter solutions to short- and midterm housing needs. The Ukraine crisis is one of the contexts in which SwS will deliberately promote locally led humanitarian action and will carry out a pilot initiative of direct funding to local and national organisations.
Purpose
The evaluation has a twofold purpose: โข It is intended to strengthen SwSโs accountability as it will analyse whether the activities funded are relevant and are implemented efficiently and in line with the foundationโs mandate and objectives, for instance to provide assistance based on need alone and to the most vulnerable. โข The evaluation process should provide learning opportunities to SwS, its accredited partners and the local/national organisations that lead and implement the activities. It thus aims to improve the relevance and quality of current and future projects funded by SwS in Ukraine.
Evaluation questions
The following questions are relevant for both CVA and shelter/housing repairs focused projects: 1. What institutional setups and arrangements for project implementation prove the most efficient, both in terms of speed and costs, without compromising the quality of the assistance? This applies particularly for house repairs, where cost and speed should not be at the expenses of final quality and wellbeing. (Effectiveness and efficiency) 2. Do the projects reach people with vulnerabilities, are they accessible to them and to what extent do they consider their specific situation and needs? (Socioeconomic coverage and proportionality to needs) 3. Is there evidence of customized approaches to gender and inclusion that make the interventions more effective and impactful? (Gender and inclusion as a cross-cutting theme) 3 The June 2023 scoping report concluded that testing the relevance and/or coherence of the large scale MPCA programming that was prevalent in 2022 and 2023 does not come across as a priority concern. It is however of interest to investigate whether the intention of SwS to pivot to more targeted CVA interventions is being realised in the more recently approved projects: 4. What is the degree of integration of CVA initiatives into the intervention logic of the programs they are nested in? Is the theory of change of the project explicit, and does the cash intervention fit into it? (Effectiveness of targeted CVA components) Regarding housing and shelter rehabilitation (including cash for shelter interventions), the evaluation should focus on effectiveness, connectedness and impact. This concerns the implementation arrangements covered in question 1 and includes the following more specific aspects: 5. For each type of shelter response, have partners ensured adequate accommodation and enabled families to live with dignity? (Appropriateness of shelter solutions) 6. How have partners tailored shelter responses to the various short and mid-term stay hosting needs? Which assistance delivers the greatest impact and sustainability for the different needs? (Connectedness) 7. What solutions for more permanent housing show the best chance of success in terms of social and economic inclusion of internally displaced persons? (Impact) 8. To what extent and how are partners in their projects enhancing and/or including self-recovery initiatives of local organisations or self-recovery efforts of affected people?
Methodology
The evaluation team is expected to use mixed methods in their approach that could include the following: โข A review of the project documentation (funding application, reports) โข Desk research that includes a review of existing literature and reports resulting from beneficiary accountability initiatives and available evaluations โข Project visits and interviews with affected people, project staff, local authorities, aid coordination bodies, local thematic experts and other relevant stakeholders โข Quantitative surveys, to prepare or to validate the qualitative fieldwork It is the duty of the evaluators to propose and to justify a methodology and a sequence of activities in the bidding document. The evaluation team is then expected to present a more detailed methodology in the inception report. Throughout the process, the evaluation team is encouraged to allow for interactions with the implementing organisations (international and local) and to keep them updated about the methodology and the process of implementation.
Outputs
The evaluation team is expected to produce the following outputs in line with agreed deadlines: โข An inception report โข A draft evaluation report โข A final evaluation report, including a short summary of evaluation findings suitable for public communication โข Presentation of the findings, conclusions and recommendations of the evaluation at sharing and learning events, one in Ukraine and one in Switzerland
Steps and deliverables
- Desk study of project documentation
The evaluation team will have access to the project files and to discuss with SwS staff (on site or online) to develop the list of projects to be considered as well as an initial schedule.
2. Inception report
The inception report will include the following elements: โข The list of projects to be considered โข The work plan for the evaluation team โข An evaluation matrix setting out: the subsidiary evaluation questions; the criteria used for assessing the questions; the planned sources for the necessary data and the methods used to collect the data โข The detailed methodology that the team plans to use โข The risks inherent in the proposed methodology as well as the broader risks that the evaluation may face โข The planned structure of the evaluation report
3. Field visits and data gathering
The SwS partners and their local partners (if any) that implement the projects will be requested to assist with information, data and interactions with project staff. The evaluation team is however expected to be autonomous regarding their transport and accommodation arrangements and their security management.
4. Draft evaluation report
The draft evaluation report will be of no more than 30 pages plus annexes. The report should be written in an accessible style suitable for humanitarian professionals. The report should include a list of recommendations, categorized according to priority (high, mid, low). Such recommendations may go beyond the scope of the evaluation questions laid out above and hence relate to any of the OECD DAC evaluation criteria. The raw results of the quantitative survey should be added as annex to the report. The evaluators are encouraged to include vignettes of individual or family experiences of people coping with the situation in Ukraine and of the role of SwS funding projects in this regard. The draft will be reviewed by the evaluation steering committee and the technical backstopping team. The evaluation manager will provide comments to the evaluators. 5. Final evaluation report After receiving comments on the draft report, the evaluation team will prepare a final evaluation report incorporating those comments that they accept. The final report should include a two-page summary drafted in accessible language suitable for a general public.
6. Sharing and learning events in Ukraine and in Switzerland
The evaluators will present and discuss the findings, conclusions and recommendations of the evaluation and lead discussions at events in Ukraine and in Switzerland for SwS partners active in Ukraine, local organisations they partner with and other stakeholders.
For more dรฉtails please click on the following link
https://www.swiss-solidarity.org/about-us/job-vacancies/