Terms of Reference for End Evaluation
of the project
“Improving Democratizations and Human Rights in Iraq 2016-2019”
Norwegian People’s Aid (NPA) is the Norwegian labour movement’s humanitarian organisation for solidarity. NPA’s goal is to support human worth and equal rights for all, irrespective of sex, disability, ethnicity, religion, age, sexual preference or social status. The NPA vision is solidarity in practice. The vision demands commitment and action to protect life and health, build democratic societies and strengthen people’s ability to master their own lives. NPA is politically independent but not a politically neutral organisation. NPA takes a clear position in important social debates and the goal of a more just world means that NPA often chooses partners that may be opposed to, or in conflict with, those in power. NPA believes that popular participation and organisation are important measures in securing human rights and giving people greater influence over their own lives and social development.
This terms of reference (ToR) outlines specifications for an end evaluation of the project “Improving Democratizations and Human Rights in Iraq 2016-2019” funded by the Norwegian Agency for Development Cooperation (Norad).
Key project goal, objectives and outcomes:
The goal of the four-year project is to strengthen the democratisation process in Iraq. This is achieved through supporting civil society organisations in Iraq to advocate with Kurdistan Parliament and the 18 Provincial Councils in Iraq to involve the public and civil society in their decision making, planning and further to advocate towards the Iraqi government to apply the Iraqi National Plan for Human Rights (NAPHR) foc using on: prisoners’ rights, economic and social rights, labour law, minority rights and the right to work and freedom to organise unions.”
Specific objective 1:
- Outcome: 16 Provincial Councils effectively and efficiently practice the notion of participation with their constituents, including civil society organizations, in making and taking decisions at the provincial levels during a-three and half year.
- Outcome: The Iraqi government adopt the 178 recommendations of the UPR by having a plan and delegate roles and commitments on the relevant parties.
- Outcome: The parliament in the Kurdish Region engages with civil society regularly and systematically to address key issues identified as important to civil society.
- Outcome: The Kurdistan Gov. prioritise rights, responds effectively and transparently to the needs and priorities of its citizens.
Specific Objective 2:
- Outcome: The human rights condition of prisoners and detainees improved throughout Iraqi prisons, detention and transforming centers to ensure the justice system in Iraq complies with the international standards and existing Iraqi laws.
- Outcome: Minorities and their rights are fully respected throughout Iraq.
- Outcome: Relevant authorities on minority issues have adequate executive powers for consensus building.
Labour rights and syndication's:
- Outcome: Change influenced in the Iraqi legislation towards increased compliance with key National Action Plan for Human Rights recommendations concerning economic and social rights, labour law and the right to work and freedom of union organization.
Project target areas: The project targets all the 18 governorates of Iraq.
Scope of the evaluation:
The purpose of the evaluation is to establish and document the impact and effectiveness of project interventions. The findings and recommendations will contribute to a learning process which will enable NPA draw lessons from its experience in order to improve the quality service to its partners and communities, assessing the extent to which the outcomes of the project have been achieved, determine relevance, efficiency, effectiveness and sustainability of the interventions/project and document new knowledge.
The evaluation should look closely at lessons learned and recommendations for future programing as the project will continue in 2020. It is also in line with NPA evaluation guidelines to conduct end of project evaluations for projects for purpose of drawing lessons to improve our practice in design and implementation of similar projects. To aid in this purpose the evaluation will therefore respond to the following objectives:
• To review the achievement of the project in terms of attaining its objectives using the evaluation criteria: relevance, effectiveness, efficiency, impact and sustainability.
• To reflect on the successes, challenges, best practices and key lessons learned during the project period.
• To reflect on project progress, achievements and change achieved.
• To reflect on the NPA model of working in local partnerships with civil society organizations to achieve project objectives.
• Suggest key practical targeted recommendations and learning to be considered by NPA for future projects and programmes in particular inform with practical recommendations the continuation and further implementation of the current project in 2020 and beyond.
Specifically, the end of project evaluation will address the following key basic evaluation questions that shall be further developed and expanded on by the consultant:
Key Review Questions
Relevance and quality of design
• To what extent was the project relevant for the political situation in Iraq with regards to the Iraqi crisis? Among questions to be answered are:
o Forming the new government and parliaments in both KRI and Iraq significantly affected any of the premises for the project?
o Has the project strategy/methodology been adapted to the political and legal contexts in Iraq?
o What has been the actual relevance on the ground of the key target groups in the project?
• What are the main activities/methods carried out by both governments and parliaments of Iraq and Kurdistan region, partners, provincial councils and civil society to mobilize influence and change, and how was the project connected with those activities and methods? Among questions to be answered are:
o What are the actual political agendas of project stakeholders on the ground in Iraq?
o What role does the project play in terms of interaction among stakeholders on the ground?
o How has the project contributed to the development of the activities/methods of the Iraqi and Kurdistan parliament, partners, provincial councils and civil society (political dialogue, networking, funding, etc), the main strengths and weaknesses of this support?
• How has the project influenced the interconnectedness of the parliament, provincial councils, partners and civil society?
• To which extent the community and stakeholders participate in planning and implementation of projects interventions?
• To what extent are the objectives of the project still valid?
Effectiveness of implementation
• To what extent did the project achieve its objectives/ outcomes? What and how much progress has been made towards achieving the overall outputs, and outcomes of the project (including contributing factors and constraints)?
• Were the inputs and strategies used effective, realistic, appropriate and adequate to achieve the project results?
• To what extent did the project’s NPA mechanism contribute in meeting project results?
• How effective were the strategies and approaches used in the implementation of the project?
• What are the recommendations in terms of effectiveness for future similar interventions?
• How satisfied are the communities with the project interventions and its results?
• What have gender-specific issues been observed and addressed through the project?
• Extent to which the outputs and results were achieved in relation to targets set in the logical/ results framework, taking into account any change in context.
• The effectiveness of the adjustments made to the programme to continuously respond to the (changing) capacity of local partners and changing context
• Were the resources used effectively and in a timely manner to produce the outputs and results are appropriate compared to the planned budget and activities?
• Were the resources effectively utilized?
• What factors contributed to implementation efficiency?
• Did the project activities overlap and duplicate other similar interventions (funded nationally and/ or by other donors?
• How efficient were the management and accountability structures of the project?
• To what extent are the objectives of the project still valid?
• What the positive and negative, intended and unintended, changes produced by the project?
• Analyse the contribution of the project to any observed impact (intended, unintended, positive, negative) and analyse what other actors and factors contributed to the impact.
• What real difference has the intervention made to the beneficiaries?
• How likely is it that any positive changes may be sustained in the short- and medium-term?
Potential for Sustainability
• What are the prospects for the benefits of the project being sustained after the funding stops particularly with reference to the local capacity (partner civil society organizations)?
• What is the partners’ capacity for management and further implementation of such projects?
• Recommendations for improving sustainability when working in partnership with local civil society actors.
• To which extent did the planning and implementation of the interventions take longer-term and interconnected problems into account?
• Did the project plan and implement an adequate transition and exit strategy that ensures longer-term positive effects and reduces risk of dependency?
• How likely will critical services and effects be sustained beyond the duration of the project?
• What were/are the major factors which influenced the achievement or non-achievement of sustainability of the project?
Partnership with local CSO
• Assess the NPA partnership model for working in partnership with local CSOs for implementing project interventions (considering NPA Partnership policy document).
• Assess the partnership process and the partnership relationship between NPA and the six local partners, identify what worked well and what did not work well and recommend possible changes.
• Assess the participation of project beneficiaries in the project (if any). implementation, monitoring and evaluation.
• Assess change in partners capacity for management and implementation of project activities.
• Identify strengths, challenges and weaknesses in the project implementation process and recommend possible changes for future learning.
• Document the lessons learned in terms of the design, implementation, and monitoring of the project, that should be applied for future projects, with consideration to NPA partnership model.
• Details of lessons learned about how change comes about, what should be done differently in the future and what needs to happen next.
The evaluation should follow a collaborative and participatory mixed methods approach that draws on both existing and new quantitative and qualitative data to answer the evaluation questions. The evaluation should follow a theory-based approach. The evaluation should combine evaluation tools based on international standards and guidelines which are OECD DAC Quality Standards for evaluating humanitarian action. It is expected that the consultant will assess the quality of the project’s impact logic and if necessary to develop a realistic impact logic based upon on the conducted interventions. The methodology design should be developed by the consultant in consideration of the information outlined in this ToR to ensure accuracy and rigour. A detailed methodology and data collection methods should be included in the technical proposal, which will be further improved in consultation with NPA team during the inception phase of the evaluation. The choice of method must also consider the needs and capacities of the different target groups and stakeholders.
Desk Review should be conducted by the consultancy firm to inform the methodology and development of the tools. In addition, the conducted desk review should cover the following documents: project proposal, baseline, mid-term evaluation, Log frame, results framework and existing documents available at NPA and among partners, sampling methodologies and secondary literature studies related to the measurement of goal and outcome level indicators. The desk review process should serve as guiding for the consultant to continue gathering resources that would enable him/her to carry out development of tools.
The document review will include NPA policies, proposal, applications and reports to donor, documentation from partners, and various forms of project documentation.
It will also include field studies to the selected locations in Kurdistan region of Iraq and other parts of Iraq. This will include field visits and interviews with NPA staff, representatives of partners, local authorities and other stakeholders.
The evaluation should have a strong learning aspect and should therefore apply participatory methods that will include various stakeholders.
The main findings should be presented at NPA Iraq country office before the final version of the report.
Key responsibilities and deliverables
a. Present to NPA Iraq Evaluation Work Plan: This document will operationalize and direct the evaluation. It will describe how the evaluation will be executed, thus refining and elaborating upon the terms of reference; it will be approved by NPA senior management and act as an agreement between parties for how the evaluation will be conducted. The work plan will include the following elements:
- Expectations of evaluation
- Roles and responsibilities
- Evaluation methodology, questions
- Evaluation framework
- Information collection and analysis methods
- Reporting formats
- Day by day work plan and time-frame for deliverables
b. Develop the tools and methodology for conducting the evaluation, using both quantitative and qualitative approaches to address the key evaluation questions listed above. The consultant will then compare the findings with baseline values, using appropriate surveys, statistical tests (relevant methodology), to draw conclusions and taking into account the qualitative information to write the report.
c. Review all of project documents (Proposal, log frame, baseline report, M&E data and partners’ reports) and relevant NPA policy document.
d. Conduct qualitative/quantitative data collection through appropriate methods ( KIIs, FGDs, etc.) and carry out data analysis (including appropriate statistical tests).
e. Prepare draft evaluation report which includes the following elements:
- A stand-alone Executive Summary
- Answered questions/ Findings
- Problems and needs
- Achievement of purpose
- Overall assessment
- Conclusions, Targeted Recommendations and Management Responses
- Annexes to the report, including data analysis report.
f. Present findings to NPA Iraq and partner staff in a briefing session.
g. Consultant receives feedback on the first draft from NPA Iraq and partner staff
h. Prepare the evaluation report incorporating all the feedback from both the document review and briefing session and submit final report to NPA Iraq.
i. All developed tools, questionnaires, reports, guidelines should be delivered to NPA in soft and hard copy.
The composition of the evaluation team is up to the consultant/firm who can choose, depending on their internal system, ideas and logic but the team members must fulfill competency criteria. NPA recommends that the team should be comprised of one Team Leader/Focal point who takes overall responsibility for this evaluation and coordinate/liaise between NPA and the evaluation team.
Supervision of the field work and quality (reliability and validity) of the data/information collected at field level is the primary responsibility of team leader. Similarly, the team Leader will work closely with NPA’s project management team. In each step and process, consultation with the NPA Programme Manager is vital.
The preliminary draft report should be submitted to NPA Iraq by 30th November 2019 the latest and the final evaluation report should be submitted to NPA by 10th December 2019.
The work is planned to commence by 1st November 2019.
Lines of communication:
The evaluators will primarily communicate with the NPA Development Programme Manager. They will also communicate with NPA MEAL Manager. NPA will facilitate the communication and contact between evaluators and local partners at the beginning of the evaluation process, however evaluation team will then have direct contact with partners.
Competency of evaluation team:
Proven expertise in design and review of programmes focused on human rights, democratization and experience in the MENA context, preference will be given to candidates that have past experience with programme design, implementation and/or evaluation of development projects in the MENA. Data analysis and writing up of evaluation reports experience is also key for this exercise. The consultant should possess extensive work experience and in depth knowledge on conducting project evaluations.
The consultant or group of consultants should have post graduate degree in development studies and human rights. Preferably, the Consultancy Company or the Individual consultant should deploy a multidisciplinary team with a background of political and statistical data analysis qualifications. It is desirable to have a post graduate in political studies. Preference will be given to those who possess past experience working with INGOs and conducting evaluation surveys.