Photo by Markus Winkler on Unsplash
Evaluation of a Multi-National Conflict, Justice and Rehabilitation Program
Client
European Institute of Peace
Countries
Syria, Yemen, Ethiopia, Iraq, Netherlands
Thematic Area
Livelihoods, Peacebuilding & Social Cohesion
Services
End-to-end research, assessments and evaluations
Methods
Key informant interviews, Desk research / Literature review, Validation workshop / survey
Start Date
June 1, 2024
Team
Sanghamitra Mazumdar
Context
The European Institute of Peace (EIP) is an independent organization dedicated to advancing sustainable peace through mediation, dialogue, and conflict resolution. It leads the Conflict, Justice, and Rehabilitation (CJR) program, which aims to strengthen justice and rehabilitation efforts in conflict-affected contexts by supporting locally driven solutions. The program operates in Iraq, Syria, Yemen, and Ethiopia, focusing on transitional justice, reintegration and inclusive governance.
EIP sought to assess the effectiveness of its CJR program, specifically its responsiveness to shifting political dynamics and contribution to sustainable justice and reintegration efforts.
The Task
EIP commissioned Nurai Global to conduct an interim evaluation of its long-term CJR program 2017-2024, including:
a formative, learning-focused review of program design and execution spanning 2021-2024,
assessing the effectiveness of strategies in addressing urgent justice and peacebuilding needs in focus countries,
developing practical recommendations for improving programme delivery and designing future interventions.
Our Approach
We applied a qualitative, multi-method approach, including a desk review of project documentation, Theory of Change validation, in-depth interviews with project stakeholders, and an online validation survey reaching wider stakeholders. The evaluation grouped learning and potential improvements into two areas: Design and Execution. It assessed the validity of the Theory of Change, the alignment of activities, outputs and outcomes, and the efficiency and effectiveness of delivery processes.Through this, we identified key factors driving success and areas for improvement. A critical component was our assumption-testing methodology, which examined whether project activities effectively translated into meaningful outcomes.
Execution
The evaluation covered the last 30 months of program delivery, prioritizing results and identifying factors contributing to successes and impediments.
To effectively respond within the client’s scope within the available timeline, instead of assessing output achievements, we pivoted to include an independent synopsis based on project documents and progress reports. Assumption testing was crucial for examining the relationship between activities and outcomes, using Theory of Change maps and interviews with key respondents to prompt reflections on approaches and assumptions.
Data was collected through semi-structured interviews with 46 respondents, including donor representatives, EIP senior experts, diplomatic officers and representatives (formal and informal diplomatic platforms), and community members.
Data analysis utilized Nurai’s proprietary QualApp for insights and thematic synthesis, applying a framework matrix approach. An online validation survey, circulated after report submission, supplemented findings with stakeholder feedback.
Conducting this evaluation presented some challenges:
Sensitive environments made data collection difficult, particularly when engaging high-level stakeholders whose responses risked misrepresentation.
Language barriers required careful management to ensure accurate data collection.
Security risks in conflict zones limited movement, requiring strategic planning to minimize risks for both researchers and participants.
To address the above challenges, we:
Ensured interviews with high-level stakeholders were conducted by senior analysts, mitigating the risk of misrepresentation.
Engaged in-country researchers, shadowing them where necessary to maintain data accuracy and address language barriers.
Selected researchers with expertise in conflict-affected areas, ensuring they had the local knowledge and contextual awareness needed to navigate sensitive discussions.
Minimized field movement and provided adequate security measures, reducing risks for our teams while maintaining robust data collection.
Implemented a rigorous Data Quality Assurance process, including close review, coaching, and performance management, ensuring accuracy even though repeat data collection was not possible.
The Results
The evaluation provided critical insights into the effectiveness of EIP’s CJR program, highlighting its strengths in addressing justice and rehabilitation needs while identifying areas for improvement. The desk review of project documentation systematically highlighted important and diverse quantitative achievements across all focus countries. A key strength of this evaluation was its focus on systems change, engaging a wide range of stakeholders—from grassroots community members to policymakers—to uncover how local and institutional efforts intersect to drive justice and reintegration.
Findings showed specific factors which allowed the program to successfully adapt to shifting political dynamics - fostering trust among local actors and enhancing reintegration efforts. Stakeholder feedback also pointed to ways in which long-term sustainability strategies could be assured, for each country. Nurai’s rigorous data quality processes ensured credible findings, enabling EIP to further refine its interventions in the future.
Client Review
"We had a complex assignment for an interim evaluation focused on learning - a multi-country portfolio with a tight deadline - which Nurai delivered effectively. They were responsive to our needs, adapted to necessary changes, and remained communicative and collaborative throughout the assignment. I was particularly impressed with their capacity to conduct data collection quickly and efficiently in parallel across several countries and contexts, while delivering a coherent and timely final report."
- Senior Programme Manager, European Institute of Peace
"The team has particularly appreciated the outstanding professionalism, flexibility, and continuous support provided throughout the evaluation process. Nurai’s commitment to understanding the projects in depth, offering constructive feedback, and maintaining clear and consistent communication has been invaluable. The work was well structured, and the lead researcher ensured that we were kept informed at every stage, delivering quality results on time. We also value the openness of the Nurai team to ongoing dialogue and feedback."
- Programme Assistant, European Institute of Peace