Call for Proposals: External Evaluation – Project Social Justice for Roma, Ashkali, and Egyptians in Kosovo (SORAE)

HEKS/EPER (Swiss Church Aid), on behalf of the project Consortium comprising VoRAE (Voice of Roma, Ashkali and Egyptians) and Terre des Hommes Kosovo, is seeking applications for an external evaluation of the project "Social Justice for Roma, Ashkali, and Egyptians in Kosovo", Phase II (2023-2025). This evaluation will assess the project’s effectiveness and sustainability on anti-discrimination, social inclusion, and economic empowerment for marginalized communities in Kosovo.

Proposals are welcome from both individual consultants and evaluation teams with expertise in minority rights, social inclusion, and policy evaluation. The selected consultant(s) will collaborate closely with HEKS/EPER and the Consortium to evaluate project outcomes and offer strategic recommendations for future project phases.

Interested applicants should submit their technical and financial proposals by December 6, 2024 to [email protected] . For full details on scope, deliverables, and application guidelines, please refer to the Terms of Reference below.

Terms of Reference (TOR) for evaluation of project “Social Justice for Roma, Ashkali, Egyptians”,

Phase II 2023-2025

Description of the intervention

Context:

Roma, Ashkali and Egyptian communities in Kosovo are trapped in a vicious cycle of exclusion. They do not enjoy their rights and generally lack opportunities, without adequate health, social protection and care. They are not empowered to be part of societal decision making. All of these are sustained by discrimination and antigypsyism which leads to disengagement, demotivation and disinterest - again reinforcing exclusion.

Overall Goal of the Project:

The overall goal of the Social Justice for Roma, Ashkali, and Egyptians in Kosovo project is to promote a multi-ethnic and inclusive society in Kosovo by addressing social injustices faced by these communities. The project aims to improve the legal and institutional framework, enhance awareness of and action against antigypsyism, and foster social inclusion to ensure equal opportunities for these groups. Key objectives include empowering community members, particularly women and youth, to engage in political, economic, and civic activities, and improving access to services and employment opportunities. Building on the progress of Phase I, the project seeks to consolidate mechanisms, improve policies, and create pathways from education to employment. By promoting systemic change, advocacy, and capacity building, the project envisions a society where Roma, Ashkali, and Egyptians can exercise their rights, contribute equally, and trust in state institutions.

Theory of Change:

IF the policies and institutional framework for social and economic inclusion of Roma, Ashkali and Egyptian people in Kosovo are strengthened; and if antigypsyism is addressed institutionally, and the mentioned communities are empowered to recognise manifestations of antigypsyism and act upon them; and if integrated actions for social inclusion – of women and children in particular – are implemented, THEN the integrated actions towards social inclusion will become more effective, the quality of life of Roma, Ashkali and Egyptian women and men in Kosovo will improve, they will all be treated as equals by state and society and will be in a position to exercise the same opportunities and rights as everybody else.

The following outcomes were defined for the project:

Outcome 1: Policies and practices for social inclusion and equality are better integrated into the legal and institutional framework of Kosovo, their implementation improved, and quality is monitored, so that they can positively impact the lives of Roma, Ashkali and Egyptian people.

Outcome 2: Service providers, the media and civil society are more aware about and take action against antigypsyism and its manifestations, and Roma, Ashkali and Egyptian people, especially women, are empowered to report cases, take action and strengthen their trust in institutions;

Outcome 3: Integrated actions for social inclusion and equality improve the quality of life of Roma, Ashkali and Egyptian people in Kosovo, and the barriers to their participation in economic, political and civic life are overcome, especially for youth and women.

Project Set-Up:

The project "Social Justice for Roma, Ashkali, and Egyptians in Kosovo (Phase II)" project is implemented by a Consortium led by HEKS/EPER, along with VoRAE (Voice of Roma, Ashkali and Egyptians) and Foundation Terre des hommes Lausanne in Kosovo (TdHK)[1]. Following a first phase completed in 2022, the current project phase runs from 2023 to 2025, aiming to address social exclusion, discrimination, and barriers faced by these marginalized communities in Kosovo.

The project follows a multi-faceted strategy focusing on policy dialogue, advocacy, community mobilization, and mainstreaming successful practices. Key areas include improving social protection through legal reforms, addressing housing issues with a settlement transformation approach, and sensitizing service providers and the media to the manifestations of antigypsyism. The implementation approach centers on policy influence, innovative social inclusion actions, capacity building, and knowledge transfer on local and regional levels.

The Consortium partners play specific roles: HEKS/EPER coordinates the project and oversees financial management, VoRAE focuses on community-based actions, and TdHK contributes expertise in child protection and advocacy. Regular meetings and a steering committee ensure efficient coordination among stakeholders.

The project’s core principles include a Human Rights-Based Approach (HRBA), gender equality, good governance, and an explicit focus on combating antigypsyism. The Project aims for systemic change by embedding interventions within Kosovo’s public policy framework, ensuring the sustainability of efforts beyond the Project’s timeline.

In addition to the HEKS Project Manager as the Consortium lead and coordination, the Directors and teams of each organisation play a critical role. It is their inputs that ensure the Project’s influence and outcomes extend from grass-roots level in communities up to the level of national authorities, international agencies and private sector businesses in Kosovo and beyond in the wider European context.

A project steering committee consisting of representatives of the project partners (VoRAE, HEKS/EPER, TdHK, Office of Good Governance and Swiss Agency for Development and Cooperation (SDC) meets twice a year. The Consortium leadership and implementation teams meet regularly to discuss challenges and opportunities to project implementation and take decisions.

Purpose and scope

Purpose:

The purpose of this evaluation is for both accountability and learning purposes that will in turn serve for strategic project steering. The results will be used to measure the effects of the project to date and help developing the next project phase.

The evaluation will assess the sustainability and effectiveness of selected Project interventions and advise on possibilities on how to best ensure sustainability of project benefits and how to work effectively towards the long-term goal and outcomes of the Project in the coming phase.

Building on the internal mid-term review of the project conducted in June 2024 and given the broad scope of the project, the Consortium has decided to focus the external evaluation on four intervention areas that present specific sustainability challenges:

1. Institutionalized Practices from previous phase

1.1 Mainstreaming Educational Support: The project concluded mainstreaming efforts in education, resulting in municipalities taking over the operational responsibility of 24 Learning Centres. Additionally, the National Scholarship Program for secondary school students was institutionalized.
1.2 Case Management Roundtables (CMRs): 33 Case Management Roundtables were handed over to local municipalities, integrated into the child protection framework, and their existence codified within the new Child Protection Law.
1.3 Antigypsyism Institutional Mechanisms: The project established a National Platform for Protection from Discrimination for the Roma, Ashkali and Egyptian Communities managed by the Office of Good Governance and an Interinstitutional Team for the promotion of employment for the same communities.

2. Policy Dialogue and Legislative Reform

This area includes three interventions. The Project is actively involved in the amendment of the Law on Anti-Discrimination to specifically address anti-gypsyism. This intervention includes policy dialogue with the government and advocacy to recognize and combat this form of racism. The second intervention focuses on revising the Law on Social and Family Services to expand social protections, improve service delivery, and introduce new community-based provisions. The third intervention supports advocacy for a new Law on Social and Sustainable Housing, emphasizing the improvement of informal settlements and integrating a legal framework for sustainable housing.

3. Newer Interventions

3.1 Settlement Transformation Pilot intervention designed to address issues related to housing and settlement conditions for Roma, Ashkali and Egyptian communities by providing a sustainable and replicable model for legalizing settlements, improving housing conditions of marginalized communities and prompting desegregation through demonstrating a model for desegregated and secure housing, promoting community participation in planning and designing the settlement transformation, collaborating with stakeholders to address the legal complexities around land ownership and informal settlements and improving economic opportunities by training and building capacity of Roma, Ashkali and Egyptians construction workers and companies in the process.

3.2 Leadership and Confidence Building for Women: Training programs have been organized to promote gender equality, entrepreneurship, and advocacy, including encouraging men to participate in social meetings to challenge traditional roles. This is designed to change traditional gender roles and increase women's involvement in decision-making.

3.3 Access to Employment

3.3.1. The “My First Job program of training and support for young Roma, Ashkali and Egyptians focuses on ICT skills development, self-confidence training and collaboration with private companies to promote job placements and retention.

3.3.2. Project’s efforts to support the government’s commitment to employment quotas for minority communities to improve public sector inclusivity and reduce workplace discrimination

3.3.3. Project’s collaboration with chambers of commerce and private companies to foster inclusive hiring practices and tackle workplace prejudice.

4. Anti-Gypsyism:

The objective of the anti-gypsyism component is to combat prejudice and systemic discrimination against Roma, Ashkali, and Egyptian communities. Key activities include;

4.1. Building a social movement through community-led actions supported by a Small Grant Scheme which supports local initiatives promoting civil rights and community empowerment.
4.2. Empowering local CSOs to participate in and monitor relevant government policies
4.3. Promoting ethical and correct media reporting
4.4. Training service providers to recognize and address anti-gypsyism manifestations and biases.
4.5. Empowering communities to recognize and report cases of discrimination
4.6.Mobilizing support from influential individuals Promoting public multicultural events to foster understanding between communities
4.7. Revising school curricula and textbooks to address negative stereotypes and promote inclusivity.

In addition to assessing effectiveness and sustainability of the selected intervention areas, the Evaluator(s)/Evaluation Team should analyze linkages and connections between all the selected intervention areas.

Target audience

Project Consortium, SDC, major back-donors; in addition, information can be shared with other civil society actors and relevant line ministries of the Kosovo Government.

1. Criteria and questions

The selected intervention areas will be evaluated on the basis of the OECD-DAC criteria of effectiveness and sustainability with the purpose to advise on how to best ensure sustainability of project benefits and how to work effectively towards the long-term goal and outcomes of the project in the coming phase. Therefore, the evaluation will focus on whether products and processes created by the project are being maintained and integrated into institutional structures and legal framework or taken over by non-institutional stakeholders and project participants through an ownership process. This includes assessing whether advocacy, technical support, training, media campaigns, awareness-raising and other initiatives are leading to measurable changes in attitudes and practices, and whether these changes are sustainable.​

The effectiveness and sustainability questions in Annex I were developed by the Consortium. They aim at providing a strategic blueprint for the evaluation including:

• Designing Methodologies: The questions inform the selection of the overall evaluation design and the data collection tools (e.g., surveys, interviews, focus group discussions) and analysis frameworks.
• Framing the Evaluation: The questions indicate the thematic emphasis so that the evaluation matrix or strategy as well as findings are organized around the key themes and objectives laid out in the ToR.
• Ensuring Consistency and Relevance: the questions aim that the evaluation remains consistent with the project’s goals and relevant to its stakeholders, including the Project Consortium, donors, and government bodies.

1. Methodology and approach

Evaluation Design and Research Methods

The Evaluator(s) / Evaluation Team are expected to use a variety of research tools, including participatory methods that involve key stakeholders as respondents. These methods may include structured and semi-structured interviews (conducted face-to-face or by phone), focus group discussions, surveys, and the analysis of existing statistical data and budgets. The Evaluator(s) / Evaluation Team should propose the most appropriate methods for addressing the evaluation questions and plan to triangulate qualitative data with quantitative trends and findings derived from existing project monitoring data.

Document Review

The Evaluator(s) / Evaluation Team are encouraged to consult a wide range of internal and external background documents related to the program.

Essential documents include:

1) The agreement for the Project award signed between the Consortium and SDC.

2) Annual performance reports submitted by the Consortium to the donor, detailing Project results.

4) Program reports, write-ups, and any special monitoring and evaluation data.

5) Relevant government documents such as the National Strategy for the inclusion of Roma and Ashkali communities in Kosovo society - 2022-2026 and reports on its implementation

Additional documentation will be provided to the selected consultant during the inception phase.

Evaluation Management

This evaluation will be managed by the Consortium, with the Evaluator(s) / Evaluation Team working closely with senior management staff, including the Head of Country Office/Directors, Program Managers, and the M&E Officer. Program Managers will provide technical insights into program structure and implementation, while Project Officers will assist with logistics, arranging meetings, lodging, and other practical matters. The evaluation team will be free to draw their conclusions independently, ensuring that findings are objective and free from organizational or political influence.

The evaluation will be divided in four components with thematic focus requiring specific qualifications. The Project Consortium encourages applicants to put together a team or to jointly apply with other consultants so that all qualifications can be covered. In case two or more consultants apply together, coordination needs to be clarified.

The Evaluator(s) / Evaluation Team will conduct a quality check on a sample of existing project data to assess its suitability for the evaluation. This sampling-based review will focus on key aspects such as completeness, accuracy, relevance, and consistency, while identifying any notable biases. Based on this quality check, the consultant will decide which data can be included in the evaluation.

Process

The process should follow the following steps:

Contract and Kick-off meeting: Signing of contract discussion of the assignment with the Consortium partners. First documents, including available data, are provided to the Evaluator(s) / Evaluation Team.

Inception phase: forms part of reaching final agreement on the overall evaluation design, the evaluation approach, and the tools, as well as the sampling strategy and the selection of interview partners. Includes desk study and key informant interviews (by phone/skype) with key project staff, and possibly additional external stakeholders to be agreed on in the kick-off meeting. The inception report will be reviewed by all Consortium partners.

Field visits and consultation workshop: Primary data sources and secondary data sources are used to generate necessary information for the evaluation. Conducting primary data collection using various methods defined in the inception report. Secondary data is generated from respective project documents, project data basis and external key documents. It is expected that the evaluation results will be gender sensitive, and that gender issues will be looked at along exclusion issues (intersectionality).

The Evaluator(s) / Evaluation Team will organize one-day workshop where key findings and first conclusions should be discussed with key staff of the consortium and SDC.

Report writing: Submission of draft final report, inclusion of comments by consortium. The subsequent drafting process may go through additional feedback rounds until the final report is approved. The conclusions have to address the evaluation questions in the ToR, taking into account also unexpected finding. It is expected that the evaluation/review team will present concrete recommendations which are addressed to the specific stakeholders.

Data triangulation and quality control are very important and need to be discussed in the inception report. The field visits will only take place upon official approval of the inception report by the contractor.

Deliverables

The Evaluator(s) / Evaluation Team are expected to produce the following deliverables:

• An inception report containing the evaluation design and design methodology (including attached inception report/evaluation matrix)).
• Data collection tools
• A final evaluation report responding to the evaluation objectives and questions; its structure will be outlined and agreed upon as part of the inception report. The final report shall have maximum 25 pages without annexes
• Final PowerPoint presentation which summarizes the content of the evaluation report.
• Full data analysis
• Raw data (quantitative data / database, transcripts of interviews, etc.)

Schedule and budget

Applicant(s) are free to propose the number of days they need for the evaluation; however, it is recommended that the total working days for the whole evaluation should not exceed 30 days. If the different assignments are done by an evaluation team, it is suggested that respective field phases for the different assignments are conducted in parallel by different team members.

Management roles and responsibilities

The evaluation is mandated by the project Consortium – HEKS/EPER, VoRAE, and Terre des Hommes Kosovo. The overall coordination of the contract is in the hands of HEKS/EPER as a lead Consortium member. The responsible person is Riccardo Lepri, HEKS/EPER Project Manager. For VoRAE, the responsible person is Isak Skenderi, the Executive Director and for Terre des Hommes Kosovo, the responsible person is Pajtim Zeqiri, Head of Country Office.

The consortium will assign a contact focal point to liaise with the selected Evaluator(s) / Evaluation Team. This person will be responsible for:

• Management of communications between the consortium and the consultant and between the consultant and partners-beneficiaries.
• Organization and provision of the internal documentation required by the consultant.
• Logistic arrangements regarding the organization of fieldwork.
• Approve the deliverables in consultation with consortium members and notify to the Consortium lead to process the installments associated to each of them.
• Provide feedback of Consortium on every aspect of the evaluation,

The Evaluator(s) / Evaluation Team has responsibility for:

• Preparing the overall evaluation design,
• Propose a methodology and tools (in consultation with the Consortium),
• Suggest selection and sampling procedures
• Develop the data collection tools, plan the data collection, brief the local team, conduct the data collection, analyse all data, and write the reports (inception, final).

The consortium will:

• Provide the evaluator with all necessary project documents and organise a project presentation/briefing with key project staff.
• Collect comprehensive background documentation and inform partners and selected project counterparts
• Provide a list of key stakeholders and their contact details
• The program staff members will be responsible for liaising with partners, logistical backstopping and providing relevant documentation and feedback to the evaluation team

Follow up of the EVALUATION

Upon finalization of the evaluation report, a management response will be developed by the Consortium. The management response will respond to the recommendations and outline a plan of action for implementing accepted recommendations from the report.

As part of the evaluation process, the Evaluator(s) / Evaluation Team is expected to facilitate a feedback and reflection workshop for the Consortium and SDC. Results and findings from the evaluation will be used in the development of the next project phase and will be shared with the project design team for consolidation.

EVALUATION TEAM qualifications

The bidder can either apply for one or two assignments of this evaluation or propose an evaluation team qualified for evaluating all four assignments. In case separate bidders are selected for individual assignments, the evaluators are expected to coordinate how they can cooperate.

The Evaluator(s) / Evaluation Team should be able to comply with the following list of required qualifications, competencies and experiences:

Methodological Expertise in Evaluation Design and Approach:

• Evaluation Approach: Advanced experience with diverse evaluation approaches, including but not limited to Contribution Analysis, Outcome Harvesting, Most Significant Change, and Process Tracing, and the capacity to justify the selection based on project goals.
• Mixed-Methods Proficiency: Demonstrated expertise in both quantitative and qualitative methods for data collection and analysis, ensuring comprehensive and triangulated insights. Must be able to design and implement surveys, interviews, focus groups, and participatory methods.

Technical Expertise in Thematic Areas:

• Social Inclusion and Antidiscrimination: Knowledge of issues affecting Roma, Ashkali, and Egyptian communities, particularly around social inclusion, access to services, employment, and systemic discrimination.
• Policy and Legislative Evaluation: Familiarity with policy and legislative evaluation, especially related to social justice and minority rights, and understanding of Kosovo’s governmental and legal frameworks.

Experience in Project Evaluation:

• At least three completed evaluations in thematic areas relevant to the project, such as Roma inclusion, labor market integration, child protection, urban planning, gender equity, or similar fields.
• Experience in evaluating multi-stakeholder projects and working with consortiums, particularly in transitional or post-conflict settings in Eastern Europe.

Data Collection and Field Experience:

• Proven skills in planning, coordinating, and conducting field evaluations in complex or challenging environments, with a focus on sensitive issues involving marginalized communities.
• Capacity to adapt methods and tools in response to field conditions, ensuring data quality and respecting cultural and ethical considerations.

Reporting and Communication Skills:

• Strong ability to synthesize complex findings into clear, actionable recommendations for diverse audiences, including project consortium, donors, government agencies, and local community representatives.
• Experience facilitating participatory workshops and feedback sessions, with sensitivity to the perspectives and experiences of the target communities.

Bidders must not have been involved in the design, implementation, or monitoring of this project.

11. How to Apply

Interested candidates should present proposals consisting of a Technical Offer, a Financial Offer, CVs of consultant(s) or team members, two past evaluation reports and references. No alternative formats will be accepted.

• The technical offer (max. 5 pages, plus annexes) shall include:
• Methodological Reflection (1 Page) on how the applicant would approach the methodology for a complex evaluation like this. This should include potential evaluation designs they might consider (e.g., Theory-based, Realist), possible evaluation approaches (e.g., Contribution Analysis, Outcome Harvesting) and their initial thoughts on selecting data collection and analysis tools (both quantitative and qualitative).
• Brief description of the methodology and tools proposed to answer each of the key questions;
• A timeline showing details for the realization of each of the evaluation phases, based on the above table;
• Distribution of tasks if application as a team.
• A financial offer with a detailed cost breakdown as necessary.
• An updated one-page CV (of all team members, in case the proposal entails a team submitting)
• At least two evaluation reports in similar fields
• Contacts of 3 references

Applicants should send the complete documentation in electronic form to Riccardo Lepri ([email protected] ) by 6 December 2024 COB. All documentation should be submitted electronically, using formats compatible with MS Word, Excel, and PDF, ensuring compliance with the 2MB limit per email.

Only applicant(s) shortlisted for an interview will be contacted.

Annex I

The following tables show the evaluation questions. The methodology for addressing each cluster of questions should be developed by the evaluation team as part of the inception phase.

Evaluation Questions Assignment 1: Institutionalized Practices from Previous Phase (Mainstreaming Educational Support, Case Management Roundtables, Anti-gypsyism Institutional Mechanisms)

Evaluation Question Assignment 2: Policy Dialogue and Legal `Reforms

Evaluation Questions Assignement 3: Newer Interventions (Settlement Transformation, Leadership and Confidence-building for Women, Access to Employment, Child Marriage)

Evaluation Questions Assignement 4: Anti-gypsyism

[1] Homepage EN | HEKS, VoRAE and Welcome | Terre des hommes