Terms of Reference (TOR) for evaluation of project “EMPOWERMENT AND SOCIAL JUSTICE FOR ROMA WOMEN”,

2021-2024

For evaluation

Made externally

In Kosovo

ToR for EVALUATION were reviewed by M&E team?

Signature of M&E:____________

1. Description of the intervention

Context: Roma, Ashkali and Egyptians people are trapped in a vicious cycle of exclusion. They do not enjoy their rights and lack opportunity, without adequate health, social protection and care. They are not empowered to be part of societal decision making. All of these are sustained by discrimination and anti gypsyism which leads to disengagement, demotivation and disinterest - again reinforcing exclusion.

Overall Goal of the Project: The project aims to improve the rights and status of Roma women and girls in Kosovo, focusing in particular in the employment and employability. At the same time, the project increases the capacity of the local partner and civil society in gender mainstreaming.

Theory of Change: In a nutshell, the project’s theory of change is that Supported employability and increased awareness of minority rights among rights holders and duty bearers increases the self-determination and equality of Roma women in their communities and in society. The details of ToC are described below.

Roma, Ashkali and Egyptian women face multiple discrimination due to their gender, ethnicity and economic situation. Kosovo RAE are still living according to strong patriarchal rules and traditions, resulting very limited rights and opportunities for women and girls: they are only supposed to take care of the children, maintain the household, and hold together the extended family and quite often hard physical labour. RAE girls are disadvantaged as a result of their early marriage. There are Roma girls who go to school, but irregularly and even they often drop out at age of eleven or twelve. The amount of Roma girls that do not finish even primary school is 41%. Families do not value schooling, because the most important thing for a girl is to start a family and take care of it. This also leads to a poor level of employability later.

Therefore, the project aims at addressing the specific educational and employment needs of both Roma girls and Roma women. This is done by implementing some measures to the most vulnerable in both groups, but also through capacitating the authorities and other CSOs to better address and assist vulnerable Roma women in their education and employment. In addition, the project challenges CSOs and political parties to advocate for Roma inclusion.

2. Purpose and scope

Purpose: The purpose of the evaluation is to provide the project partners with external views and insights about the effectiveness, efficiency and sustainability of key project interventions and advise on possibilities on how to best build on, sharpen and prioritize them, to work as effectively as possible towards the long-term goal and outcomes of the project in next period.

The specific objective of the evaluation is to examine and answer the following questions under the programme:

• Human Rights-Based Approach (Impact):
  How has the project advanced the realization of human rights for its participants? How could the project strengthen its human rights-based approach? What specific steps should be taken to elevate the project to the next level of human-rights-based development?
• Empowerment of the RAE Community (Relevance):
  How has the project succeeded in empowering the Roma, Ashkali, and Egyptian (RAE) community? What best practices should be carried forward into the next project phase?

How has the employment rate progressed within the RAE communities because of the constant work of the project (together with the employment agencies)? What is the impact of the project's intervention?

• Sustainability of Livelihood and Skills Development (Sustainability):
  How sustainable are the results achieved together with the employment agency under the strategic priority of livelihood and skills development for the most marginalized? What lessons have been learned, and how can these inform future initiatives?
• Disability Inclusion (Effectiveness):
  How can the project improve the integration of disability inclusion in the next project phase? What measures can be taken to strengthen the incorporation of cross-cutting objectives, particularly regarding disability inclusion?

Target audience: project partners, HDF, donors and in addition, information can be shared with other civil society actors and relevant line ministries of the Kosovo Government.

The purpose of this evaluation is for both accountability and learning purposes that will in turn serve for strategic project steering. However, given the potential application of the findings for the next project phase, learning is prioritized in the evaluation processes. Therefore, the evaluation is mainly formative and the results will be used to develop the next project phase.

Evidence Base: The evaluation shall take into account existing evidence on social inclusion of Roma and other vulnerable groups in Kosovo, which is available through the yearly governmental reports on the implementation of the National Strategy for the Inclusion of Roma and Ashkali, studies conducted by other donors, the MICS from 2020 (which was used for some baseline data by the project) as well as the Anti gypsyism Baseline Study conducted in 2021.

3. criteria and questions

The evaluation questions were developed by the project team and address the DAC criteria of effectiveness, efficiency and sustainability/

4. Methodology and process

The overall approach is expected to be based mainly on a mix of qualitative methods for data collection and analysis but making strategic use of internally and externally available quantitative data for the analysis and interpretation. The evaluation can build on existing project monitoring data.

The evaluation team is expected to use methods such as: analysis of documents and of existing statistical data and budgets, structured interviews, semi-structured interviews face-to face or by phone, focus group discussions (FDGs), etc.

Selection strategies for the different project components are to be defined as part of the inception phase.

The evaluation will be divided in three components with thematical focus requiring specific qualifications. The project consortium encourages applicants to put together a team or to jointly apply with other consultants so that all qualifications can be covered. In case two or more consultants apply together, coordination needs to be clarified.

The process should follow the following steps:

Contract and Kick-off meeting: Signing of contract discussion of the assignment with the Consortium partners. First documents, including available data, are provided to the evaluation team.

Inception phase: forms part of reaching final agreement on methodological tools and procedures, including selection of interview partners / sampling strategy. Includes desk study and key informant interviews (by phone/skype) with key project staff, and possibly additional external stakeholders to be agreed on in the kick-off meeting.

A template for an inception report, including an evaluation matrix, is attached as annex 2. The inception report will be reviewed by all consortium partners.

Field visits and consultation workshop: Primary data sources and secondary data sources are used to generate necessary information for the evaluation. Conducting primary data collection using various methods defined in the inception report. Secondary data is generated from respective project documents, project data basis and external key documents. It is expected that the evaluation results will be gender sensitive, and that gender issues will be looked at along exclusion issues (intersectionality).

The evaluators will organize one-day workshop where key findings and first conclusions should be discussed with key staff of the consortium and SDC.

Report writing: Submission of draft final report, inclusion of comments by consortium. The subsequent drafting process may go through additional feedback rounds until the final report is approved. The conclusions have to address the evaluation questions in the TOR, taking into account also unexpected finding. It is expected that the evaluation/review team will present concrete recommendations which are addressed to the specific stakeholders.

Data triangulation and quality control are very important and need to be discussed in the inception report. The field visits will only take place upon official approval of the inception report by the contractor.

The following table shows the evaluation questions and specifies to which project outcome (and in some cases outputs) each question refers. The methodology for addressing each cluster of questions should be developed by the evaluation team as part of the inception phase.

5. Deliverables

The evaluator(s) is expected to produce the following deliverables:

• An inception report containing the evaluation design and design methodology (including attached research framework/evaluation matrix)).
• Data collection tools
• A final evaluation report responding to the evaluation objectives and questions; its structure will be outlined and agreed upon as part of the inception report. The final report shall have maximum 25 pages without annexes
• Final PowerPoint presentation which summarizes the content of the evaluation report.
• Full data analysis
• Raw data (questionnaires, documentation of interviews, lists, data base)

6. Schedule and budget

The bidder is free to propose the number of days he/she needs for the evaluation; however, we wish to recommend that the total working days for the whole should not exceed 30 days in total. If the different assignments are done by the same evaluator, it is suggested that respective field phases for the different assignments are conducted in parallel by different team members.

The consultant is expected to submit his / her daily rate, including VAT (if applicable), as part of the application. The consultant may apply together with a local/international partner/consultant. In this case, the daily rates of all team members should be indicated. Accommodation, travel and related costs will be reimbursed.

7. Management roles and responsibilities

The evaluation is mandated by the project partners – VoRAE and Helsinki Deaconess Foundation. The overall coordination of the contract is in the hands of VoRAE as an implementing partner. For VoRAE, the responsible person is Vesa Batalli, the Program Director, for Helsinki Deaconess Foundation is Johanna Tuominen.

The partners will assign a contact focal point to liaise with the selected consultant. This person will be responsible for:

• Management of communications between the project partners and the consultant and between the consultant and partners-beneficiaries.
• Organization and provision of the internal documentation required by the consultant.
• Logistic arrangements regarding the organization of field work.
• Approve the deliverables in consultation with project partners
• Provide feedback of project partners on every aspect of the evaluation, and they will provide the consultant with all required documents. Provide the consultant with any necessary logistical support (transport, lodging, etc.).

The consultant has responsibility for:

• Preparing the overall evaluation design,
• Propose a methodology and tools (in consultation with all stakeholders),
• Suggest selection and sampling procedures.
• Develop the data collection tools,
• Plan the data collection, brief the local team,
• Conduct the data collection, analyze all data, and write the reports (inception, final).

The consortium will:

• Provide the evaluator with all necessary project documents and organize a project presentation/briefing with key project staff.
• Collect comprehensive background documentation and inform partners and selected project counterparts
• Provide a list of key stakeholders and their contact details
• The program staff members will be responsible for liaising with partners, logistical backstopping and providing relevant documentation and feedback to the evaluation team

8. Follow up of the assessment

Upon finalization of the evaluation report, a management response will be developed by the Project partners. The management response will respond to the recommendations and outline a plan of action for implementing accepted recommendations from the report.

As part of the evaluation process, the consultant is expected to facilitate a feedback and reflection workshop for the Project Partners. Results and findings from the evaluation will be used in the development of the next project phase and will be shared with the project design team for consolidation.

9. List of documents

• Project proposal
• Log-Frame
• Overall Budget
• Annual Planning and Annual Reports
• National Strategy for Inclusion of Roma and Ashkali in Kosovo Society 2022-2024
• Govern. Reports on Implementation of the Strategy of Roma and Ashkali Inclusion
• World Bank/UNDP Roma Survey Report
• Agendas and reports of important events
• MICS Study 2019 (which was used for some baseline data by the project)
• Anti gypsyism Baseline Study conducted in 2021

10. Assessment team / qualifications

The bidder can apply individually or propose an evaluation team qualified for evaluating the questions according to the, of skills and experience mix to adequately deliver on these ToRs.

• Skills and experiences in at least three project evaluations in relevant thematic topic (Roma inclusion with a focus on addressing labour market inclusion, education and anti gypsyism.)
• Proven experience in the collection and analysis of both quantitative and qualitative data
• Post-graduate degree in Social Sciences, International Development, Development Studies, Social Welfare, Social Policy, Social Work or related fields, Human Rights Studies, Economic Development Studies.
• Demonstrated knowledge of the political, cultural, and economic situation in the Eastern Europe region with a specific focus on social inclusion for Roma programmes
• Extensive knowledge of local government structures and programmes in Kosovo.
• A deep understanding of social services system in Kosovo.
• Proven expertise in the evaluation of projects.
• Selected consultants must not have been involved in the design, implementation, or monitoring of this project.

11. How to Apply

Interested candidates should present:

• A technical offer (max. 5 pages, plus annexes) comprising:
• Proven understanding of the objective of the evaluation and the Terms of Reference (ToR);
• Key questions of the evaluation
• Brief description of the methodology and tools proposed to answer each of the key questions;
• A chronogram showing details for the realization of each of the evaluation phases, based on the above table (steps to be adapted as necessary, submission of final report no later than Mid November 2024).
• Distribution of tasks if application as a team
• A financial offer with a detailed cost breakdown as necessary.
• An updated CV (of both team members, in case the proposal entails a pair submitting)
• At least two evaluation reports in similar fields
• Contacts of 3 references

Applicants should send the complete documentation in electronic form to [email protected] (cc to [email protected] and [email protected] ) by 13rd of September 2024.