Country: Myanmar
Closing date: 05 Jun 2016
TERMS OF REFERENCE
External independent final Evaluation
Reference:
EuropeAid/132-894/L/ACT/MM – Aid to Uprooted People, Myanmar
Title of the action:
”Improved Access to and Utilization of Health Services, Water and Sanitation Facilities, and Rights Protection for Uprooted People in Kayin State**”**
I.Introduction and Context
Malteser International is a non-governmental worldwide relief organisation of the Sovereign Order of Malta for humanitarian aid. The organisation has almost 60 years of experience in humanitarian relief and recovery and actually covers over 100 projects in more than 20 countries in Africa, Asia and the Americas. In Myanmar, Malteser International has been operational since 2001 and is currently implementing health, WASH (water, sanitation, hygiene) and DRR (disaster risk reduction) programmes in, Rakhine, Shan and Kayin States. The programmes are implemented through Programme Offices in the respective regions and supported by a Country Office in Yangon.
For over 60 years, Kayin State has been experiencing civil war and conflict until recent ceasefire agreements were signed, between non state armed groups like the Karen National Union (KNU) and the current Government of the Republic of the Union of Myanmar (GoRUM) in January 2012 and October 2015, and finally bringing a fragile peace to the area.
The protracted conflict resulted in hundreds of thousands of internally displaced persons and refugees along the Thai-Myanmar border as well as in a significant underinvestment in public services in the conflict affected areas in Kayin State. The recent ceasefires and commencement of a peace process have been perceived positively as a step towards national reconciliation by most of the people in the country and the international community.
Public health care services in Kayin State are poor in quality due to limited availability and access to health facilities, chronic shortages in drug supplies, unaffordable treatment and associated costs and limited technical health staff capacity. The health status of the population remains precarious with malaria, diarrhoea, acute respiratory tract infection and tuberculosis as leading factors of morbidity and mortality. Access to reproductive health services including safe delivery remains severely constrained. Access to primary health care services is also limited by the bad road conditions, mainly during rainy season.
Limited access to safe drinking water and sanitation in villages and schools remains a further area which results in impaired health of the target population. Consultations with communities in these areas identified access to systematic health care services as well as all-year-around access to safe water as high priority. Both are considered fundamental for recovery from years of displacement, to encourage displaced families to return and allow communities to reform.
Over the years both sides of the conflict have contributed to the vulnerability and impoverishment of local populations and have undermined uprooted peoples' recovery efforts and coping strategies. Up to 90% of the population in areas not fully under government control does not have identification documents and are therefore very restricted in their movements and access to government services.
During the implementation of the project activities the following studies and supervision field visit were done:
- Base line survey - External consultant. June 2013
- Mid Term Evaluation - External Consultant. November 2014
- ROM Mission – EU Consultant. July 2015
- Monitoring field visit from the Myanmar EU delegation. December 2015
- End line survey - External consultant. May 2016.
Malteser International and ADRA Myanmar jointly address these limitations through the Europe Aid-funded Aid for Uprooted People (AUP) Project:
Project: ”Improved Access to and Utilization of Health Services, Water and Sanitation Facilities, and Rights Protection for Uprooted People in Kayin State”
Project Period: 01 February 2013 – 31 July 2016
Project Budget: 3,130,000 Euro
Donor: EuropeAid (Grant contract DCI-ASIE/2012/308-772)
Target area: 84 villages in Hpa An and Hlaingbwe Township (approx. 80,000 persons/13,000)
Project Objective:
To improve the living conditions of the highly vulnerable population in Hpa An and Hlaing Bwe townships in Kayin State by enhancing access to and utilisation of governmental and community based health services, improving access to water and sanitation through community led initiatives and promoting the protection of people's rights
Expected Project Results:
- Access to and utilisation of primary health care services has increased through strengthening and expansion of public health services and increased cooperation with and among trained community-based healthcare providers.
- Access to reproductive health services has increased through a network of trained community maternal and child healthcare workers, government health staff and local health structures.
- Access to safe water and appropriate sanitation facilities and hygiene awareness has improved in 50 villages.
- Awareness and protection of their rights as human beings and as citizens have increased significantly within the project communities, particularly among the most vulnerable groups.
II.Objective of the Evaluation
The objective of this end-of-project, field-based, external and independent final evaluation is to assess the outcome and impact of the above cited Europe Aid grant funded project, with feedback on the achievements to the donor, Malteser International and ADRA Myanmar regarding the intervention logic, objectives and expected results/indicators. In doing so, the evaluation should provide a clear picture on the following aspects:
- Detect the real changes in the field and analyses the changes attributable to the intervention
- Check and explain achieved results;
- Identify and judge unexpected results;
- Assess the sustainability of the intervention's benefits;
- Report and give accountability to the institutions that have allocated the resources;
- Transferability or adaptability of acquired experiences to other countries or sectors.
- Best practices and lessons learnt for future projects.
The evaluation is expected to enable an internal review of both implementing partners, Malteser International and ADRA Myanmar, through participatory methods and a reflection workshop.
III.Expected Outcome
The evaluation will focus on:
1. Relevance:
a) Is the project in line with local needs and priorities of the target population?
b) Has the project design and implementation been appropriate to meet the health and WASH needs of the target groups?
c) Did the project successfully reach out in addressing the needs of women and children and other vulnerable groups?
d) Was the project design coherent with policies of the Myanmar Government?
2. Effectiveness
a) Were the planned results achieved and did the outputs lead to the intended outcomes?
b) Were the activities undertaken effective in order to meet the health/WASH needs of the targeted population?
c) Which project activities have had the greatest positive effect and which had the least effect? Were there any activities that should have been included in the action but were not? And were there activities or indicators that were less relevant or not needed?
d) Were activities supporting the capacity of the Ministry of Health at state and township levels and communities leading to a strengthening of their services to the local populations?e) Asses appreciation of the program by the beneficiaries as well as their participation for various level in the project cycle
f) Did Malteser’s and ADRA’s intervention achieve a better quality of health care service?
g) How did women and children benefit from health services and hygiene promotion?
h) Were health education methods appropriate to achieve behaviour change?
i) Was the operation successful in reaching the most isolated hamlets within the target area?
j) Was the established monitoring system adequate to provide oversight and steer the implementation?
3. Efficiency
a) Were the project funds used in a cost efficient way?
b) How efficient inputs have been converted to outputs?
c) Were the resources allocated in the health sector justified and adequate to improve the health status of the local population, especially women and children?
d) Were the resources allocated in the WASH sector adequate to achieve an improvement in hygiene and sanitation of the local population?
4. Outcome/Impact
a) What is the cumulative effect of the operation in relation to the situation of the beneficiaries in general (including positive, negative, primary and secondary long-term effects produced by the action, directly or indirectly, intended or unintended impacts)?
b) What are the wider effects of the operation on individuals, different gender groups, communities and institutions?
c) What visible/evident impacts have emerged from the project implementation and its outcomes? Kindly specify according to health and WASH sectors and, if relevant, other areas.
d) Are beneficiaries satisfied with the assistance provided? What real difference does the project make to the beneficiaries?
e) Were cross-cutting issues (e.g. gender, social inclusion, disability, trust-building) between communities and public health care services adequately taken into consideration in the set-up and implementation of the project?
5. Sustainability
a) How did the project adapt to the changing context in Kayin/Myanmar? Were ADRA and Malteser sufficiently incorporating a longer-term view into this project?
b) Did the project develop an exit strategy and to what extend are the project’s achievements likely to last after the project’s end?
c) What efforts were made to ensure that the approach used in the project was participatory and creating ownership in the communities? Were these successful? How are these efforts monitored?
d) What will remain the unfulfilled health/WASH needs in the target area after the intervention and what further initiatives to ensure greatest possible sustainability would best address these needs?
6. Mutual reinforcement (coherence)
a) The extent to which activities undertaken allow the EU to achieve its development policy objectives without internal contradiction or without contradiction with other EU policies. Extent to which they complement partner country's policies and other donors' interventions.
b) Considering other related activities undertaken by Government or other donors, at the same level or at a higher level:
- likeliness that results and impacts will mutually reinforce one another
- likeliness that results and impacts will duplicate or conflict with one another
Connection to higher level policies
Extent to which the project/programme (its objectives, targeted beneficiaries, timing, etc .):
- is likely to contribute to / contradict other EU policies
is in line with evolving strategies of the EU and its partners **
EU value added**
The criterion is closely related to the principle of subsidiarity and relates to the fact that the action financed through the Commission has added benefits to what would have resulted from Member States' interventions only. For example:
1) The European Union has a particular capacity, for example experience in regional integration, above that of EU Member States;
2) The European Union has a particular mandate within the framework of the '3Cs' and can draw Member States to a greater joint effort; and
3) The European Union's cooperation is guided by a common political agenda embracing all EU Member States.
Extent to which the project/programme (its objectives, targeted beneficiaries, timing, results, etc .)
· is complementary and coordinated to the intervention of EU Member States in the region/country/area · is creating actual synergy (or duplication) with the intervention of EU Member States and involves concerted efforts by EU Member States and the EC to optimise synergies and avoid duplication.
8. Conclusions and Recommendations
- a) Main conclusions drawn from analysis with evidence provided during field visits and observations;
- b) A set of recommendations linked to the conclusions that would provide guidance to Malteser and ADRA for future interventions in Kayin.
IV.Evaluation Methodology
An independent international consultant familiar with EU evaluation stipulations and a national evaluation facilitator will be contracted. In addition, Malteser will provide one translator for the evaluation team. The selected consultant will receive all relevant documents from Malteser International for desk study 15 days prior to the start of the evaluation in the field. The evaluation team will conduct an approx. 10 day field assessment in Kayin Hpa An and Hlaing Bwe Townships which will include, visits to selected target communities, other I/NGOs operational in the area, discussions with both partners, Malteser and ADRA, as well as government stakeholders and key staff of the EU delegation.
The consultant is expected to interview and/or conduct focus group discussions with staffs, beneficiaries of the project activities, target groups, and other stakeholders such as:
- Malteser International and ADRA project staff working in Hpa An and Hlaing Bwe Township;
- Community Health Networks (CHNs): Mothers Support Groups (MSGs), Village Health Committees (VHCs), Water User Groups (WUGs). Health Volunteers (HVs): Auxiliary Mid Wifes (AMWs) and Community Health Workers (CHWs).
- Government health care providers (Township Medical Officer, Basic Health Staff);
- Local authorities (village administrators, village leaders, religious leaders).
- Key staff of the EU delegation in Yangon
- A joint review workshop with Malteser and ADRA field teams will be conducted in Hpa An
The consultant will provide Malteser International a list of the specific meetings he/she wishes to conduct and hamlets to be visited during the field assessment in Hpa An and Hlaing Bwe Townships to allow for the related administrative and logistics arrangements (e.g. accommodation, transport, travel authorisations) and schedule of appointments. A briefing meeting with representatives of Malteser International and ADRA is foreseen in Yangon in order to finalise the organisation of the evaluation. Likewise, a joint Debriefing session will be held in Yangon before departure.
V.Timeframe
The evaluation is expected to take place in mid-June 2016 once the endline survey data report is received with with the following preliminary timeframe:
Preparation and desk study, preparation of interview guidelines, team coordination: 3 days
(If applicable) International travel to and from Yangon: 2 days
Briefing with Malteser International and ADRA in Yangon and travel to Kayin: 1 day
Joint review workshop with Malteser and ADRA field teams in Hpa An (including preparation): 1 days
Field visit to Kayin, Hpa An and Hlaing Bwe Townships: 10 days
Debrief Kayin Team and incorporate feedback from field teams: 1 day
Return travel from Kayin and Debriefing in Yangon; Meeting with EU: 1 days
Reporting: 5 days
Total:Max. 24 days
VI.Reporting
Key deliverable will be: an inception report and a consolidated evaluation report that complies with the requirements of Europe Aid’s evaluation guidelines and quality of reporting.
The final report will follow the attached basic structure consisting of three parts:
A) Executive summary (that will be of standalone character, max. 5 pages)
B) Main Report focusing on the questions raised in this ToR under the headings (relevance, effectiveness, efficiency, impact and sustainability, conclusions and recommendations, max. 30 pages)
C) Annex of main supporting documents, agenda, maps, interview guidelines, list of interviewees by title except where by identity of the respondent is protected, etc.
As part of the process, the evaluation team will submit the following documents:
Before leaving the field: A Debriefing note (2-3 pages) which will contain a summary of main findings and recommendations to Malteser International and ADRA. The debriefing note/summary will be discussed during the debriefing sessions in Kayin Field Office and in Yangon Country Office respectively;
3 weeks after departure from the field: Draft report (electronic version, max. 30 pages, no annexes) in English, in user-friendly format (MS-Office) to Malteser International and ADRA in Yangon and Kayin Field Offices; Malteser International HQ
2 weeks after feedback from Malteser/ADRA:The Final report will be submitted in 4 printed copies of the final report in English to Malteser International Country Office and ADRA in Yangon, and 1 electronic copy to Malteser International’s Headquarters in Cologne/Germany, the Country Offices in Yangon, and Kayin Offices.
The report and all background documentation will remain the property of Malteser International and ADRA Myanmar, and will be promulgated as appropriate by the above partners.
VII.Required Expertise
Essential:
• At least 5 years’ experience in evaluating development and/or humanitarian projects, preferably in the field of health care as well as water/sanitation;
• Health qualification (e.g. Public Health, Health Management Systems) or equivalent, MSc preferred;
• Familiarity with Community Based Health Care and WASH programmes in under-developed and complex environments;
• Knowledge of health systems and experience of assessing health systems, among which is drug management;
• Knowledge and application of outcome harvesting techniques will be added advantage.
• Knowledge of European Commission rules and regulations or previous experience of working with the European Commission;
• Fluency in written English.
Desirable:
• Previous work experience in Kayin State or Myanmar;
• Understanding of Myanmar language;
• Valid Myanmar MoH visa / Business visa (facilitation of business visa possible).
How to apply:
Bids must include:
• 1-2 page outline covering a) overall framework and b) sampling/survey methodologies, comments and suggestions on this ToR;
• CV and evidence/references of past evaluations.
• Proposed Budget and work plan
Bids should be submitted electronically to Malteser International by the closing date of 5 June 2016. COB to:
Maren Paech, Country Group Manager Myanmar/Thailand at maren.paech@malteser-international.org
Serge Birtel, Partner & Programme Coordinator at serge.birtel@malteser-international.org
Oscar Garcia Linarte at oscar.linarte@malteser-international.org