Myanmar: Mid-Term Review (MTR) of BRACED Alliance Myanmar Project (2015-17)

Mid-Term Review (MTR) of BRACED Alliance Myanmar Project (2015-17)**

1.Background of the Project

As a part of its global BRACED Programme, DFID has awarded a three year (2015-17) grant to implement the BRACED programme in Myanmar to an alliance led by the Plan International. The alliance member organizations include ActionAid, BBC Media Action, the Myanmar Environmental Institute, World Vision, UNHabitat and Plan. Further background information on BRACED is available from DFID at: http://www.braced.org/

Myanmar, the second most vulnerable country to climate extremes globally, is at a moment of unprecedented institutional, economic and social change. The BRACED Alliance Myanmar Project contributes to this transition, from household to national policy level, building the resilience of over 350,000 individuals (directly working at 155 villages in 8 townships) across three at-risk climatic zones (dry, hilly and coastal). Through a unique model of policy, action and media outreach, a diverse collaboration amongst local and international partners will combine DRR, CCA, environmental, community-development, policy, gender, and livelihoods expertise, to improve community and institutional understanding of climate risk, drive increased responsiveness, and promote scale-up and replication of resilient practices in the face of climate extremes.

The BRACED Alliance Myanmar Project's log-frame is based on the following impact, outcome and outputs aims:

· Impact: Improved well-being and reduced losses and damages of vulnerable populations despite climate shocks and stresses

· Outcome: Vulnerable communities, driven by women and children and supported by effective institutions, are more resilient to climate extremes and disasters by 2017

· Output 1: Communities, especially women and children, are equipped with the knowledge and skills to mitigate the risks of and recover from climate shocks and stresses

· Output 2: Institutions are coordinated, responsive, accountable and inclusive in their management of climate risks

· Output 3: The Evidence Base is strengthened and learning on managing climate extremes is disseminated to inform and influence the resilience related policy strategies and agenda at sub national, national and global levels.

The project has developed and rolled out a comprehensive evaluation strategy to evaluate its final impacts using quasi-experimental evaluation approach surveying representative sampled villages from both project target and non-target villages establishing best possible matches. A project baseline survey was carried out in Q4 2015 surveying 2,400 households (50% from target villages and 50% non-target villages.

2.Scope of MTR

The proposed MTR will cover the time span from 1 January 2015 (starting of the project) to June 2016.

Utilization: The primary users of the MTR findings and recommendations will be BRACED Myanmar Project and its partners, but they will equally be useful to other relevant government ministries/departments, national/international agencies working on resilience in Myanmar, donors, and BRACED Knowledge Manager and Fund Manager.

The MTR findings will serve to validate the project’s ToC and corresponding indicator framework presented, and inform adjustments to programming for the remainder of the project as well as contribute to the BRACED evidence base on effective ways of increasing people’s resilience to climate extremes and disasters. This learning will serve to influence wider policy decision-making processes and strategies in country and at regional and global levels. The MTR findings will also inform both project as well as the BRACED evidence base at programme level.

In addition, the review aims at critically assessing and identifying what has worked well in the project and/or what is working well, what challenges have been faced, what lessons can be learned to improve future programming. The review will also serve as a quality assurance tool for both upward and downward accountabilities. Following the MTR recommendations, BRACED Myanmar ACU will prepare a management response in collaboration with implementing partners, Fund Manager and Knowledge Manager, and implement follow-up actions as per Plan International’s Evaluation guidance/policies and international standards set for evaluations.

Geographic coverage: The BRACED Alliance Myanmar Project targets 155 communities in 8 Townships in 3 diverse climatic zones - the Central Dry Zone, Coastal Zone and Hilly Areas – where vulnerable groups are increasingly exposed to extreme climate events including: i) cyclones/strong winds; ii) floods/storm surges; iii) intense rains; iv) extreme temperatures and v) drought[1]. The impacts of these climate extremes are further exacerbated locally by processes attributed to climate change and variability. These include the shortening and intensification of monsoons; an increase in sea surface temperature and an overall increase in heat and drought indices[2]. Environmental degradation and deforestation in particular, combined with diminishing water resources compound the overall vulnerability of Myanmar communities[3].

Target groups and primary stakeholders: Total BRACED Myanmar Alliance Project aims to provide supports to over 350,000 beneficiaries focusing women and children from targeted townships and communities in Myanmar to improve their resilience to climate extremes. Following the global BRACED beneficiary definitions (DFID and the International Climate Fund’s Key Performance Indicator 1 – ICF KPI 1 guidance[4]); the project beneficiaries are categorized in to two types: 1) targeted and high intensity and 2) targeted and medium intensity.

Beneficiaries at targeted communities[5]

Targeted & high intensity*

• Around 17,000 vulnerable community members with at risk livelihoods from 153 villages (eg: recipients of micro finance, climate resilient agriculture or resilience building interventions, etc.)

Targeted & medium intensity*

• Around 160,000 community members from 153 villages will benefited from resilience model

• Around 172,000 community members in 8 targeted township will be exposed to BBCMA media output

In addition, around 2,000 national and sub-national officials/journalists will receive targeted high intensity supports including trainings, capacity building.

3.MTR Overall Objectives:

The overall objectives of this mid-term review are to:

a) To inform project management and its partners at mid-point of BRACED Myanmar Project about necessary adjustments to the project theory of change and implementation strategy (what worked and what did not work; and what is working and what needs adjustment for improvement).

b) To assess progress and to contribute to evidence and learning on if and how packages of interventions deliver resilience in different climatic contexts (coastal, hilly and dry zones) and influence factors.

4.MTR specific objectives:

I. Review relevance of the project theory of change and implementation strategy

II. Assess how the project inputs and activities are producing the outputs contributing to expected outcomes and impact

III. Assess how the packages of resilience services provided by the project under 5 activities themes[6] included in the project design led or likely to lead in building resilience to people, household and community

IV. Assess value for money in the project

V. Assess both negative and positive factors that have effected in achieving or non-achieving the project outputs and outcomes

VI. Provide forward looking programmatic recommendations for remaining project period and beyond the project period.

VII. Encourage and celebrate the achievements of partners

VIII. Identify and document substantive lessons learned, good practices and opportunities for potential scaling up of BRACED Myanmar’s interventions

IX. Promote accountability and learning

5.Mid-term review Criteria and Questions:

The MTR will use four key evaluation criteria: relevance, efficiency, effectiveness and potential sustainability addressing suggested below questions.

Relevance:

· To what extent the project theory of change, results chain and implementation strategy are still consistent with the needs and interest of the target beneficiaries and needs of national/subnational policy landscapes?

· Are the activities and outputs of the project consistent with the overall goal and the attainment of its objectives?

· Are the activities and outputs of the project consistent with the intended impacts and effects?

· To what extent packages of resilience assistance offered by BRACED Myanmar are innovative to improve people’s resilience compared to other existing or usual DRR works?

Effectiveness:

· To what extent packages of resilience assistance provided by the implementing partners to target beneficiaries led and/or likely to lead to strengthening resilience to disasters and climate extremes? Which packages of resilience assistance under 5 activity themes[7] provided are found more effective than other? Which direct field implementing partners are delivering packages of resilience assistance more effective than other?

· Specifically focusing on understanding ‘mechanisms’, how and why have particular intervention packages led and/or likely to lead to observed results and changes?

· Has the project changed or realigned its intervention packages design and why? To what extent is beneficiary feedback is influencing the design and delivery of BRACED?

· To what extent have the planned and implemented activities and outputs likely to led to the achievement of the outcomes?

· To what extent and in what ways has ownership - or the lack of it - by the implementing partner impacted on the efficiency of the project?

· What unanticipated, positive or negative, enablers or constraints had the project encountered? How they contributed to the achievement or non-achievement of the project outcomes and outputs?

· How effective was the risk management strategy in the project?

Efficiency:

· Are the project outputs being delivered at reasonable cost and within expected time frame?

· To what extent has the project ensured value for money?

· To what extent the project activities were implemented in the most efficient way compared to alternatives?

· To what extent did project monitoring systems provide management with a stream of data that allowed it to learn and adjust implementation?

· To what extent the project has appropriately resourced and mobilized the right people at the right place across field locations, country offices and headquarters? Is the project missing certain expertise or staff?

Sustainability:

· To what extent the project outputs and outcomes likely to results long term sustainable impacts?

· To what extent has the project contributed to improving an enabling environment for resilience-building? Has it reached beyond the initial boundaries of the project?

· What has the project learned about delivering established packages of interventions through the project?

· What evidence is there that the interventions and the mechanisms that support them have the potential to deliver ‘amplified results’ and/or ‘transformational impact’?

· To what extent the project’s approaches or packages of interventions provide basis for potential scaling up?

In addition to these questions and OECD-DAC evaluation criteria[8], the following are cross-cutting questions for the project:

· Gender – to what extent did the project contribute to increased equality between women and men, boys and girls? To what extent the project is able/likely to address gender transformation?

· Child-centeredness – to what extent were children involved in the project, what was the impact on boys and girls of their participation in the project and how did the project affect girls and boys, directly or indirectly, positively or negatively?

· Non-discrimination and inclusion – who benefited from the project and who was excluded, and why? How were marginalised/ vulnerable groups included? How were people with disabilities engaged in BRACED and benefit from it?

· Conflict sensitivity – what effects did the project have on conflict dynamics and how was conflict sensitivity addressed through project implementation? Was the do no harm approach followed so far in the project?

6.Methodology

The MTR team is expected to propose and determine a sound review design and methodology (including detailed data collection and analysis methods to answer each evaluation question) and submit it to BRACED Alliance Myanmar Alliance Coordination Unit (ACU) in the inception report following a review of all key relevant documents. The MTR team will review the suggested evaluation questions and the existing draft evaluation matrix, and fine tune them to meet evaluation objectives appropriately. It is suggested that the evaluation should use a mixed-method approach – collecting and analyzing both qualitative and quantitative data using multiple sources in order to draw valid and evidence-based findings and conclusions and practical recommendations. The evaluation team is not expected to conduct specific surveys to collect quantitative data, but the team will review all available relevant reports produced by implementing partners and external agencies that provide quantitative data.

Methods to be used by the evaluation team to collect and analyze the required data shall include but not be limited to:

• Desk review of relevant BRACED Alliance Myanmar documents, including project document, project progress reports, database, studies relating to the country context and situation, evaluation reports, etc.
• Focus group discussions and Key informant interviews with relevant stakeholders/rights holders/duty bearers from government agencies, BRACED partners (CBOs, national NGOs) donors, UN Agencies, civil society organizations, both at national and local levels. It is important that participation of different gender and social groups are covered including building data comparisons between the different male and female only focus group discussions. Other key stakeholders are Plan UK and FM and KM.
• Field visits/observation to selected project sites.
• Analysis of project budgets, forecasts and expenditure. However, please note an independent audit will take place in parallel to this MTR.
• Analysis and interpretation of qualitative and quantitative data available from credible sources both government and non-government agencies.

All data collection approaches will be in line with DFID’s Ethics Principles for Research and Evaluation[9]. Furthermore, all underlying dataset and/or transcripts must be available upon request for information sharing with Plan and the KM Evaluation Synthesis and Support Team.

7.Deliverables

The review team is responsible for submitting the following deliverables to BRACED Alliance Myanmar ACU as per the agreed work plan:

· Inception Report: This report will be around 10 pages in length and will elaborate how each evaluation question will be answered along with proposed methods along with reason for choosing the methods, proposed sources of data, and data collection and analysis procedures. The inception report will include the complete evaluation matrix elaborating the existing version tightly defining activity packages and their intended outcomes, and will also include a proposed timeline of activities and submission of deliverables. BRACED Alliance Myanmar ACU and its IPs will review the inception report and provide useful comments for improvement. This report will serve as an initial point of agreement and understanding between the review team and BRACED Alliance Myanmar ACU.

· Draft MTR Report: the MTR report will be around 30 pages without annexes, contain the same sections as the final report and shall follow the standard structure including executive summary, introduction, scope and objectives, description of the intervention, methodology, data analysis, findings,, recommendations (inclusive of proposed “course corrections” – adjustments to the project theory of change, assumptions and work plan), key lesson learned, conclusion and annexes. The draft report will be reviewed by BRACED Alliance Myanmar ACU, MTR reference group, IPs, Plan UK and KM. The draft report will ensure that each agreed evaluation question in inception report is answered with in-depth analysis of information and back up the arguments with credible quantitative and/or qualitative evidences.

• Presentation/Debriefing: a meeting will be organized with key stakeholders including BRACED Alliance Myanmar ACU, MTR reference group and IPs to present findings, conclusions, recommendations and lessons, and receive inputs.

· Final Review Report: the final report will incorporate comments and feedbacks from the stakeholders including the feedback provided during the Presentation/Debriefing meeting.

· Review Brief: a concise summary of the review report will include methodology, findings, conclusions and specific recommendations using plain language targeting wider audience. This concise summary will be not more than 4 pages.

8.Review Team Composition and Required Competences

The evaluation team will be comprised of one team leader (an international consultant) and a team member (national consultant). The presence of an international consultant is deemed desirable given the complexity and sensitivity of some of the issues concerned, and therefore to safeguard the independence and impartiality of the review.

International Consultant/Team Leader

· A masters’ degree or equivalent in economics, environmental science, geography, international relations, social sciences or other relevant fields.

· At least 10 years of working experience in development evaluations/reviews along with demonstrated experience in conducting evaluation or review of large scale policies and programs in resilience/climate change/disaster/adaption funded by bilateral donors.

· Past experience as a team leader for similar complex project evaluations/reviews.

· Possess strong analytical and writing skills, with the ability to conceptualize, articulate, write and debate about resilience issues.

· Experience in implementing a range of qualitative and quantitative data collection tools and methods in project evaluation.

· Knowledge of current issues and innovation in results-oriented monitoring, including trends, principles and methodology.

· Excellent research and evaluation report writing skills.

· Good knowledge and experience of working in Myanmar or South East Asia –desired.

· Advanced level of proficiency in both written and spoken English.

National Consultant

· A masters’ degree or equivalent in economics, environmental science, geography, international relations, social sciences or other relevant fields.

· At least 7 years of working experience in development evaluations/reviews along with demonstrated experience in conducting evaluation or review of large scale policies and programs in resilience/climate change/disaster/adaption funded by bilateral donors.

· Past experience as a team member for similar complex project evaluations/reviews.

· Possess strong analytical and writing skills, with the ability to conceptualize, articulate, write and debate about resilience issues.

· Experience in implementing a range of qualitative and quantitative data collection tools and methods in project evaluation.

· Knowledge of current issues and innovation in results-oriented monitoring, including trends, principles and methodology.

· Strong knowledge and understanding on national and local development context particularly related to disaster and climate change programming in Myanmar

· Very good research and evaluation report writing skills.

· Advanced level of proficiency in both written and spoken in English and Burmese.

Key competencies for the Team Leader and Team Member

· Excellent communication, reporting and writing skills

· Plans, prioritizes, and delivers tasks on time and produces quality results to meet established goals

· Creative and innovative in producing the desired outputs

· Ability to work with a multicultural environment and a multidisciplinary team

· Willingness to travel to project sites

9.Quality Standards and Ethics

This review will be conducted in accordance with the principles outlined in the ‘Quality Standards for Development Evaluation - OECD’[10].

The consultant will abide by Plan International Child Protection Policy.

10.Implementation Arrangement

MTR Management:

BRACED ACU Myanmar will be responsible for initiating, coordinating and managing the MTR throughout the entire process, and will provide required necessary supports including technical, logistic and administrative issues.

BRACED ACU Myanmar – Monitoring and Evaluation Manager will serve as the MTR Manager.

Reference Group for MTR

BRACED Alliance Myanmar Project has been operationalizing a vibrant M&E Working Group - engaging M&E Specialist/representatives from all six implementing partners - chaired by BRACED ACU M&E Manager. The M&E Working Group will serve as a Reference Group for the MTR and provide technical guidance and quality assurance.

11.Time Frame and MTR Process

The review is expected to start on 15 June 2016 and is foreseen to be completed no later than 30 July 2016. The following activities are expected to be commenced as per suggested timeline

Sn

Key activities

Duty station

Working days

Indicative Timeline

Team Leader

National Consultant

1.

Comprehensive review of relevant documents including project proposals, progress reports, project data, and other relevant reports.

Home Based

2

2

3^rd week of June 2016

2.

Travel for international consultant (if living outside of Myanmar)

(not counted as working days)

3.

Kick-off meeting with BRACED Alliance Myanmar ACU and IPs

Yangon

0.5

0.5

3^rd week of June 2016

4.

Preparation and submission of inception report including adjustment of work plan

Yangon

2

2

5.

Data collection including consultations, focus group discussions, key informant interviews, on site visits, etc.

Yangon and field visits

14

14

4^th week of June to 2^nd week of July 2016

6.

Preparation and submission of draft report

Yangon

4

4

2^nd week of July 2016

7.

Debriefing meeting with key stakeholders including BRACED ACU and IPs to present findings, conclusions and recommendations, and receive feedbacks

Yangon

0.5

0.5

2^nd week of July 2016

8.

Incorporate comments from stakeholders and feedback at debriefing meeting

Finalize and submit the final evaluation report and evaluation brief

Home based

2

2

4^th week of July 2016

Total daysconsultantwise

25

25

Note: The evaluation team must produce fully referenced MTR Draft Report and share with BRACED Alliance Myanmar ACU before leaving Yangon.

12.Application Procedure

Qualified candidates who meet requirements are invited to apply.

Qualified candidates can apply to the assignment either in a team (International Team leader + National Team Member) OR individually to the International Team Leader Position or National Team Member Position separately.

If an individual specialist is applying for either International Team Leader Position or National Team Member Position, she/he must clearly mention on his/her application about the position s/he is interested.

The proposal should consist:

· Brief outline MTR methodology,

· CV of the candidates

· Financial proposal having costs breakdowns for travel, renaturation, and associated costs.

· An example of previous similar work.

[1] Myanmar’s National Adaptation Programme of Action (NAPA) to Climate Change (Government of Myanmar: 2012)

[2] Climate Change Adaptation in Myanmar (Department of Hydrology and Meteorology: 2012)

[3] Myanmar’s National Adaptation Programme of Action (NAPA) to Climate Change (Government of Myanmar: 2012)

[4] https://assets.digital.cabinet-office.gov.uk/media/53da0af340f0b60b9f000030/BRACED-KPI-1-guidance.pdf

[5] https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/328261/BRACED-KPI-1-guidance.pdf

[6] Activity theme 1: Resilience planning cycle provides robust and inclusive planning support and integration of resilience into community and local institutional planning;

Activity theme 2: access to and use of meaningful weather and climate information to support the design of credible community and institutional action;

Activity theme 3: strategies to empower most marginalized groups, especially women and children, such as participatory institutional decision-making processes;

Activity theme 4: improved access to required assets and core systems to address key basic needs that strengthen livelihood resilience, such as access to relevant ecosystem services and greater access to financial services (VSLA/MF)

Activity theme 5: utilization of spaces for knowledge sharing and transfer going beyond a short-term project-cycle planning focus towards a longer term outlook that considers scaling up from the outset through integrated approaches that have engaged with relevant institutions from local to national scale; including the development and dissemination of evidence of effectiveness based on the above activity streams to inform policy and replication.

[7] Activity themes as outlined in footnotes 6 (further elaborated in evaluation matrix)

[8] See: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm

[9]https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67483/dfid-ethics-prcpls-rsrch-eval.pdf

[10] http://www.oecd.org/development/evaluation/qualitystandards.pdf

The proposalsto be submittedelectronically to Human Resource ofthe Plan International Myanmarat:Myanmar.CO@plan-international.orgby 22^nd April 2016, 5:30 PM Yangon Time**.**

For technical queries and clarification related to this assignment, please write to: bhushan.shrestha@plan-international.org