Summary
Program/Project,
Project Number Matthew Response CBM Haiti
Project Number: 3562-MYP

Partner Organisations BSEIPH, DPOs FNGA , Tous Ensemble , BRENDA , Rehab Center of Port-a-Piment, CESI, RANIPH

Project start and end dates,
Phase of project Start date: October 2016

End date: 31st of September
Evaluation Purpose Assessment of the emergency response with regard to its efficiency, effectiveness, relevance and sustainability to generate learnings for future response and to promote accountability towards donors and towards beneficiaries by documenting outcomes
Evaluation Type
(e.g. mid term, end of phase) End of Emergency Response evaluation.
Commissioning organisation/contact person Emergency Response Unit CBM
Contact person: Alberto Tonon
Evaluation Team members (if known) Consultant
Primary Methodology Document review, key informant interviews and staff and beneficiary focus groups
Proposed Evaluation Start and End Dates 1st October (total duration 5 weeks)
Anticipated Evaluation Report Release Date 29th October
Recipient of Final Evaluation Report CBM International Office, CBM ERU, CBM partners CO and CBM Member Associations



Background of Project

On 4 Oct 2016 Haiti was hit by the most powerful Caribbean hurricane in nearly a decade. Hurricane Matthew, a Category Four storm, brought 230km/h (145mph) winds, heavy rain and dangerous storm surges. The hurricane immediately led to a number of deaths in Haiti and the Dominican Republic. Southern Haiti has been most affected, and is effectively been cut off after the bridge linking it to the capital, Port-au-Prince, collapsed.
Hundreds deaths have been reported as well as hundreds of injuries. The Haitian government has officially requested Humanitarian Assistance as more than 2 million people had been affected and were in needs of support.
CBM’s Response
The emergency response strategy had a total duration of 10 months.

The response, has be directly implemented by Emergency Response Unit of CBM in close cooperation with the CBM Country office and the office of the State Secretary for Inclusion of Disability (BSEIPH) as well as other local partners that CBM supported during the response.

The areas of the intervention were the regions of Sud and Grand’Anse, and the project aim to reach the most vulnerable people in the rural area as well as in the urban areas of those areas.
The 4 main objectives of the response were:
• Needs and protection of communities in Jeremie and Les Cayes, and especially persons with disabilities and other marginalized/at risk groups are identified
• Community-based protection and psychosocial support is available to affected communities, including households with person with disabilities and other marginalized/ at risks groups, in Les Cayes and Jeremie
• Persons with disabilities and their families as well as specialized services recover from the disaster and rebuild their life/ services.
• CBM Country office in Haiti, Regional Office and ERU are able to support and follow-up emergency response and to further develop next phase proposals
The expected beneficiaries foreseen were:
- 3’500 persons with disabilities and other at risk groups individual are identified and at least 30% successfully referred to access relief services.
- At least 250 people attend psychosocial support groups
- 100 persons with complex mental health issues are followed by the psychologist
- 235 people receive training/ technical support
- About 20 people received PFA training
- At least 2000 people received inclusion factsheets
- At least 10 organisations received technical inclusion support from CBM staff
- About 20’000 people benefit from the projects indirectly in Les Cayes and Jeremie (average households members = 5)
- At least 400 at risk households including some with persons with disabilities, older people received livestock and farming/garden kits
- 2 school/rehabilitation centres are equipped benefiting at least 100 persons

Evaluation Objective

The emergency response strategy was developed by CBM ERU with the collaboration of the CBM Country office in Port au Prince and the office of the State Secretary for Inclusion of Disability (BSEIPH), the evaluation will cover the 10 months of the CBM response in the two most affected areas by the hurricane Matthew, the South and the Grand’Anse with all its aspects including processes and procedures for defining and implementing the emergency response.
The purpose of the evaluation is to provide evidence about the results of the emergency response that can be used to inform management decisions, drawing on the learnings, and enable both field and ERU staff to undertake corrective actions for future response.

The objective of the evaluation is to provide an assessment of the progress of the components as compared to the objectives, incl. the current situation in medical/rehabilitation and psycho social support and disability inclusion. The evaluation is to cover the relevance and appropriateness of the action, the effectiveness and efficiency, the connectedness and coherence with other actions, the coverage of the target region and population and its likely impact on the target population.
The results are envisaged to support (a) learning by active participation of the ERU team and CBM CO Haiti, which will be the most immediate user of the evaluation and its recommendations; and (b) to promote accountability through documentation of achievements and outcomes of the response.


Evaluation Questions

The evaluation will focus in large parts on the effectiveness and efficiency of the current coordination and management systems. The evaluation will identify the results of the response, namely the extent to which it achieved or did not achieve its objectives and meet the needs of all segments of the affected population, and will identify factors contributing to these results at every level (implementation, planning, assessment, judgment of context, cross‐cutting issues). As such, the evaluation will focus on the following specific aspects:

• the effectiveness, efficiency, timeliness and relevance of the assessment, planning and management processes and systems put into place;
• the effectiveness, efficiency, timeliness and relevance of the initial response, including identification of critical gaps and bottlenecks;
• the adequacy of coordination and connectedness at large, not only in the response itself but also on issues such as, cross-cutting themes, and so on; and
• the evolution of critical features in the context, as well as the evolution of needs and the way they are assessed.

The key initial question for the evaluation would be: How adequate was the response as a whole, and what results as well as positive and negative outcomes for the affected population did it produce?

Relevance, appropriateness and quality of design:

This measures the extent to which the project is suited to the priorities and policies of the target group, the partner organisation, to local and national development priorities, to CBM policies, including changes over time.

Overarching question:

Did the response cover the needs? Have appropriate common standards been developed within the coordination systems and to what degree have these been met?

• Have relevant, inclusive and appropriate strategic and response plans been developed in a timely way and based on analysis of the common needs assessment? Do these plans respond to critical gaps identified, avoiding duplication and addressing unmet needs of most affected population? Did these plans identify risk and address them?
• To what extent has the response design ensured that persons with disabilities are active participants in the response as well as recipients/beneficiaries?
• To what extend the response reached out affected population in an inclusive manner?
• How were the recipients of the services informed about complaint/ feedback mechanisms? Were these mechanisms used?
• How timely and successful is the emergency response in delivering against stated objectives/indicators?
• What critical factors (e.g., security events, infrastructure, procedures, access, enabling environment, etc.) help explain why the response was or was not delivered in an adequate and timely manner?
• What is the response’s level of commitment and compliance to standards (such as SPHERE, HAP 2007 Standard in Humanitarian Accountability and Quality Management, Code of Conduct for the International Red Cross and Red Crescent Movement and NGOs in Disaster Relief, guidance on civil-military relations and protected humanitarian space, etc.)?
• Have appropriate common inclusive standards been developed within the coordination systems and to what degree have these been met?
• Has CBM engaged with the established UN Cluster coordination mechanisms to ensure relevance, coherence, adherence to standards and promote disability inclusion

Effectiveness:

This measures the extent to which the project actually achieved what it set out to do.

Overarching question:

• In how far have the results as set out by the project partners been delivered? Quantitative and qualitative measures of anticipated outputs and outcomes.
• Assess the quality and quantity of immediate medical and psychological support against the objectives and targets set at the beginning.
• How was inclusiveness achieved in the various components ( psychologic supportadvocacy, capacity building, education, livelihood and rehabilitation)
• What results did the advocacy work with mainstream organisations achieve?
• How effective was the networking with different providers and their service delivery as a result of this networking. How did the referral system function and with what positive or negative results?
• Assess the effectiveness of the technical guidance to promote age and disability inclusion, resources and tools towards this end.

Efficiency or cost-effectiveness (of planning and of implementation):

This measures the outputs in relation to the inputs (costs, both financial, staffing, time) and whether funds are used in the least costly way in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted.

• How effective has the overall inter-agency planning and management process been?
• To what extent have these been based on an inclusive and coordinated needs assessment and analysis that reflects the views of various stakeholders, including government, civil society organizations and various segments of the affected population (including socially excluded groups and groups and individuals vulnerable to human rights violations due to discrimination and stigma)? How quickly and adequately have these appeals been responded to?
• How adequate is the continuity of funding and staffing within CBM?
• How adequately are recovery considerations incorporated into assessments, planning and provision of relief interventions?
• How adequately has the political dimension of Haiti’s context been considered in assessments, planning and provision of relief and transition to recovery efforts?
• How sufficient have funding flows been, both in quantity and timeliness, so as to allow CBM and its partners to respond effectively to both humanitarian and time-critical early recovery needs?
• To what extent have local (in-country or regional) resources – including material as well as human – been used, or at least considered, when designing/implementing activities?

Sustainability:

The likely ability of an intervention to continue to deliver benefits for
an extended period of time after completion. Projects need to be environmentally as well as financially and socially sustainable.

• To what extent are the basic tenets of disaster risk reduction (DRR) being incorporated into planning and efforts in order to reduce further vulnerability?
• In how far have local/national individual/organizations (especially persons with disabilities and DPOs) been supported to be actively involved in response, and had their capacity built, where necessary
• To what extent have stakeholders orientated/trained on disability inclusion have incorporated this in their response plans and guidelines?
• To what extent is there evidence that the benefits to community life of disability inclusion (and/or universal design) will feature long-term in communities reached in CBM/partner response
o Connectedness refers to the need to ensure that activities of a short-term emergency nature are carried out in a context that takes longer-term and interconnected problems into account. Connectedness means also that Interventions should be linking relief, rehabilitation and development.

Overarching questions:
• Has CBM engaged with the established UN Cluster coordination mechanisms to ensure relevance, coherence, adherence to standards and promote disability inclusion? And if so, we have added this in relevance and quality of design?
• Were response activities planned in support to pre-existing response plans, structures and capacities in Haiti ?
• Was CBM’s response part of the UN cluster coordination mechanism? Did CBM and its partner contribute to addressing unmet needs, avoiding duplication and adhering to humanitarian standards? What systems have been put into place to monitor, report and evaluate the efficiency and effectiveness of the overall response? How adequate are these for measuring progress against objectives?
• Was an inclusive common strategy for security and access developed?
Specific questions:
• In what ways, if any, has the program approach led to a more strategic response in terms of leadership, partnership, cohesiveness and accountability?
• How effective has partner and inter-agency coordination been?
• In what ways, if any, has the government’s institution leadership (BSEIPH) capacity been strengthen as it has the primary responsibility to respond to its people’s needs?
These might include the following principles, among others: (1) ensuring that all projects are designed with a multi-hazard approach and do not pose new threats or increase the existing ones; (2) ensuring a ‘build back safer’ approach addressing underlying causes of risk and is informed by national and local assessments of risk; (3) promoting community participation (especially women and children) in all stages (from assessment, implementation to evaluation); (4) building on existing capacity of government, civil society and people; and (5) embracing partnership to ensure the effective use of resources at national and sub‐national level.
• In what ways, if any, have national and local capacities been capitalized on and strengthened (e.g., in needs assessments?)
• How effectively have partnerships with civil society organizations and the affected communities themselves been built up in order to maximize local ownership, and thereby enhance effectiveness, accountability and sustainability?
o Coherence: The need to assess security, developmental, trade and military policies as well as humanitarian policies, to ensure that there is consistency and, in particular, that all policies take into account humanitarian and human-rights considerations.
o Coverage and inclusion: The need to reach major population groups facing life-threatening suffering wherever they are. The evaluation should assess the inclusion of various populations/groups, and whether there has been an exclusion bias and whether an equitable approach was taken.
• What segments of the affected population could and could not be assisted, and why?
Specific questions:
• Has a common needs assessment and analysis been carried out?
• What proportions of the affected population could be assisted? Who was excluded, and what were the key barriers to full access?
• How adequate and timely were situation timelines and statistical evidence on contextual factors (such as situation of the population (casualties, wounded, sick, degree of infrastructure destruction) to the needs of operational decision making?
• Has analysis of different needs, vulnerabilities and capacities and response design included a vision of the variant effects on men and women, girls and boys, and persons with disabilities? Has the disaggregated data (by age, disability and gender) been available to inform the analysis?
• To what extent have the needs of all segments of the population, men and boys, women and girls and disproportionately affected groups/PwDs been assessed and the differential needs of specific subpopulations been addressed?
• How far has the humanitarian response been tailored to meet national and local needs and ensure ownership at these levels by, and accountability to, affected populations? How far has the response been tailored to the divergent needs, vulnerabilities and capacities of girls and boys, women and men, young and old, and socially excluded groups?
• Has information about the humanitarian response been communicated in a manner that is widely accessible to the people in the areas of implementation of the project?

• Gender

The evaluation must consider gender aspects of the project and report on this. Data collected must be disaggregated according to gender.
Some gender related questions are already included above.

Methodology
The evaluation of the immediate ER program will be carried out through analyses of various sources of information, including desk reviews CBM ERU will provide the evaluation team with data gathered during its monitoring preceding the field visits interviews with key stakeholders (such as I/NGOs, donors, affected communities, government and others will be held and triangulated with monitoring data.
Specific evaluation methods that might be considered to be applied are: key stakeholder analysis, standardized questionnaires, individual face-to-face meetings, small group semi structured interviews, participant observation, and open-ended questioning.
Evaluation Team and Management Responsibilities

Commissioning responsibility
The evaluation will be managed by the ERU team which will oversee the conduct and quality of the evaluation. The external consultant team (team leader) will report to the ERU. The ERU will have the following responsibilities:
• Manage the entire evaluation (including financial resource mobilization, recruitment of the consultant , reviewing the inception report, reviewing the report and the management response plan);
• Offering in-country support to the evaluation through its staff there;
• Monitor and assess the quality of the evaluation and its process;
• Provide guidance and institutional support to the external consultant;
• Provide and/or coordinate logistical support to the evaluation team;
• Facilitate the consultants access to key stakeholders and specific information or expertise needed to perform the assessment;
• Ensure that all stakeholders are kept informed;
• Approve the final report; All evaluation products will be submitted to the ERU at the end of the assignment and upon approval of the final report.

Stakeholder Involvement:
The evaluation team will engage with staff from CBM ERT (Emergency Response Team), CBM CO (Country Office) and CBM ERU (Emergency Response Unit) and its partners on the ground, with international NGOs, national NGOs, national stakeholders, government (BSEIPH) and donor organizations.

Management of the evaluation and logistics

Emergency Response Team in Haiti has responsibility for: logistics in country (accommodation, transport, providing communication means to evaluation team, organizing meetings etc.)
Project Partner has responsibility for: being available to be interviewed by the consultants, facilitating access to data and project team, facilitating meeting beneficiaries if requested by the consultants.
Emergency Response Unit has responsibility for: Overall management of the evaluation, providing all information related to emergency response projects in Haiti and being available for interviews if requested by the consultants.

Expected Results

1) An inception report including the detailed plan for the conduct of the evaluation to be delivered after 3 days. It will include the mission schedule, a list of persons and organizations to be interviewed and the methodology of the evaluation and a draft of all the Data Collection Tools (FGD guides, KII guides, workshop agenda…)
2) A final report containing analytical elements related to the issues specified in this ToR. The report shall contain a short executive summary of no more than 2 pages and a main text of no more than 60 pages, both inclusive of clear and concise recommendations. Annexes should include a list of all individuals interviewed, a bibliography, a description of method(s) employed, a summary of survey results (if applicable), and any other relevant materials.
The report will be submitted two weeks after the completion of the mission.
3) Draft reports will be submitted within two weeks of the consultants’ return from the field mission, upon which the ERU will have 7 days to comment.
The evaluation team is solely responsible for the final products. While maintaining independence, the team will adhere to professional standards and language, particularly that which may relate to the protection of staff and operations. Direct consultations with affected populations will be a formal requirement of the evaluation unless security conditions are overriding. Additionally, agencies at the country level and the ERU will be consulted prior to the dissemination of any products emanating from the evaluation.
All analytical results and products arising from this evaluation will be owned by CBM. The team leader and/or members will not be allowed without prior authorization in writing to present any of the analytical results as his or her own work or to make use of the evaluation results for private publication purposes.
Compliance with CBM evaluation principles and ALNAP standards is expected and the evaluation report will be judged in this regard. All external evaluation reports will also be submitted to CBM International Office for inclusion in the regular meta evaluation process that rates the quality of evaluation reports.

Duration and Phasing

Task Location Number of Days Expected Dates
Desk Review of Project Documents
Home based 3 9th to 11th of October
Travels Travel to Les Cayes and Jeremie 4 12nd of October
Field mission, including inception workshop with CBM partners, briefing meetings and interviews

Les Cayes and Jeremie 16 13th of October
De-briefing workshop
1 29th of October
Report writing Home based 6 30th of October
Finalisation of report Home based 3 15th of November

Attachements
The Country Emergency Program strategy paper and all project proposals including detailed work plans, budget and all monitoring reports will be made available to the selected consultants.

The evaluation will employ the services of a consultant.
The consultant shall share between them the following experience:
• Proven senior-level evaluation experience (at least 8 years) and ability to provide strategic recommendations to key stakeholders;
• Demonstrated experience in conducting evaluations of humanitarian program and the capacity to work collaboratively with multiple stakeholders and on a team;
• Good knowledge of strategic and operational management of humanitarian operations in Haiti context ;
• Good knowledge of humanitarian system and local government disaster response structures and systems in Haiti
• Strong experience in disability and in disability inclusive humanitarian actions;
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions and to prepare well‐written reports in a timely manner;
• Strong workshop facilitation skills;
• Excellent writing and presentation skills in English and French;
• Availability for the period indicated.

Proposals, including quotations and timeframe, should provide complete details on the following:
1. Personal profile and proven experience of the lead evaluator (CV)
2. Cover letter
3. Previous experiences in disability and humanitarian sector
4. Proposed methodology/approach
5. Cost break-down

The future job holder adheres to CBM’s values and commits to CBM’s Child Safeguarding Policy.


CBM encourages persons with disabilities to apply for this position.

Interested applicants should submit a proposal structure by 02/10/2017 to 13/10/17

Interested Firms/Individuals should send the requested documents at cbmhaiti.hr@gmail.com .
The client is happy to conduct telephone interviews for overseas candidates.

The future job holder adheres to CBM’s values and commits to CBM’s Child Safeguarding Policy.


CBM encourages persons with disabilities to apply for this position.