Summative Evaluation of Canada's Afghanistan Development Program - Appendix 1 Terms of Reference

Table of Contents

Figures and Charts

Tables and Boxes

Back to Report

Appendix 1 - Terms of Reference

A. Title

Afghanistan Program Summative Evaluation (FY 2004-2005 TO FY 2010-2011)

B. Definitions

  1. "Conflict-Affected State," includes Afghanistan, Burundi, Central African Republic, Chad, Democratic People's Republic of Congo, Republic of Congo, Cote d'Ivoire, Georgia, Guinea, Guinea-Bissau, Iraq, Liberia, Somalia, Tajikistan, West Bank & Gaza. This list does not represent the official views of CIDA or the Government of Canada.
  2. "Evaluation" is the systematic and objective assessment of an on-going or completed project, program or policy, its design, implementation and results. The aim is to determine the relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors.

    Evaluation also refers to the process of determining the worth or significance of an activity, policy or program. An assessment, as systematic and objective as possible, of a planned, on-going, or completed development intervention. Note: Evaluation in some instances involves the definition of appropriate standards, the examination of performance against those standards, an assessment of actual and expected results and the identification of relevant lessons. (Source: OECD, Development Assistance Committee (DAC) Working Party on Aid Evaluation)
  3. "CIDA Evaluation Committee," is an independent body of external development and evaluation experts, which advises the CIDA's President in ensuring the efficient implementation of the evaluation function and activities within CIDA.
  4. "Consultant's Evaluation Team," mean the Consultant's proposed professional and non-professional personnel who will fulfill the terms of this contract. 
  5. "Evaluation Advisory Committee," is a consultative body from CIDA and other federal departments, which provides advice to the CIDA Evaluation Directorate on the Afghanistan Program evaluation.
  6. "Evaluation Peer-Review Group", is an external consultative body, which provides advice on the evaluation design, implementation and the draft reports only to CIDA to ensure methodological rigour and neutrality.
  7. "Domain of Experience," refers to specific experience in the following areas:
    1. Evaluation experience (program/thematic) with multilateral organization;
    2. Evaluation experience (program/thematic) with a national development agency (i.e. CIDA, DFID, etc.);
    3. Evaluation experience (program/thematic) with an NGO;
    4. Evaluation experience on a single development project; and
    5. Other evaluation experience.
  8. "Fragile State," includes Afghanistan, Angola, Bosnia, Burundi, Central African Republic, Chad, Democratic People's Republic of Congo, Republic of Congo, Cote d'Ivoire, Eritrea, Georgia, Guinea, Guinea-Bissau, Haiti, Iraq, Kosovo, Liberia, Myanmar, Nepal, Sierra Leone, Somalia, South Sudan, Tajikistan, Timor-Leste, West Bank & Gaza, Yemen. This list does not represent the official views of CIDA or the Government of Canada.
  9. "Thematic Expertise" refers to the Afghanistan Program Evaluation themes (in alphabetical order): economic programming (including infrastructure and rural development), education, gender equality, governance, health, and humanitarian assistance.

C. Background

1. Introduction

CIDA is required to evaluate 100% of its direct program spending, including all grants and contributions every five years, in accordance with the 2009 Treasury Board (TB) Policy on Evaluation.  Therefore, a summative evaluation of the Afghanistan Program during 2011-12 and 2012-13 is required to fulfill both TB Policy and CIDA's commitment to undertake a second review of the Afghanistan Program, as per Treasury Board Submission B-07/0217. The objective of the Consultant is to design and implement the Afghanistan Country Program Evaluation.

2. Afghanistan Development Context

Afghanistan is a fragile state with a fluctuating security situation.  It remains among the world's least developed countries with most of its population lacking access to basic services.  In 2010, Afghanistan ranked 155th out of 169 countries on the UN's Human Development Index (HDI).  Forty-two percent of the country's population lives below the poverty line and life expectancy at birth is 44.6 yearsFootnote 45.  Years of conflict and poverty have undermined the human, infrastructure and state capacity of Afghanistan.
Gender inequality is prevalent and persistent in AfghanistanFootnote 46, creating additional barriers to sustainable development and poverty reduction.  Traditional social norms often restrict women and girls rights, such as freedom of movement.  They also impose particular burdens, such as early and forced marriage and gender-based violenceFootnote 47.  Continued gender gaps in education and high maternal mortality ratios are indicative of women's unequal access to services, and meaningful participation in decision-making.

However, the country has made important progress during the last decade.  For instance, Afghanistan has seen infant and under-five mortality rates drop by a fourth in recent years.  This reduction is the result of major efforts in constructing health centres and district hospitals, training community health workers and applying simple technologies such as standardized drug kits.  Similarly, the country has experienced significant but volatile economic growth, partly as a result of its reliance on the agricultural sectorFootnote 48.

Initial development results indicate that more girls are going to school than ever before, and women are increasingly engaging in politics.  Other important achievements include the Afghan Government accession to the UN Convention on the Elimination of All Forms of Discrimination Against Women without reservation; and the inclusion of gender in the Afghanistan National Development StrategyFootnote 49.

Progress on human rights has been slow and Afghanistan continues to suffer from a pervasive culture of impunity and a weak rule of law.  Freedom of expression remains threatened, with continuing reports of arbitrary arrests, detention and intimidation of journalistsFootnote 50.  Afghans perceive corruption as the country's third biggest problem after insecurity and unemploymentFootnote 51. In fact, Transparency International identified Afghanistan as the third most corrupt state in the world in 2010Footnote 52.  Finally, implementation and enforcement of legislation to protect social and economic rights also remains limited due to weak judicial institutionsFootnote 53.

The prevalent poverty and ongoing conflict affecting Afghanistan influence the scale and dimension of the country's humanitarian situation.  In 2011, the combination of poverty, conflict and natural disasters, many of which are linked to environmental degradation, is expected to leave 7.8 million Afghans in need of food assistance.  In addition, over 440,600 internally displaced people and half a million returning refugees are in need of emergency supportFootnote 54.

Women's access to humanitarian assistance has been difficult, with few actors identifying strategies to assess and respond to men and women's different needs and priorities. Humanitarian actors have expressed significant concern over the "shrinking of humanitarian space" due to insecurity, attacks on humanitarian workers and the real and perceived involvement of international and Afghan military forces in delivering humanitarian assistance. These factors have increased the risks for both humanitarian actors and beneficiaries.

Finally, the security situation in Afghanistan remains fragile, although the country has seen some improvements in recent years.  Since 2008, the Afghanistan government has taken significant steps to strengthen its ability to address the country's security needs.  However, the human cost of the armed conflict grew to 2,777 civilian deaths in 2010, an increase of 15% compared to 2009Footnote 55.  Growing insecurity has forced the closure or delay of a number of development projects and has limited the ability of both government officials and non-governmental organizations to extend services to conflict affected populationsFootnote 56.  Insecurity has simultaneously increased the need for humanitarian assistance, while also limiting its reach.

3. Afghanistan Program Context

The Afghanistan Program is CIDA's most significant in terms of magnitude, complexity, visibility, and challenges.  These challenges derive from a formidable combination of risks in areas such as the security situation, weak infrastructure and governance systems, limited Afghan institutional capacity for financial administration and risk management, and significant absorptive capacity constraints within the Government of the Independent Republic of Afghanistan (GIRoA).  Since 2001, CIDA's development efforts have supported a number of key Afghan national priorities aligned with Canada's priority areas of focus.Footnote 57

Canada's total allocation of development aid to Afghanistan over the 2001 to 2011 period stands at $1.9 billion, of which CIDA's total contribution is $1.6 billion (See Table 1).  Prior to, and including 2001, CIDA's focus was on emergency humanitarian assistance.  Beginning in 2002, this emphasis broadened to include investments supporting the process of democratic governance (such as presidential and legislative elections), and institutions at the national level, as well as the planning of national development programsFootnote 58.

Canada's strategy for Afghanistan during 2003-2005 aligned Canadian development assistance programming with the priorities of the Afghan government's National Development Framework (NDF).  CIDA selected three priorities for concentration from the NDF, in addition to support for the Afghanistan Reconstruction Trust Fund.

  1. Rural livelihoods and social protection: CIDA supported sustainable rural livelihoods for Afghan families and increased access to social services through improved governance and community-driven development.
  2. Natural resources management: CIDA supported improved food security and standards of living for Afghan citizens through rehabilitation and development of agriculture and the sustainable use of natural resources.
  3. Security and rule of law: CIDA contributed to the restoration of a secure environment for reconstruction and development, through peace building, de-mining, legal and judicial reform, and the strengthening of democratic institutions. 
  4. Afghanistan Reconstruction Trust Fund (ARTF):Given the lack of capacity for financial management and expenditure controls within the Afghan Government, Donors agreed on the establishment of the ARTF, a mechanism administered by the World Bank that channels international resources towards key government programs. 

In May 2006, CIDA's effort broadened further towards more sustainable development programs across Afghanistan, including an emphasis on Kandahar Province.  In fact, CIDA's Interim Strategy for Afghanistan (2006 – 2008) included four central components:

This broader engagement complemented the efforts of other government departments, including through a whole-of-government approach.  The 2007 Review of the Afghanistan Program stated that the needs of Afghanistan have been evolving since the inauguration of the provisional government in 2001, and Canada's involvement and strategy have had to keep pace with this evolution.

Table 1: CIDA Aid Disbursements to Afghanistan, 2000 to 2010Footnote 59
figures in $ millions2000 to 20062006/072007/082008/092009/10*Total
Bilateral Aid
Geographic Program Branch
Country programs335.96169.25270.14219.70204.831,199.87
Canada fund for local initiatives4.470.750.790.980.787.77
Canadian Partnership Branch1.980.770.690.760.925.12
Multilateral and Global Programs Branch (MGPB) Country Specific
Humanitarian Assistance Programs70.787.358.302.2023.63112.26
Other Programs with International Organizations58.290.750.340.000.3759.75
Other Bilateral Aid0.070.060.030.020.000.18
Sub-total Bilateral Aid 471.54 178.93 280.29 223.65 230.14 1,384.55
Multilateral Aid
Imputed Multilateral Core Funding22.905.549.193.187.9840.81
Total CIDA494.44 184.47 289.48 226.83 238.14 1,433.36

In 2007, the Afghanistan Program's Results and Risk Management and Accountability Framework (2007 – 2011) focused the Program on three pillars of the Afghan CompactFootnote 60.  These pillars were aligned with the Interim Afghanistan National Development Strategy (I-ANDS)Footnote 61.  A description of the revised focus and related programming is provided below:

  1. Governance.  CIDA supported service delivery and capacity-building initiatives for justice, electoral and human rights institutions, and helped develop an accountable and capable civil service.
  2. Social and Economic Development. CIDA supported the mainstreaming of gender equality across Afghan policies and programs, as well as funding initiatives that improve access for all Afghans to health and education services. It also implemented program activities related to economic governance, private sector development, basic infrastructure, and natural resources development.
  3. Security. CIDA funded initiatives that improved the security of Afghans (removing and destroying landmines, providing mine risk education); assisted with the resettlement and repatriation of internally displaced persons and returnees; and provided social assistance to vulnerable populationsFootnote 62.

In June 2008, CIDA's programming in Afghanistan for the 2008–2011 period was aligned with both the ANDS 2008-2013 and the recommendations from the Report to Parliament entitled Canada's Engagement in Afghanistan - Setting a course to 2011Footnote 63As a result,CIDA increased its aid and development programming in Kandahar by 50% of its total investment in the country. In addition, CIDA focused on decentralizing staff and delegating authorities to the field to increase effectiveness.  Promotion of equality between women and men and environmental sustainability were cross-cutting themes.Footnote 64

A series of key benchmarks, indicators and targets were developed in consultation with relevant departments in order to measure performanceFootnote 65.  Communication of results involved closely monitoring and reporting to Parliament on progress against key priorities, established benchmarks, and associated targets on a quarterly basis (CIDA was responsible for delivering on 24 of the 44 whole-of-government indicators).Footnote 66  Programming for this period was guided by the implementation of three priorities and three signature projects:Footnote 67

Priorities:

Signature Projects:

4. Evaluation Context

The Consultant must take into consideration the Agency's international aid effectiveness commitments, including the Paris Declaration (2005) and the Accra Action Agenda (2008). It will also consider the emerging international consensus on development cooperation (e.g. Millennium Development Goals) and CIDA's key steps to implement Canada's Aid Effectiveness Agenda (i.e. greater focus, effectiveness and accountability)Footnote 68.

5. Principles for Evaluating Interventions in Fragile and Conflict-Affected States

The Consultant must conform to the principles, standards and practices set out by the DAC's Evaluation Quality Standards for Development Evaluation when performing the Afghanistan Evaluation Program.
CIDA will ensure that the Afghanistan Program evaluation adheres to the Treasury Board of Canada 2009 Policy on EvaluationFootnote 69 and accompanying DirectiveFootnote 70 and StandardFootnote 71

Constraint: The Afghanistan Program evaluation must integrate in its methodological approach and analysis the implications of Afghanistan's humanitarian crisis, its conflict and fragile situation.Footnote 72  The OECD/DAC Networks on Development Evaluation and on Conflict, Peace and Development Cooperation provide direction on integrating these considerations, mainly through the following documents:

In particular, the evaluation will assess the extent to which the Afghanistan Program incorporated in its programming design, delivery and management the OECD/DAC Principles for Good International Engagement in Fragile States and Situations.  These principles are:

  1. Taking context as the starting point;
  2. Doing no harm;
  3. Focusing on state-building as the central objective;
  4. Prioritizing prevention;
  5. Recognizing the links between political, security and development objectives;
  6. Promoting non-discrimination as a basis for inclusive and stable societies;
  7. Aligning with local priorities in different ways and in different contexts;
  8. Agreeing on practical coordination mechanisms between international actors;
  9. Acting fast but staying engaged long enough to give success a chance; and
  10. Avoiding pockets of exclusion.

Given the complex nature of the situation in Afghanistan, it is paramount that the Afghanistan Program evaluation integrates the notion of conflict sensitivity in its design, planning and implementation.  Sometimes policies, projects and programs working "in" or "on" conflict do harm – often without intending to.Footnote 75  Doing harm in a conflict situation means having impacts (intended or not, direct or indirect) that aggravate grievances, increase tension or vulnerabilities, or perpetuate conflict in some way.  The notion of conflict sensitivity is intended to mitigate such harm by encouraging systematically taking into account both the positive and negative impacts of interventions on the conflict contexts in which they are undertaken.Footnote 76

Conflict can also play a major role in forming a society's understanding of, and responses to, gender roles.  Violent conflict is often accompanied by a surge in violence towards women.Footnote 77.  Therefore, the Afghanistan Program evaluation will present a clear and critical understanding of gender within the particular context of Afghanistan.

6. Evaluation Audience

The Afghanistan Program evaluation will provide to CIDA's President, CIDA's senior management, CIDA's external Evaluation Committee, the Afghanistan Program Task Force (APTF) program management, central agencies, and other government departmentsFootnote 78 with key lessons and recommendations to support evidence-based decision making on policy, expenditure management and program improvements. 

The audience for this evaluation also includes the Government of Afghanistan, partner NGOs and other development agencies working with CIDA in this country.  The evaluation will inform these stakeholders about what was achieved by and learned from CIDA's development interventions in Afghanistan.  It will also provide them with information methodologies and approaches related to evaluating development interventions in conflict and fragility situations.

As per CIDA's evaluation policy, sharing evaluation reports with key audiences in Canada and abroad demonstrates accountability and transparency to Canadians, and benefits development cooperation.  The final Afghanistan Program evaluation report will be made available to Parliament and Canadians through the Agency's website, as per CIDA's efficiency and accountability commitments and Treasury Board of Canada Evaluation Policy. 

Disseminating the evaluation findings will be an integral component of the evaluation process. The CIDA Technical Authority will work with the Consultant to ensure that evaluation knowledge is disseminated, based on CIDA Evaluation Directorate's Strategy for the Dissemination of Evaluation Knowledge.

7. Evaluation Challenges

The Consultant will need to take into account the following challenges when conducting the Afghanistan Program evaluation:

D. Rationale and objectives

1. Evaluation Rationale

The Consultant will conduct the Afghanistan Program Evaluation to satisfy the reporting requirements of the Canadian Government, including those stemming from the Federal Accountability Act (2006) and the Treasury Board Secretariat (TBS) of Canada.  In particular, the rational for the Afghanistan Program evaluation is threefold. 

2. Scope – Objectives and Questions

The Consultant will assess the Program in terms of the following six criteria: relevance, coherence, performance (effectiveness, efficiency, and economy), sustainability, management principles/Paris declaration, and cross-cutting issues.  The Consultant will assess these six criteria for the period covering fiscal year (FY) 2004/05 to FY 2010/11.  

The Consultant will consider in his/her analysis the broader whole-of-government effort in which CIDA's effort was situated.  S/he will differentiate between traditional development activities and those that were part of stabilization efforts (e.g. development work in Kandahar).  S/he will also consider the degree to which the tools available to team leaders in the field helped or hindered the delivery of programming in a conflict environment.  Finally, the Consultant will assess how CIDA's efforts evolved pre and post 2008, including an analysis of successes and challenges.

The specific objectives for the Consultant are to:

At a minimum, the following questions will be addressed for both the pre and post 2008 period of the Afghanistan Program.  However, these questions may be modified during the evaluation design phase to reflect the particular objectives, challenges and focus of the Afghanistan Program pre and post 2008.

The Consultant must answer the following questions in the final report:

Relevance

Coherence

Performance (effectiveness; efficiency; and economy)

Sustainability

Management principles/Paris declaration

Cross-cutting issues

Was the Afghanistan Program effective in addressing the cross-cutting themes of gender equality, governance and environmental sustainability?

E. Description of services to be provided

Evaluation Approach and Methodology

Given the scope, nature and challenges of the Afghanistan Program, the Consultant will carry out the evaluation in two separate phases: i) Evaluation Design and ii) Evaluation Implementation.  The Terms of Reference cover both phases.

Stakeholder participation is fundamental to CIDA's Afghanistan Program Evaluation.  The Consultant must provide for active and meaningful involvement by investment partners, beneficiaries and other interested parties.  The Consultant must ensure that stakeholder participation is an integral component of the evaluation design and implementation, including during data collection, the development of findings, evaluation reporting, and results dissemination.  The Consultant must include an analysis of stakeholders' views on evaluation needs and priorities in the evaluation design.

A detailed description of the two phases is provided below:

Phase 1 – Evaluation Design

he Consultant must define a realistic and useful methodology, approach, focus, and timing for the Afghanistan Program evaluation.  The Afghanistan Program staff will have an opportunity to comment at the various milestones of the evaluation design phase, including during development of the evaluation methodology. 

The specific objectives for the Consultant during this phase are:

During the evaluation design phase, the Consultant will engage in a field mission to Afghanistan, which will include interviews with key stakeholders, preliminary data collection and review of documents, reports and analysis provided by stakeholders.  The scoping mission is expected to be no longer than one week in duration. 

The Consultant will also analyze the security risks implications for the evaluation budgeting and management.  In particular, the Consultant's proposed methodology will provide for a clear distinction between the different result levels (immediate, intermediate and ultimate outcomes) and delivery mechanisms (program and project-based approaches through grants and contributions).  The Consultant must provide justification for his/her proposed evaluation methodology, including his/her approach to data collection and analysis.

The Consultant must explore the possibility of joint evaluation work with other donors.  The Consultant must assess the logistical and other considerations related to undertaking joint evaluations, including by providing details on costs, timing, etc.  The Consultant must provide a final recommendation on this matter in the work plan.

NOTE:  Upon approval of the Work Plan for the Implementation Phase from Phase 1, the CIDA Contracting Authority will notify the Consultant in writing either that CIDA intends to proceed with Phase 2 or that CIDA wishes to withdraw any further support in the project, the Consultant selected to implement the first phase of this mandate may or may not implement the second phase.  Upon completion of Phase 1, the CIDA Technical Authority reserves the right to seek senior management guidance and approval for the methodology, approach, focus, timing, and other issues.  If CIDA decides to withdraw its support, the CIDA Contracting Authority will so inform the Consultant in writing and the Contract will be deemed to have come to an end without any cost or liability to CIDA.

Phase 2 – Evaluation Implementation

Upon receiving the approval from CIDA to proceed with Phase 2, the Consultant must implement the evaluation work plan, including its fieldwork in Afghanistan.  The Consultant must make all logistical decisions in consultation with the APTF and other government departments.  Subject to the timelines provided in the CIDA-approved implementation phase work plan, the Consultant must engage in data collection and analysis during the Fall 2012 (see tentative timeline below).

The Consultant must deliver its draft report to the Technical Authority upon the agreed timeline by both parties.  The final report must describe the evaluation and present the evaluation findings, recommendations and lessons learned.  The presentation of results is to be intrinsically linked to the evaluation issues, establishing a logic flow from the information collected.  The Final Evaluation Report will be approved by the CIDA Evaluation Committee.

F. Resources and Timelines

CIDA's tentative timeline for the Afghanistan Program evaluation is provided below.

Afghanistan Evaluation Tentative Timeline
(from date of contract signature)
Activities & Deliverables \ Target DateMonths
123456789101112131415
Phase 1
Work Plan for the Design Phase  (draft)X
Work Plan for the Design Phase  (final)X
Literature Review Report (draft)X
Literature Review Report (final)X
Scoping MissionX
Scoping Mission Report (draft)X
Scoping Mission Report (final)X
Evaluation Methodology (draft #1)X
Evaluation Methodology (draft #2)X
Evaluation Methodology (final)X
Work Plan for the Implementation Phase (draft #1)X
Work Plan for the Implementation Phase (draft #2)X
Work Plan for the Implementation Phase (final)X
Data Collection and AnalysisXXX
Data Gathering Report (draft)X
Data Gathering Report (final)X
Evaluation Report (draft #1)X
CIDA Technical Authority ReviewX
Phase 2Footnote 80
Evaluation Report (draft #2)X
GoC ConsultationsX
Management ResponseX
Evaluation Report (draft #3)X
Evaluation Committee Review and ApprovalX
Evaluation Report (final)X

G. Description of the team

The Consultant must propose a multi-disciplinary team to design and implement the Afghanistan Program evaluation, including one Evaluation Team Leader and sector specialists as required.  The Team Leader and Sector Specialists will participate in both Phase 1 and Phase 2.  Specifically, the team must include:

1. Evaluation Team Leader

Specific Responsibilities

This resource will be responsible for the following:

2. Sector Specialists

Specific Responsibilities

Each of these resources will be responsible to conduct the following (as requested by the Evaluation Team Leader):

3. Local Professionals

Specific Responsibilities

The Consultant is to identify relevant and appropriate local consulting expertise available in Afghanistan.  Each of these resources will be responsible to conduct some or all of the following:

H. Language requirements

The following resources must possess the following levels in English:

The description associated with the language requirements can be found at the Department of Foreign Affairs and International Trade (DFAIT) website.

I. Roles and responsibilities

Consultant

Phase 1: Evaluation design

(including all draft and final versions as noted under "Deliverables")

Phase 2: Evaluation implementation

CIDA's Technical Authority

CIDA's Evaluation Advisory Committee

The Afghanistan Program evaluation will be guided by CIDA's Evaluation Advisory Committee, which is composed by a Director-level representative from the Afghanistan Program Task Force (APTF) and the International Humanitarian Assistance Directorate.  It is chaired by the CIDA Director from the Evaluation Directorate. Representatives from the Department of Foreign Affairs and International Trade (DFAIT), and the Department of Defence will be invited to participate in the CIDA's Evaluation Advisory Committee meetings on an ad-hoc basis. CIDA's Evaluation Advisory Committee will act as a consultative body.

CIDA's Evaluation Advisory Committee provides guidance and recommendations to the CIDA Technical Authority on the overall evaluation. CIDA's Evaluation Advisory Committee communicates their advice and recommendations directly to the CIDA Technical Authority, who may request the Consultant to incorporate the feedback within the deliverables.

Evaluation Peer-Review Group

The Evaluation Peer-Review Group will advise the CIDA Technical Authority on methodology. The group is made of:

The Evaluation Peer-Review Group provides advice on the evaluation design and implementation as well as provides recommendations to the CIDA Technical Authority on how to improve draft deliverables to be completed under this evaluation. The Evaluation Peer-Review Group communicates their advice and recommendations directly to the CIDA Technical Authority, who may request the Consultant to incorporate the feedback within deliverables.

J. Deliverables

Phase 1: Evaluation design

These deliverables will be prepared in English and submitted to the CIDA Technical Authority electronically (i.e. via email or memory stick).

i. Work Plan for the Design Phase

The Consultant must prepare a detailed work plan for the design phase (see Annex 1). The work plan for the design phase will elaborate on the proposed approach to implement the TORs mandate, providing specific timelines, deliverables, level of effort per team member, and budget information.

The Consultant must submit a draft work plan for the design phase to the CIDA Technical Authority within (2) weeks of signing the Contract. The CIDA Technical Authority will provide comments on the draft work plan for the design phase within (1) week from its receipt. Within (1) week of receiving comments from the CIDA Technical Authority, the Consultant submit for CIDA's approval a final work plan for the design phase that incorporates CIDA's comments.

CIDA will base its approval of the document on the sample Work Plan Table of Contents provided in Annex 1 and on the DAC's Evaluation Quality Standards for Development Evaluation (PDF)

ii. Literature Review Report

The Consultant must conduct a literature review of Afghanistan's socio-economic situation, development challenges and achievements.  The literature review must present a clear and critical understanding of gender within the particular context of Afghanistan.  The Consultant must also discuss current issues on evaluating development interventions in fragile states and situations, including evaluating conflict prevention and peace building activities, as well as potential evaluation approaches.

In the Literature Review Report, the Consultant must include a profile of the Afghanistan Program, which provides a detailed description of the Program context and its programming evolution since 2004.  This section of the literature review report must be based on various documents and reports provided by CIDA (including internal analysis prepared in anticipation of the Afghanistan Program Evaluation).  The Literature Review Report must also include a complete list of sources.

The CIDA Technical Authority will provide comments on the draft literature review report within (1) week from its receipt. Within (1) week of receiving comments from the CIDA Technical Authority, the Consultant will submit for CIDA's approval a final literature review report that incorporates CIDA's comments.

iii. Scoping Mission Report

In this report, the Consultant must summarize the findings and recommendations emanating from the one-week scoping mission to Afghanistan.  The report will also present the data collected during the scoping mission, including information collected through interviews with key stakeholders as well as a list of documents and reports provided by local stakeholders.

The Consultant must provide in the report specific recommendations for the development of the evaluation methodology, including on managing security risks, engaging local consultants, access to project sites, etc.

The Consultant must submit the draft scoping mission report within (2) weeks from returning from Afghanistan.  The CIDA Technical Authority will provide comments on the draft scoping mission report within (1) week from its receipt. Within (1) week of receiving comments from the CIDA Technical Authority, the Consultant must submit for CIDA's approval a final report that incorporates CIDA's comments.

iv. Evaluation Methodology

The Consultant must ensure that the evaluation methodology maintains coherence with the larger contribution of the Afghanistan Program for the period FY 2004/05 – FY 2010/11, while taking into consideration the different objectives/logic models guiding programming during this period.  The Consultant must consider the implications of Afghanistan's humanitarian crisis and its ongoing conflict and fragility.  Based on the literature review report, the Consultant's methodology must integrate best practices derived from the OECD's recent experience in evaluating development interventions in fragile states and situations, including integrating the notion of conflict sensitivity.  The Consultant's methodology must also take into consideration the security risks related to implementing fieldwork in Afghanistan.

CIDA's Technical Authority will provide comments on the first draft of the evaluation methodology within two (2) weeks from its receipt.  Within two (2) weeks of receiving comments from the CIDA Technical Authority, the Consultant must deliver a second draft evaluation methodology that incorporates CIDA's comments.

The Evaluation Peer-Review Group and CIDA's Advisory Committee will provide comments on the second draft of the evaluation methodology within two (2) weeks from its receipt.  Within two (2) weeks of receiving these comments, the CIDA Technical Authority will provide these comments to the Consultant, the Consultant must submit for CIDA's approval a final evaluation methodology that incorporates the comments from the Evaluation Peer-Review Group and CIDA's Advisory Committee.

v. Work Plan for the Implementation Phase

Based on the literature review and methodology, the Consultant must determine the most appropriate approach for implementing the Afghanistan Program evaluation.  The Consultant must expand the work plan for the design phase to include the tasks contained in the implementation phase of the Afghanistan Program evaluation.  The Consultant must prepare a detailed evaluation work plan that fulfills these TORs and provides recommendations to address the expected challenges (see section C: Background, paragraph 7, Evaluation Challenges).  Finally, the work plan for the implementation phase will integrate multiple information sources and lines of evidence, including documentation review, interviews and field visits, where and if appropriate. At a minimum, the work plan for the implementation phase will address the following elements:

CIDA's Technical Authority will provide comments on the draft work plan for the implementation phase within two (2) weeks from its receipt.  Within two (2) weeks of receiving comments from the CIDA Technical Authority, the Consultant must deliver a second draft work plan for the implementation phase that incorporates CIDA's comments.

The Evaluation Peer-Review Group and CIDA's Advisory Committee will provide comments on the second draft work plan for the implementation phase within one (1) week from its receipt.  Within one (1) week of receiving these comments, vis-à-vis the CIDA Technical Authority, the Consultant must submit for CIDA's approval a final work plan for the implementation phase that incorporates the comments from the Evaluation Peer-Review Group and CIDA's Advisory Committee.

Note:  The Phase 2 evaluation implementation will begin only if CIDA's Contracting Authority has authorized, in writing, the Consultant to continue with the evaluation. 
CIDA will base its approval of the document on the sample Work Plan Table of Contents provided in Annex 1 and on the DAC's Evaluation Quality Standards for Development Evaluation (PDF).

Phase 2: Evaluation implementation

These deliverables will be prepared in English and submitted to the CIDA Technical Authority electronically (i.e. via email or memory stick).

i. Data-Gathering Report

In this report, the Consultant will summarize the activities of the data-gathering mission to Afghanistan.  The report will include the following:

The Consultant must submit the draft data-gathering report within two (2) weeks from returning from Afghanistan.  The CIDA Technical Authority will provide comments on the draft data gathering report within (1) week from its receipt. Within (1) week of receiving comments from the CIDA Technical Authority, the Consultant must submit for CIDA's approval a final report that incorporates CIDA's comments.

ii. Progress Reports

The Consultant must produce monthly progress reports during the implementation phase.  The monthly progress reports will provide an update on the status of the evaluation implementation, lessons learned, challenges, and steps undertaken by the Consultant to mitigate these challenges.

iii. Evaluation ReportFootnote 81

The draft Evaluation Report must be formatted using the CIDA's Evaluation Directorate's Report template.  The Consultant must submit a first draft evaluation report for review to the CIDA Technical Authority within four (4) weeks of the end of the data collection process.  At a minimum, the report should have the following structure:

CIDA's Technical Authority will provide comments on the first draft Evaluation Report within one (1) week from its receipt.  Within one (1) week of receiving comments from the CIDA Technical Authority, the Consultant must deliver a second draft Evaluation Report that incorporates CIDA's comments.

The Evaluation Peer-Review Group and CIDA's Advisory Committee will provide comments on the second draft of the Evaluation Report within two (2) weeks from its receipt.  Within one (1) week of receiving these comments, the Consultant must submit for CIDA's approval a third draft of the Evaluation Report that incorporates the comments from the Evaluation Peer-Review Group and CIDA's Advisory Committee.

The CIDA Technical Authority will send the third draft of the Evaluation Report to the CIDA Evaluation Committee for their comments and approval.  If there are comments from the CIDA Evaluation Committee, the Consultant will address these comments in the final Evaluation Report.

iv. Source Data

Upon completion of the contract, the Consultant must provide electronic versions of data compiled for the Afghanistan Program evaluation.  For any new data that the Consultant directly collects on behalf of this evaluation and provides to CIDA, all identifying information about participants must be removed from the data before it is given to CIDA.

v. Approval of the documents

CIDA will confirm that all deliverables meet OECD-DAC Standards for evaluation.  CIDA will also use the following criteria when reviewing the draft Evaluation Report approval:

Good evaluation reports strike a balance between depth and length.  Reports should be approximately 30 pages, with additional materials provided in annexes. The report should include:

The Evaluation Report must be credible (ensure the information is validated and triangulated) and convincing to readers within CIDA. The flow of information must make sense and does not confuse the reader. The Evaluation Report is also the primary instrument by which the Canadian public will access the evaluation results. Thus, it is essential that the report be clear, readable, concise, powerful, and persuasive to the general professional reader.

Annex 1: Sample Outline of the Evaluation Work Plan

Annexes

Appendix 2 - Logical Framework for CIDA's Afghanistan Program (2004-2009)

Appendix 2 Logical Framework for CIDA’s Afghanistan Program (2004-2009)
Priorities/ PillarsCore Budgetary FundingSecurity and Rule of LawRural Livelihoods and Social ProtectionNatural Resources Management
Specific Program Activities1. Budgetary Support for the GoA.2. Programming in Security and the Rule of Law Pillar: ASP, Justice Sector Reform, Demining, DDR, SSR, Elections, Aid coordination.3. Programming in Rural Livelihoods and Social Protection Pillar: MISFA, NABDP, NSP, National Gender Program.4. Programming in Natural Resources Management Pillar: Water for Agriculture Studies and Projects.
Outputs1. Operating budget for the functioning of the GoA.2. Judicial infrastructure; training of judges, police, prosecutors, legislators, public servants; mine removal; demobilization of ex-combatants; appropriate legal framework.3. Micro loans for poor people, particularly women; basic social and physical infrastructure; vocational training and education for women.4. Studies on irrigation possibilities in the Hari Rud and Kajakai Basins; irrigation infrastructure.
Immediate Outcomes
(Short Term)
1. The government functions effectively as its recurrent budgetary expenses are met and is enabled to maintain and expand the delivery of basic services to its citizens.2. An appropriate legal framework, a functioning judicial system, effective law enforcement mechanisms and functioning governance institutions (national and sub-national) are developed, exist and/or are enforced.3. Micro and small enterprises established by rural poor; basic human needs in areas such as health and basic infrastructure, increasingly met; community participation strengthened; financial self-reliance of women engendered.4. Increase in food and non-food agricultural crops generate improved food security and additional income generating activities.
Intermediate Outcomes (Medium Term)1. and 2. Emergence of a government administrative capacity and of a well structured and functioning security sector that can provide services and promote safety and security for an increased number of Afghans at the national and sub-national levels.3. and 4. Selected basic social and economic needs of the people are increasingly met.
Final Outcome (Long Term)Consolidation of central government's authority and legitimacy across the nation, and improvement in people’s well being, enable the increased stabilization of the State of Afghanistan.

Appendix 3 Afghanistan Program Logic Model (2007-2011)

Appendix 3 Text Alternative
Afghanistan Program Logic Model (2007-2011)
PillarsPrioritiesPointsCIDA ActivitiesProgram Activities Executed by the Recipients of CIDA FundsOutputsImmediate OutcomesIntermediate OutcomesFinal Outcome
GovernanceRule of LawRule of LawPerforming policy research, planning sectoral direction and viable entry points; designing programs and projects and managing proposals from prospective implementing agencies; providing funding to implementing agencies; coordinating program and policy dialogue with donors, stakeholders, and GoA partners; participating in steering committees and working groups; monitoring and reporting of program delivery and results; leading Canadian monitoring and sectoral level missions to Afghanistan.
  • Training lawyers, judges, and legal advisors;
  • supporting legal reforms
  • Policy advice provided on legal reforms;
  • legal services provided;
  • trained judges, lawyers and advisors
  • Increased access for citizens to justice system
  • Enhanced capacity of judges, lawyers and legal counsels to impart fair and transparent legal aid
  • Increased use of legal services
  • Greater trust and confidence in the justice system
A more stable, self-reliant, and democratic Afghanistan that contributes to national, regional, and global security
Democratic Development and Effective GovernanceDemocratic Institutions and Processes
  • Supporting the dev’t of policies and electoral processes;
  • dev’g community outreach materials & establishing networks
  • Policies and electoral processes developed;
  • public education activities completed
  • Enhanced capacity of the GoA to manage public institutions and deliver services
  • Better functioning and efficient National Assembly
  • Increased awareness of electoral processes and institutions
  • A legitimate and effectively functioning state
  • Greater trust and participation of men and women in political and electoral processes at national, provincial and local level
Administration Reform
  • Training public sector staff;
  • developing infrastructure to support sub-national governance for key government services
  • Financial, mngt and technical training provided;
  • capacity building inside line ministries
  • performed; subnational governance institutions est.
Human RightsHuman Rights
  • Supporting the dev’t of human rights legislation, policies, and legal reforms;
  • training on human rights awareness; establishing human rights fora
  • Policy advice provided on legal reform;
  • human rights’ fora established; human rights training provided
Enhanced awareness of and support for human rights by AfghansEntrenchment of universal rights of all Afghans in society
Social and Economic DevelopmentGreater Access to Social ServicesGender Equality
  • Supporting the mainstreaming of GE legislation, policies and programs;
  • providing skills training to women;
  • establishing women’s organizations
  • Advice provided on gender mainstreaming, gender legislation and reforms;
  • women’s organizations established
  • Increased capacity to deliver essential health and education services
  • Improved policy and programming that furthers women’s socio-political and economic participation
  • Improved access to health and education services
  • Increased participation of women in political, economic and social sectors of society
Access to Health
  • Health training to staff, developing primary health infrastructure;
  • delivering child and maternal health services & immunization campaigns
  • Trained health workers; child and maternal health services provided;
  • primary health infrastructure developed
Access to Education
  • Teacher-training; capacity building within all levels of govt educational portfolio;
  • life & vocational skills training for women;
  • building educational
  • Teacher-training completed; educ. facilities built;
  • capacity-building performed within educ. sector;
  • life & vocational skills train’g courses est
Economic Development for the PoorEconomic Governance
  • Providing budgetary and capacity building support;
  • supporting the design and implementation of business macro-economic reforms
  • Macro-economic advice provided;
  • public programs & services for economic dev’t established;
  • business-support services established
  • Enhanced public sector capacity to develop and implement effective and transparent macro-economic, fiscal, and monetary measures, regulations, policies and services, and to finance development through domestic revenues
  • Improved public and private sector capacity to sustain enhanced productivity, competitiveness, trade and investment, and licit economic diversification (e.g. reduction on reliance on narcotics)
  • Improved public and private sector capacity to sustainably operate and maintain economic infrastructure, and to manage and use natural resources
  • Sustained macroeconomic stability and sound economic, fiscal and monetary management in the public sector
  • Increased employment and income-generating opportunities
  • Reliable and affordable access by private sector to sustainably managed economic infrastructure and natural resources
Private Sector Development
  • Providing financial and technical services, and microcredit;
  • providing business and market development services that encourage licit economic opportunities
  • Microfinancial & tech. services provided; licit market development services provided;
  • agriculture & livestock services provided;
  • licit trade and investment ops generated
Infrastructure and Natural ResourcesDeveloping land and water management systems and small-scale economic infrastructure
  • Water and land management systems developed;
  • economic infrastructure established
SecurityHuman SecurityHuman and Physical Safety
  • Providing humanitarian assistance to returnees and IDPs;
  • surveying and destruction of ammunitions;
  • delivering demining activities and mine awareness training
  • Return and reintegration assistance provided to returnees and IDPs;
  • ammunitions surveyed and destroyed;
  • demining and mine action programs completed
  • Increased role and capacity of the GoA in mine action coordination
  • Improved planning, responsiveness and management of humanitarian needs
  • Improved protection for vulnerable populations and for return and reintegration of IDPs and returnees
  • Improved human security (with respect to landmines and armaments)

Appendix 4 Afghanistan Program Logic Model (2008-2011)

Appendix 4 - Text Alternative
Afghanistan Program Logic Model (2011-2014)
ANDS PillarsCIDA PrioritiesCIDA Priorities 2 levelCIDA ThemesCIDA ActivitiesProgram Activities Executed by Recipients of CIDA FundsOutputsImmediate OutcomesIntermediate OutcomesUltimate Outcome
Social and Economic DevelopmentBasic ServicesAccess to EducationChildren and Youth
  • Performing policy research, planning sectoral direction and viable entry points;
  • designing programs and projects and managing proposals from prospective implementing agencies;
  • providing funding to implementing agencies; coordinating program and policy dialogue, and implementing principles of aid effectiveness with donors, stakeholders, and Government of Afghanistan partners; participating in steering committees and working groups;
  • monitoring and reporting of program delivery, risks and results;
  • leading Canadian monitoring and sectoral level missions in Afghanistan.
  • Providing teacher training;
  • developing community-based education services;
  • building or rehabilitating educational infrastructure;
  • providing literacy, vocational and life skills development training;
  • supporting community capacity development.
  • Teacher training completed;
  • community-based education services established;
  • educational facilities built or rehabilitated;
  • literacy, vocational and life skills development training provided;
  • local community capacity development supported.
  • Improved capacity of national, provincial and local institutions to deliver basic educational services to Afghan girls and boys
  • Enhanced capacity of communities to identify and implement community development projects
  • Improved access to financial and business services for SMEs and entrepreneurs (Kandahar focus)
  • Improved public and private sector capacity to sustainably manage natural resources in Kandahar province
  • Improved access to quality education services for women and girls (Kandahar focus)
  • Improved access to employment and income opportunities for women and men (Kandahar focus)
  • Improved productive potential of land and livestock (Kandahar focus)
A more secure Afghanistan, and Kandahar Province in particular, that is able to deliver key services to Afghans, and better provide for its longer term stability and sustainable development
Economic GrowthEconomic Growth
  • Planning and implementing community-based infrastructure and livelihoods projects;
  • developing water management systems;
  • providing market-oriented vocational and skills development training;
  • supporting development of agriculture and extension services.
  • Community-based infrastructure and livelihoods projects delivered;
  • water management systems developed;
  • market-oriented vocational and skills development training provided;
  • agricultural extension services supported.
SecurityHumanitarian AssistanceAccess to HealthChildren and Youth
  • Conducting immunization campaigns;
  • providing health care training to health workers;
  • developing essential health infrastructure.
  • Immunization campaigns conducted;
  • health workers trained;
  • essential health infrastructure supported.
  • Increased capacity of public health institutions to deliver essential quality health services to vulnerable populations
  • Improved capacity of public institutions and communities to plan and coordinate emergency assistance
  • Increased capacity and role of the GoA in mine action coordination
  • Improved access to essential quality health services for women and children among other vulnerable populations
  • Reduced vulnerability to poliovirus and other diseases among populations
  • Improved protection of returnees, refugees, IDPs, and among other vulnerable populations
  • Improved human security with respect to mines and explosive remnants of war
Emergency AssistanceFood Security
  • Supporting the development of Afghan disaster management organizations and structures;
  • providing food assistance to vulnerable populations.
  • Afghan disaster management organizations and structures supported;
  • food assistance provided to vulnerable populations.
Mine ActionNo thematic link
  • Surveying and clearance of explosive remnants of mines and other remnants of war;
  • providing mine risk awareness training.
  • Land surveyed, cleared and released to communities;
  • mine risk awareness education delivered.
GovernanceNational Institutions and Democratic DevelopmentPublic InstitutionsEnabling environment contributing to children and youth, food security, and economic growth
  • Providing technical assistance to key Afghan institutions;
  • supporting development of provincial strategic plans;
  • supporting the development of subnational governance institutions.

Technical assistance provided to key Afghan institutions;

provincial strategic plans developed;

sub-national governance institutions established and supported.

  • Improved capacity of key public institutions to plan, budget, resource, manage and maintain the delivery of services at national and subnational levels
  • Strengthened capacity of Afghan electoral institutions and stakeholders to participate effectively in political and electoral processes
  • Enhanced democratic governance, effectiveness and accountability of public institutions
  • Increased participation of men and women in political and electoral processes at national, provincial and local levels
Electoral ProcessesNo direct contribution to thematic areas
  • Conducting candidate, voter and civic education and outreach campaigns;
  • developing institutional and management capacity of electoral institutions and oversight entities.
  • Candidate, voter and civic education and outreach activities completed;
  • electoral management and oversight entities established and supported.

Appendix 5 Afghanistan Program Logic Model (2011-2014)

Appendix 5 Afghanistan Program Logic Model (2011-2014)
Ultimate OutcomeIntermediate OutcomesImmediate OutcomesOutputsActivities
Basic needs met and reduced vulnerability of the people in Afghanistan, with a focus on women and girls  Children and Youth:  EducationIncreased access for girls and boys to relevant learning opportunities that respond to their different priorities and interestsIncreased availability of quality learning spaces for both girls and boys.Educational facilities establishedfor girls and boys (either mixed or single-sex facilities) Enabling infrastructure established (e.g. latrines, school boundary walls, transportation) to respond to the different needs and requirements for girls and boys to go to schoolOperational:
EQUIP, AKFC
BRAC, QPEP
UNICEF-BEGE
Afghanistan Challenge Planned:
Increased support to community based education
Other Program Activities:
Participate in EQUIP Donor Group, and the Afghanistan Girls' Education Initiative working group.
Increased availability of male and female school teachers who reflect community need and meet gender-specific demand.CBE teachers, particularly female ones, are identified and recruited. Community-based classes established, especially for women and girls.Operational:
BRAC, QPEP, UNICEF-BEGE Planned:
Increased support to community based education
Other Program Activities:
Chair of CBE working Group (promotes coordination amongst partners and with the MoE)
 Enhanced delivery of quality learning opportunities for girls and boys through the education systemImproved ability of local, provincial and national institutions to deliver basic education services nationally.Technical assistance in management provided to the Ministry of Education Policy development processes supported and coordinated within the Ministry of EducationOperational:
EQUIP, AKFC
BRAC
UNICEF-BEGE, Education in emergencies (CARE) Planned:
TC&A (MoE and Partner)Technical assistance
Other Program Activities:
Participate in the Education Management Working Group (strategic guidance and support services for implementation of education programs)
Improved ability of teachers and educators to deliver quality basic education.Teachers and educators provided with appropriate training. Teacher Training accreditation system established
Teacher certification system established
Operational:
EQUIP, AKFC
UNICEF- BEGE
BRAC, QPEP Planned:
Teacher certification and accreditation
Other Program Activities:
Increased ability of communities to engage in basic education-related decision-making processes.Local community school governance bodies established, inclusive of both women and men. Infrastructure and other needs-based grants disbursedOperational:
EQUIP, AKFC
BRAC, QPEP Planned:
Other Program Activities:
Children & Youth: Maternal, newborn and child health Increased equitable and gender-sensitive health services to mothers, newborns and children under-five. Footnote 82Enhanced capacity of government to plan, implement, and evaluate maternal, neonatal and child health strategies and programs.Technical support provided to the Ministry of Public Health to strengthen its leadership and coordination of the health sector, especially the maternal, newborn and child health sub-sector. Routine collection of health information, specific to maternal, neonatal and child health in particular, including gender-sensitive indicators and reliable sex-disaggregated data, at various levels of the health system.Operational:
WB SHARP
HPIC CBAM Planned:
H4+ Secretariat
H4+ Action Plan
GPFSS Other Program Activities:
Chair – Health Donor Coordination Group
Participant – Consultative Group on Health and Nutrition
Technical support to the Ministry of Public Health for Strategic Framework development
Increased capacity of government and health institutions to deliver effective, safe, quality and standardized health services for mothers, newborns and children under-five.Pre- and in- service training on evidence-based, high-impact health interventions, including health promotion and disease prevention, provided to health practitioners, male and female. Health infrastructure strengthened for the provision of maternal, neonatal and child health services at the appropriate level of the health system.Operational:
WB SHARP
HPIC CBAM
UNICEF MCH Planned:
Regional MNCH centre
H4+ Action Plan Other Program Activities:
TBD
Enhanced utilization of essential health commodities and supplies needed to prevent, manage and treat the main causes of death among mothers, newborns and children under five, including gender-based inequalities and harmful practices.Enhanced awareness among men, women and children of the existence and importance of using available health resources to prevent, manage and treat the major causes of excess maternal, neonatal and child morbidity and mortality.Local health practitioners and volunteers, male and female, trained to communicate the existence and importance of using available health resources to prevent, manage and treat the major causes of excess maternal, neonatal and child morbidity and mortality. Messages on behaviours and practices and available resources related to health promotion and disease prevention/management/treatment disseminated at community gatherings.Operational:
WB SHARP
GPEI
UNICEF MCH
WHO TB Planned:
Regional MNCH centre (TBD)
H4+ Action Plan Other Program Activities:
TBD
Increased and equitable access to effective, safe and quality health goods and services aimed at preventing, managing and treating the most common and severe diseases affecting mothers, newborns and children under-five, especially those in underserved areas.Health practitioners, male and female, trained to provide services encompassing prevention, diagnosis, management and treatment of leading maternal, neonatal and childhood diseases (e.g., Integrated Management of Pregnancy and Childbirth, neonatal resuscitation, Integrated Management of Childhood Illnesses). Skilled and knowledgeable health practitioners and volunteers, male and female, deployed to underserved areas to address the most common and severe diseases affecting mothers, newborns and children under-five. Community outreach, including vaccination campaigns and integrated health services, conducted.Operational:
WB SHARP
GPEI
HPIC CBAM Planned:
Regional MNCH centre
H4+ Action Plan Other Program Activities:
TBD
Enhanced healthy nutritional practices by mothers, newborns, and children under-five (by among other factors, addressing gender and socio-cultural determinants).Increased availability of nutrition-related goods and services that minimize risks and optimizes nutrition, health and survival outcomes, for men, women, and children, particularly pregnant and lactating women and children under-twoHealth practitioners, male and female, trained on nutrition issues and interventions. Dry or ready-to-use supplementary food rations and key vitamins and minerals (e.g., vitamin A, zinc, folic acid, iodized salt, multiple micronutrient powders) provided to pregnant women and children under-five. Skilled breastfeeding support provided to lactating women.Operational:
WB SHARP Planned:
Community-based nutrition initiative (TBD)
H4+ Action Plan Other Program Activities:
TBD
Increased awareness and knowledge among men, women and children of moderate/severe acute malnutrition and micronutrient deficiencies at the community level.Awareness and knowledge -raising information and education on nutrition, hygiene, water and sanitation provided to men, women, and children under-five. Nutritional surveillance system established at the local level.Operational:
WB SHARP Planned:
Community-based nutrition initiative (TBD)
H4+ Action Plan Other Program Activities:
TBD
Human Rights Increased participation and inclusion of women and their concerns in decision-making processes at the national and sub-national levels.Enhanced ability of Afghans, women in particular, to actively and effectively participate in community level governance and developmetTraining provided to community leaders -women in particular - in basic governance and decision-making processes (e.g. priority planning; consultations, negotiations and budget literacy).Operational: Planned:
Women's rights/decision-making project (TBD) Other Program Activities:
Increased awareness by Afghans -  women in particular about their rights, state commitments and rights promotion/ protection mechanismsTraining and awareness raising activities delivered to Afghans - women in particular - at the community level on human rights and human rights resources - women's and girls' in particularOperational:
AIHRC Planned:
Women's rights/decision-making project (TBD) Other Program Activities:
Enhanced effectiveness of human rights and civil society organizations to promote human rights, as well as investigate and act on human rights violations, especially for women and girls.Increased capacity of human rights institutions, including the AIHRC, and organizations to promote, protect and monitor human rights issues, including women's rights.Technical (individual skills development and/or strategic organisational direction) and financial support provided to human rights institutions, including Afghanistan's Independent Human Rights Commission (AIHRC).Operational:
Multi-year institutional support to Afghanistan's Independent Human Rights Commission (AIHRC) Planned: None Other Program Activities:
Chair AIHRC Donor Group
Participate in EU Human Rights Working Group (?)
Participation in Gender Donor Coordination Group
Coordinate with DFAIT on human rights (mutual information-sharing and leveraging investments so that GoC positions also reflect CIDA views/concerns)
Humanitarian Assistance Enhanced responsiveness of humanitarian assistance provided by international and Afghan actors to address the basic human needs of crisis-affected populations.Increased access by affected populations in targeted areas to appropriate humanitarian assistance, including protection, food, shelter, water, sanitation and health services for affected populations in targeted areas.Financial support provided to priority humanitarian actions.  Food assistance provided. Response to appeals coordinated and consulted with other government departments, donors and stakeholders.Operational (2011):
  • 2011 Appeals (ICRC, ARCS/IFRC, UNHCR)
  • WFP Protracted Relief & Recovery Operation 200063 (PRRO)
  • CARE Canada's Emergency Response & Capacity Building for Emergency Preparedness in Afghanistan for FY11/12
  • Mine Action Program for Afghanistan for FY11/12
Planned:
  • Assess the option of funding humanitarian appeals and operations in Afghanistan, including food and non-food assistance, for 2012, 2013, 2014
  • Assess the option of funding natural disaster response operations in Afghanistan for 2012, 2013, 2014
  • Assess the option of funding  Mine Action programming in Afghanistan for 2012, 2013, 2014
Other Planned Program Activities:
  • Continue to chair and/ or participate in the Donor Coordination Group
  • Participate in humanitarian cluster meetings that remain open to donors - potentially the following: (i) Emergency Shelter; (ii) Food Security & Agriculture; (iii) Nutrition; (iv) Emergency Telecommunications; (v) Water/Sanitation (WASH); (vi) inter-cluster meetings during disasters
  • Coordinate and consult with IHA/MULTI, DFAIT, and Canadian missions abroad (for e.g. Geneva)
Improved resilience of crisis-affected vulnerable populations, with a focus on women and girls.Increased ability of implementing partners to apply durable and gender-sensitive approaches in humanitarian responses.Food assistance provided for the re-establishment of livelihoods. Cash grants provided for the re-integration of returnees. Stakeholders engaged in policy dialogue on the issue of gender equality and early recovery/humanitarian assistance.Operational (2011):
  • 2011 Appeals (ICRC, ARCS/IFRC, UNHCR)
  • WFP Protracted Relief & Recovery Operation 200063 (PRRO)
  •  CARE Canada's Emergency Response & Capacity Building for Emergency Preparedness in Afghanistan for FY11/12
  • Mine Action Program for Afghanistan for FY11/12
Planned:
  • Assess the option of funding humanitarian appeals and operations in Afghanistan, including food and non-food assistance, for 2012, 2013, 2014
  • Assess the option of funding natural disaster response operations in Afghanistan for 2012, 2013, 2014
  • Assess the option of funding  Mine Action programming in Afghanistan for 2012, 2013, 2014
Other Planned Program Activities:
  • Continue to chair and/ or participate in the Donor Coordination Group
  • Participate in humanitarian cluster meetings that remain open to donors - potentially the following: (i) Emergency Shelter; (ii) Food Security & Agriculture; (iii) Nutrition; (iv) Emergency Telecommunications; (v) Water/Sanitation (WASH); (vi) inter-cluster meetings during disasters
  • Coordinate and consult with IHA/MULTI, DFAIT, and Canadian missions abroad (for e.g. Geneva)
  • Support at least one study/assessment/evaluation integrating and/or focussing on gender equality and early recovery/humanitarian assistance
Enhanced performance of national and sub-national actors in disaster risk reduction and emergency preparedness.Improved ability of Afghan authorities to mitigate the effects of natural and man-made disasters on the Afghan population, including most vulnerable groups such as women and girls.Preparedness activities undertaken to address the occurrence of natural and man-made disasters. Information regarding disaster risk reduction and emergency preparedness/response disseminated to stakeholders through studies/assessments/evaluations, including information on gender equality.Operational (2011):
  • 2011 Appeals (ICRC, ARCS/IFRC, UNHCR)
  • WFP Protracted Relief & Recovery Operation 200063 (PRRO)
  • CARE Canada's Emergency Response & Capacity Building for Emergency Preparedness in Afghanistan for FY11/12
  • Mine Action Program for Afghanistan for FY11/12
Planned:
  • Assess the option of funding humanitarian appeals and operations in Afghanistan, including food and non-food assistance, for 2012, 2013, 2014
  • Assess the option of funding natural disaster response operations in Afghanistan for 2012, 2013, 2014
  • Assess the option of funding  Mine Action programming in Afghanistan for 2012, 2013, 2014
Other Planned Program Activities:
  • Continue to chair and/ or participate in the Donor Coordination Group
  • Participate in humanitarian cluster meetings that remain open to donors - potentially the following: (i) Emergency Shelter; (ii) Food Security & Agriculture; (iii) Nutrition; (iv) Emergency Telecommunications; (v) Water/Sanitation (WASH); (vi) inter-cluster meetings during disasters
  • Coordinate and consult with IHA/MULTI, DFAIT, and Canadian missions abroad (for e.g. Geneva) on humanitarian issues related to Afghanistan and the region
  • Support at least one study/assessment/evaluation integrating and/or focussing on disaster risk reduction and emergency preparedness/response, and that includes the issue of gender equality

Appendix 6 - Methodological approach

The methodological approach for this evaluation was elaborated in the "Final Evaluation Design and Work Plan, October 15th 2013" for this evaluation in line with the requirements of the ToR (see Appendix 1). The evaluation design is an internal working document and some elements of the design are presented in Chapter 1 and in this appendix. In addition, an appendix to the evaluation design was developed on May 7th 2014, presenting the "Approach regarding integration of findings on the Dahla Dam or Arghandab Irrigation Rehabilitation Program (AIRP)".

This appendix complements the main information provided in chapter 1, which means that information that is extensively dealt with such as evaluation purpose, data collection and analysis methods, and risk assessment and mitigation is not repeated in this appendix.

A6.1 Guiding principles

Against the background of the evolving context in Afghanistan and the guidance on evaluations in fragile states the following specific guiding principles for this evaluation have been identified:

A6.2 Consolidated Intervention Logic

Figure A6.2 Text Alternative

A6.2 Consolidated Intervention Logic

Education

Inputs/ activities:

  • Build/rehabilitate schools
  • Provide teacher training
  • Set-up community schools
  • Literacy, vocational training

Outputs:

  • Schools built/improved
  • Teachers trained
  • Community-based schools
  • People trained

Outcomes:

  • Improved access esp. for girls
  • Improved learning
  • Improved quality of education
  • Enhanced capacity

Impact: A more secure and democratic Afghanistan, able to deliver services to Afghans

Health

Inputs/ activities:

  • Immunization campaigns
  • Training of community health workers, midwives and others
  • Support to essential health services
  • Provision of BPHS

Outputs:

  • Children immunized
  • Trained health workers
  • Better performing health facilities
  • Improved MMIS indicators

Outcomes:

  • Improved access to health services
  • Improved quality of health services
  • Progress towards polio eradication
  • Enhanced capacity

Impact: A more secure and democratic Afghanistan, able to deliver services to Afghans

Economic Growth

Inputs/ activities:

  • Provision of community-based infrastructure
  • Provision of microfinance
  • Support to SMEs
  • Training of craftsmen

Outputs:

  • Completed infrastructure
  • CDC's established
  • Loans provided
  • SMEs supported
  • People trained

Outcomes:

  • Improved access to employment and income
  • Improved access to microfinance
  • Enhanced capacity

Impact: A more secure and democratic Afghanistan, able to deliver services to Afghans

Humanitarian Assistance

Inputs/ activities:

  • Provision of food assistance
  • Provision of shelter, etc.
  • Training on disaster management
  • Demining

Outputs:

  • People reached
  • Vulnerable households reached
  • People trained
  • Demined area

Outcomes:

  • Improved food and nutritional status
  • Increased resilience
  • Enhanced capacity

Impact: A more secure and democratic Afghanistan, able to deliver services to Afghans

Human rights

Inputs/ activities:

  • Support to AIHRC
  • Support to civic education
  • Support to women's organizations

Outputs:

  • Trained staff
  • People trained
  • Grants provided

Outcomes:

  • More human rights activities
  • Enhanced capacity
  • Increased awareness of human rights
  • Increased access of women's decision making

Impact: A more secure and democratic Afghanistan, able to deliver services to Afghans

A 6.3 Detailed sampling approach

In order to arrive at a meaningful sample to study the portfolio in more detail the evaluation team deployed the following methodology:

The sampling resulted in a selection of 50 initiatives, covering a total disbursement of $852 million, approximately 55% of overall disbursements during the period 2004-05 to 2012-13 ($1.546 million). All selected projects have a 100% Afghanistan focus. The three signature projects, that accompanied Canada's military engagement in Kandahar, are included in the sample. Table A6.1 represents the sample by sector of focus.

Table A6.1 Evaluation sampleFootnote 86
Sector of FocusInitiativesSample
Budget
Sample
Proportion
Portfolio
Budget
Sample/
Portfolio
Strengthening basic education6133.951.83716%153.524.54487%
Improving health8127.342.61915%167.126.86176%
Democratic governance11209.080.11825%333.042.42163%
Private sector development9163.966.05119%333.366.72649%
Emergency assistance11101.066.52712%184.130.02955%
Peace and Security499.895.13812%121.094.08282%
Other (KLIP)116.952.1182%224.893.1078%
Totals50852.254.408100%1.546.257.49755%

The sample is considered to be representative of the entire portfolio, although slight differences between the evaluation sample and the total portfolio could not be avoided. For example, education, health and peace and security are slightly overrepresented and this is also the case for the gender equality focus.Footnote 87 This is the case in those sectors where the Program was more active in recent years, which also serves to enhance learning opportunities. On the other hand, the category 'other/multisector' is underrepresented.Footnote 88

The sector Democratic Governance was the subject of further sampling, given the wide range of sub-sectors under this sector, which does not always correspond with the country strategies. The sub-sector Human Rights with a strong gender equality focus has been selected as the sub-sector on which the evaluation will focus as it is expected that for this sub-sector most relevant findings can be found, which will provide a good opportunity to draw lessons. The gender equality focus is another reason to concentrate on this sub-sector. The sub-sector election support took place in 2004-05 and in 2009, and these elections have been thoroughly evaluated and therefore it was not expected that this evaluation could add any value in terms of new insights. Furthermore, two main projects related to Democratic participation and civil society, and specifically focussing on community development were selected. These projects have focused on the delivery of rural infrastructure to community-based organisations and in the intervention logics for the Afghanistan Programs these projects were not classified in the governance sector, but rather as part of the sector "Economic Growth".

A specific problem that the evaluation team faced was that that the sector classification used in the portfolio does not correspond entirely with the sector priorities for the Afghanistan Program as they were defined over the course of the evaluation period. Therefore, consistent sector labels have been used throughout the evaluation: democratic governance with human rights as sub-sector; economic growth (including private sector development); education, health, and humanitarian assistance (combination of emergency assistance and peace and security).

The table below outlines the specific projects that have been investigated under each sector heading. A project can consist of various initiatives as initiatives concerning the same projects or type of activities have been combined. Therefore, the 50 initiatives result in 25 projects.

Table A.6.2 Sample by name of project
Sector of FocusName of ProjectNo. of
Initiatives
Executing AgencyPeriodTotal Disbur
sem.
Gender
focus
Kandahar focus
EducationEducation Quality Improvement Program (EQUIP)1World Bank – ARTF2007
ongoing
91.50Gender equality integratedNation-wide, and incl. Kandahar
Basic Education and Gender Equality (BEGE), Kandahar2UNICEF2008
ongoing
11.26No gender equality resultsFootnote 89Specific for Kandahar
Girls Education Support Program (GESP)Footnote 902BRAC/AKF2006-201326.25Gender equality integratedSelected provinces incl. Kandahar
Basic Education for Afghanistan Consortium (BEACON)1International Rescue Committee2011
ongoing
4.94Gender equality integratedSelected provinces incl. Kandahar
HealthGlobal Polio Eradication Initiative – WHO4WHO2006
ongoing
74.00Gender equality integratedNation-wide, and incl. Kandahar
Global Polio Eradication Initiative – UNICEF2UNICEF2009
ongoing
28.00Gender equality integratedNation-wide, and incl. Kandahar
Strengthening Health Activities for the Poor (SHARP)1World Bank - ARTF2010 – 201215.00Limited gender equality resultsNation-wide, and incl. Kandahar
Regional Maternal New-born and Child Health (MNCH) Program1Aga Khan Foundation2011 ongoing9.71Gender equality integratedNo Kandahar focus
Democratic governance-
human rights
Support to the Afghanistan Independent Human Rights Commission2AIHRC2007 – 201314.41Gender equality integratedSelected provinces incl. Kandahar
Responsive Fund for the Advancement of Women1PSU – Asia2007 – 20123.31Gender equality specificNo Kandahar focus
Women as Decision Makers1UN-Women2009 – 20112.00Gender equality specificNo Kandahar focus
Women Political Provincial Participation1National Democratic Institute2009 – 20100.76Gender equality specificNo Kandahar focus
Economic GrowthDahla Dam Arghandab Irrigation Rehabilitation3SNC Lavalin International/
Central Asia Development Group
2008 – 201247.07Footnote 91No gender equality resultsSpecific for Kandahar
Microfinance Investment Support Facility for Afghanistan (MISFA)3World Bank - ARTF2004 – 201094.50Gender equality integratedNation-wide, and incl. Kandahar
Rural Enterprise Development in Kandahar (REDKan)1UNDP2009 – 20119.40No gender equality resultsSpecific for Kandahar
Turquoise Mountain1Turquoise Mountain Trust2008- 20127.70No gender equality resultsNo Kandahar focus
Kandahar Rapid Village Development Plan1Development Works Inc.2007- 20105.26Gender equality integratedSpecific for Kandahar
Humanitarian AssistanceWFP Appeals6World Food Programvarious years84.52No gender equality resultsNation-wide, and incl. Kandahar
Including Peace and SecurityMine Action Program Afghanistan4UNMAS2004-201299.90No gender equality resultsNation-wide, and incl. Kandahar
Kabul Widows3CARE Canada2004 – 20119.42Gender equality specificNo Kandahar focus
Refugee Return and Reintegration1UNHCR2004 – 20055.00No gender equality resultsNation-wide, and incl. Kandahar
Emergency Preparedness and Response Project1CARE Canada2012 – 20132.13Limited gender equality resultsNo Kandahar focus
OtherKandahar Local Initiatives Fund (KLIP)1Multiple implem. Partners2007 -201217.00No gender equality resultsSpecific for Kandahar
community developmentNational Solidarity Program (NSP)3World Bank - ARTF2004 – 2010147.50Gender equality integratedNation-wide, and incl. Kandahar
National Area Based Development Program (NABDP)3UNDP2004 – 201141.10Gender equality integratedNation-wide, and incl. Kandahar

Smaller projects are included under two fund management programs, namely the Kandahar Local Initiative Program (KLIP), managed through the KPRT and the Responsive Fund for the Advancement of Women managed by the CPSU.

A6.3 Approach to data collection and methodology

In the ToR (see Appendix 1), preliminary evaluation questions were formulated and grouped under six evaluation criteria and specific topics. Upon discussion, the ToR evaluation questions were regrouped in line with the four main evaluation criteria and four lead questions were defined that were presented in the Introduction of this synthesis report. A total of 39 sub-questions were defined in relation to all lead questions.

In order to answer the evaluation questions at the three levels (Program, sector and project/intervention), a 'top-down and bottom-up' approach was carried out by linking downstream, project-level efforts with upstream sector policy dialogue and by linking the international conferences on Afghanistan since 2001 and the decision-making by the Government of Canada on Afghanistan to strategic development programming and implementation. A program level evaluation matrix was elaborated, specifying for each sub-question, indicators and sources and methods for data collection (see Box A6.3).

Box A.6.3 Program level Evaluation Matrix (as presented in final Evaluation Design, 15 October 2013)

Relevance

Alignment with Afghan government priorities

1.1 To what extent is the Afghanistan Program aligned with the strategic priorities of the Government of Afghanistan?

Sources and Methods:

Taking into account needs and conflict dynamics

1.2 To what extent are the main needs of the people of Afghanistan (in particular women and children) addressed through the CIDA Afghanistan Program?

Document review:

1.3 Have the Afghanistan conflict dynamics as well as the priorities and rights of affected populations been considered in the Afghanistan Program, including in its international humanitarian assistance programming?

Document review:

Risk analysis and risk mitigation

1.4 Were risks adequately assessed and risk mitigation strategies formulated? In particular, were program responses appropriately designed to mitigate or prevent further conflict?

Document review:

1.5 To what extent did CIDA have an adequate contextual understanding of Afghan society and the programming context to design and implement relevant and effective programming? How were choices on aid channels (executing agencies) and aid modalities made based on contextual understanding?

Sources and Methods:

1.6 How appropriate and realistic were the (evolving and changing) overall Program objectives and/or the envisaged results/targets of specific projects?

Sources and Methods:

Alignment with WoG approach

1.7 Have the Afghanistan Program interventions been aligned with federal government priorities, CIDA's strategic outcomes and Canada's Whole of Government approach?

Sources and Methods:

1.8 What were the main implications of the Manley report for CIDA's programming and implementation of the Afghanistan Program? How did the increased coordination of Government of Canada efforts affect CIDA's performance?

Sources and Methods:

1.9 What was the CIDA strategy regarding activities in Kandahar before and after the shift of policy related to the Manley report in 2008? What were the implications of this concentration on Kandahar for CIDA programming and results?

Sources and Methods:

1.10 What were the implications for CIDA programming and results when Canada withdrew from Kandahar in 2010-2011?

Sources and Methods:

1.11 To what extent are CIDA's overall priorities and strategies reflected in the Afghanistan Program?

Sources and Methods:

Cross-cutting issues: gender equality and environment

1.12 Did the Afghanistan Program adequately prepare for and consider gender equality (a CIDA cross-cutting theme)?

Sources and Methods:

1.13 Did the Afghanistan Program adequately prepare for and consider environmental sustainability issues (a CIDA cross-cutting theme)?

Sources and Methods:

Canada's role in donor coordination

1.14 Has the Afghanistan Program coordinated its programming efforts with those of other international donors, including through policy dialogues and joint interventions and were the CIDA intervention complementary to the interventions of other donors?

Sources and Methods:

1.15 To what extent did the CIDA Afghanistan Program take the prevailing international principles on ownership, alignment and harmonization and on engagement in fragile States and Situations into account in its programming and implementation?

Sources and Methods:

1.16 To what extent were the principles for Good Humanitarian Donorship taken into account in the Afghanistan Program?

Sources and Methods:

1.17 What role did CIDA play to improve aid coordination and how is this perceived by other actors?

Documents on donor coordination in Afghanistan: international conferences, aid architecture, minutes of meetings, studies and reviews;
Interviews.

Effectiveness

2.1 How result-oriented was the CIDA support to Afghanistan? Was a clear distinction made between outputs, outcome and impacts in programming, implementation and finalization?

Sources and Methods:

2.2 Were the planned outputs delivered?

See sector evaluation matrices.

Sector evaluation matrices with output and outcome indicators have been developed that are the basis for the analysis of effectiveness at selected project and sector level;
Project assessment forms and related guidelines for sector experts.

2.3 To what extent were expected outcomes achieved?

See sector evaluation matrices.

Sector evaluation matrices with output and outcome indicators have been developed that are the basis for the analysis of effectiveness at selected project and sector level;
Project assessment forms and related guidelines for sector experts.

2.4 If expected outcomes were not fully achieved, what were the barriers preventing success? And were risks anticipated and mitigated against?

2.5 Was the Afghanistan Program effective in addressing the cross-cutting theme of gender equality?

Sources and Methods:

Sector evaluation matrices with output and outcome indicators have been developed that are the basis for the analysis of effectiveness at selected project and sector level. Gender equality is integrated in all sector evaluation matrices. Project assessment forms and related guidelines for sector experts;
Document review and interviews at Program level and in Afghanistan.

2.6 Was the Economic Growth sector effective in addressing environment as a cross-cutting theme?

See sector evaluation matrices.

Sector evaluation matrices with output and outcome indicators have been developed that are the basis for the analysis of effectiveness at selected project and sector level. Environmental effects will be assessed for the "Economic growth" sector. Project assessment forms and related guidelines for sector expert Economic growth.

2.7 Were synergies realized within the sectors and between the various sectors? Did the WoG approach contribute to synergies?

Evidence of activities that mutually reinforced each other to achieve the same goal;
Indications of planning of synergies within and across sectors.

Sources and Methods:

2.8 What specific internal and external factors have contributed to results? Did CIDA leadership in specific sectors or sub-sectors contribute to the results achieved? What about on-going security?

Sources and Methods:

Efficiency

3.1 What measures were taken to deliver the aid in an efficient way at Program level?

Sources and Methods:

3.2 Have adequate risk mitigation strategies been put in place to reduce various types of risks, including the risk of corruption?

Sources and Methods:

3.3 Does CIDA have the right tools, processes and procedures to support effective development programming and implementation in fragile states and situations?

Sources and Methods:

3.4 To what extent were the Afghanistan Program delivery mechanisms conducive to better programming interventions?

Statistics on delivery mechanisms.

Sources and Methods:

3.5 Was the delivery of CIDA support timely? Were there any delays?

Planning of project duration vs. actual implementation period.

Sources and Methods:

3.6 What measures were taken to deliver the projects in an efficient way?

Composition of project budgets:

Project assessment forms and related guidelines for sector experts.

3.7 Were appropriate M&E mechanisms set up and were lessons from M&E taken into account in programming and implementation

Sources and Methods:

3.7 Did CIDA learn lessons from its involvement in Afghanistan? What were the mechanisms?

Indications of lessons-learned.

Sources and Methods:

Sustainability and impact

4.1 What are the positive and negative, primary and secondary long-term effects of the Afghanistan Program?

Listing of positive and negative long-term effects including recognition by the population that the Afghan government is providing better services, improved policies and strategies (including GE) of the government at national and sub-national level that are being implemented, improved stability, improved role of women in decision-making at various levels, increased use of resources and services for the poor and in particular women.

Sources and Methods:

Project assessment forms and related guidelines for sector experts.

4.2 What is the probability of continued long-term benefits of CIDA supported interventions?

Sources and Methods:

Project assessment forms and related guidelines for sector experts.

4.3 Were plans established and implemented in a manner to allow and encourage local "ownership"? And if so, were the plans successful?

Evidence of local ownership of CIDA supported interventions during formulation, implementation and after finalization of the support.

Project assessment forms and related guidelines to sector experts.

4.4 Has the Government of Afghanistan created a policy environment that is conducive to sustaining the accomplished results?

Sources and Methods:

4.5 Have sources of funding been identified to sustain development results brought about by CIDA's programming in Afghanistan?

Availability of new sources of funding specified per type of source.

Project assessment forms and related guidelines to sector experts.

4.6 What are the implications of CIDA's strategy related to the Transition in Afghanistan in relation to the sustainability of results?

Sources and Methods:

For each of the five sectors of focus - education, health, economic growth, humanitarian assistance and human rights - an evaluation matrix was developed during the design phase specifying indicators, sources and methods for sector-specific outputs and outcomes (see Boxes A6.4 to A6.8).

Box A6.4 Evaluation Matrix Health

Outputs

Objective: Children immunized against polio

Indicators: Proportion of children who received three doses of polio vaccine by the age of 12 months by gender and Province*

Sources and methods: HMIS data, WHO and UNICEF data, MICS, Post-campaign assessment surveys, other surveys, key informant interviews

Objective: Children immunized against vaccine-preventable diseases

Indicators: Proportion of children who received one dose of measles vaccine by the age of 12 months by gender and Province

Sources and methods: HMIS data, WHO and UNICEF data, MICS, Post-campaign assessment surveys, other surveys, key informant interviews

Objective: Community Health Workers trained and deployed

Indicators: Number of CHWs active / number of CHWs trained by gender and Province

Sources and methods: HMIS data and reports, SHARP project reports, NGO reports, key informant interviews

Objective: Immunization Volunteers trained

Indicators: Number of immunization volunteers and volunteer coordinators trained by gender and Province

Sources and methods: HMIS data and reports, polio campaign reports, key informant interviews

Objective: Midwives and Community Midwives trained and deployed in Badakshan and Bamyan

Indicators: Number of Midwives and Community Midwives active / number of midwives trained in Badakshan and Bamyan Provinces

Sources and methods: HMIS data and reports, AKF project reports, USAID/JHPIEGO reports, key informant interviews

Objective: Essential health structures supported.

Indicators: Number of health facilities submitting timely HMIS reports by Province and by type of facility

Sources and methods: HMIS data and reports, NGO reports and evaluations, key informant interviews

Objective: Basic package of health services provided

Indicators:

Sources and methods: HMIS data and reports, NGO reports and evaluations, key informant interviews

Outcome

Objective: Improved access to essential quality health services

Indicators: Proportion of population residing within two hours walking distance from primary care services by Province and rural/urban distribution

Sources and methods: HMIS data and reports, population surveys (e.g. MICS), key informant interviews

Objective: Increased quality of essential health services provided to vulnerable populations, esp. women and children.

Indicators: Proportion of births delivered by skilled personnel by Province and by type of provider

Sources and methods: HMIS data and reports, population surveys (e.g. MICS), SHARP/NGO reports, key informant interviews

Objective: Increased use of health services, esp. by women and children

Indicators:

Sources and methods: HMIS data and reports, population surveys (e.g. MICS), key informant interviews

Objective: Progress towards polio eradication

Indicators:

Sources and methods: HMIS data, Disease Early Warning System Weekly and Annual Reports, key informant interviews

Objective: Improved child health

Indicators:

Sources and methods: HMIS data and reports, population surveys (e.g. MICS)

Objective: Enhanced capacity of the Ministry of Public Health to provide leadership and coordination of the health sector at national and provincial level, especially for maternal, new-born and child health.

Indicators: Qualitative assessment of reporting, monitoring and supervision systems, and of the availability of policies, standard operating procedures and guidelines

Sources and methods: Evaluation reports, organizational reviews and assessments, review of supervision plans and reports, review of health information system, key informant interviews

* Throughout the table, indicators to be evaluated "by province" refers to focal provinces of the CIDA/DFATD program (number of provinces to be determined).

Box A6.5 Evaluation Matrix Education

Outputs

Objectives: Provide teacher training.

Indicators: Number of teachers trained per year (gender-disaggregated).

Sources and methods:

Objectives: Building or rehabilitating education infrastructure.

Indicators: Number of schools built, expanded or repaired (per category) in Kandahar.

Sources and methods:

Objectives: Set-up of community based schools.

Indicators: Number of community based schools established

Sources and methods:

Objectives: Literacy, vocational and skills development training provided.

Indicators: Number of individuals who received literacy training (gender- disaggregated)

Sources and methods:

Objectives: Local community capacity development supported.

Indicators: Number of school management committees established

Sources and methods:

Outcome

Objectives: Improved access to quality education services for women and girls (Kandahar focus).

Indicators: Student enrolment rate in education at national and provincial levels by gender

Sources and methods: EMIS and other data sources; verify at school level and with KIIs

Indicators: Number of community-based education students who transition to the formal education system

Sources and methods: Program reporting data

Indicators: Net enrolment ratio of children in primary education, nationally and in Kandahar, disaggregated by gender

Sources and methods: EMIS and other national data sources

Indicators: Number of students enrolled in Technical/Vocational Education Training (TVET)

Sources and methods: EMIS and other national data sources

Objectives: Improved learning of children in schools

Indicators:

Sources and methods:

Objectives: Improved quality of basic education.

Indicators: Improved curricula and teaching methods;

Sources and methods: Program reporting data

Indicators: Number and percentage of pupils (gender-disaggregated) finishing basic education.

Sources and methods: EMIS and other national data sources

Objectives: Improved capacity of national, provincial and local institutions to deliver government authorities to deliver quality education

Indicators:

Sources and methods:

Box A6.6 Evaluation Matrix Economic Growth Footnote 92

Outputs

Objectives: Community-based infrastructure and livelihoods projects delivered (mainly Kandahar focus)

Indicators:

Sources and methods: NSP and NABDP data, other surveys, key informant interviews,

Objectives: Microfinance services provided

Indicators:

Sources and methods: MISFA data, WB reports and surveys, key informant interviews

Objectives: Small and Medium Enterprises provided with increased access to credit and improved technical knowledge and skills

Indicators:

Sources and methods: REDKan data and reports, key informant interviews

Objectives: Traditional craftsmen and artists trained through the Institute for Afghan Arts and Architecture and gainfully employed

Indicators:

Sources and methods: Turquoise Mountain data and reports, key informant interviews, focus group with graduates

Outcome

Objectives: Improved access to employment and income opportunities for women and men, with a particular focus on Kandahar

Indicators:

Sources and methods: Household and livelihood surveys, CIDA end of project reports or final evaluation reports, key informant interviews

Objectives: Improved access to financial and business services for SMEs and entrepreneurs, run by Afghans for Afghans.

Indicators:

Sources and methods: Household and livelihood surveys, MISFA data, World Bank data, key informant interviews

Objectives: Enhanced capacity of communities to identify and implement community development projects (Kandahar focus)

Indicators:

Sources and methods: NABDP and NSP data and reports, household and livelihood surveys, key informant interviews

Box A6.7 Evaluation Matrix Humanitarian AssistanceFootnote 93

Outputs

Objectives: Food assistance provided to vulnerable populations.

Indicators:

Sources and methods:

Objectives: Shelter, winterization packages and non-food items (NFI) provided to vulnerable households.

Indicators: Number of vulnerable households to which shelter, winterization packages and NFI was provided (gender disaggregated)

Sources and methods:

Objectives: Improved capacity of public institutions and communities to plan and coordinate emergency assistance.

Indicators:

Sources and methods:

Objectives: Rehabilitation of sustainable livelihoods for affected communities and returnees

Indicators:

Sources and methods:

Objectives: Mine clearance of land of return for Afghan communities

Indicators:

Sources and methods:

Outcome

Objectives: Improved food and nutritional status of women, children and vulnerable livelihoods

Indicators: Number of persons, communities and regions who have seen an improvement of food consumption and nutritional status, particularly of children, women and vulnerable individuals

Sources and methods: Review of WFP, UNICEF, OCHA and CIDA Reports,
Interviews with WFP officials and CIDA Humanitarian Team in Kabul

Objectives: Improved preparedness of national authorities to natural and conflict driven disasters

Indicators:

Sources and methods:

Objectives: Improved capacity of farmers and communities to have access to their land

Indicators: Number of farmers able to farm their land

Sources and methods:

Box A6.8 Evaluation Matrix Human RightsFootnote 94

Outputs

Objectives: Support to Afghanistan Independent Human Rights Commission provided.

Indicators:

Sources and methods:

Objectives: Candidate, voter and civic education and outreach activities for women and men conductedFootnote 95.

Indicators:

Sources and methods:

Objectives: Women's organizations supported.

Indicators: Number of women's organizations which received grants per type of organizations and type of funded activities under the RFAW

Sources and methods:

Outcome

Objectives: Enhanced capability of the AIHRC to exert leadership and to perform promotion, monitoring and protection of human rights

Indicators:

Sources and methods:

Objectives: Increased awareness of human rights by right-holders and duty-bearers.

Indicators:

Sources and methods:

Objectives: Improved protection of human rights for women and men

Indicators:

Sources and methods:

Objectives: Increased participation of women and men in political and electoral processes at national, provincial and local levels.Footnote 96

Indicators:

Sources and methods:

Objectives: Strengthened access of women to decision-making.

Indicators: Increase in the number of women holding positions:

Quality and importance of roles performed by women in decision-making positions

Sources and methods:

Objectives: Women's CSO strengthened.

Indicators: Increased ability of CSO to conduct activities for the advancement of women (increase of the number of beneficiaries, geographical areas covered, themes and issues addressed).

Sources and methods:

In addition to the sector analyses two cross-cutting analyses were carried out – one on gender equality and another on the province of Kandahar where Canada implemented a Whole of Government approach from 2008-2011.

Evaluation principles that were applied included time sensitivity – taking into account that policies, yardsticks and goal posts changed during the period covered by the evaluation - and conflict sensitivity - designing and implementing the evaluation in such way to guarantee the safety of informants, be aware of the effects of the evaluation process, understand the conflict and pay due attention to the safety of evaluators.

The evaluation team applied a multi-method approach to gather and analyze both qualitative and quantitative information to answer the evaluation questions. The specific context in Afghanistan and the time and resources available for this evaluation meant that no surveys could be carried out. As a result, the evaluation had to rely, to a large extent, on secondary data. The main data collection methods included:

Throughout the evaluation, all information was collected and recorded in line with the evaluation matrices. This means that the questionnaires were structured in line with the evaluation questions taking into account the specificity of the sector and the background of the interviewee. Given the relatively large number of evaluation questions and indicators, choices were made prior to the interviews on the most relevant questions to be asked. In this process, gaps in information could be identified and additional information was collected to fill the gaps.

At the project level, a project assessment form was elaborated for each selected project, including findings from the desk review and data collection in the field. The project assessment forms were also based on the evaluation questions and related evaluation matrices.

Sector reports were elaborated for the five main sectors, while also two additional reports were made on gender equality as a cross-cutting theme and on Kandahar. The five draft sector reports included the project assessment forms for all selected projects. In the finalization process, the sector reports have been turned into appendices and the project assessment forms are considered as internal working documents.

In the design and implementation of the evaluation, due attention was paid to contribution analysis as Canada was in most cases not the single donor of projects and programs, but contributed to multi-donor programs. Contribution analysis, as defined by John Mayne in 'Contribution analysis: Coming of age?, Evaluation, 2012 18: 270, relies on a systematic analysis as a way of making credible claims about results achieved. This type of analysis requires a developed theory of change where assumptions and external factors are tested to confirm the results logic and this approach was followed in this evaluation. For each project, the evaluators aimed to analyse the contribution of Canada to the project both in monetary and non-monetary terms. In general, it is assumed that Canada contributed proportionally to the results in line with its monetary contribution. This assumption could be validated in most cases. In some cases, where Canada played a very active role in the policy dialogue and donor coordination, Canada's impact was greater than what could be expected on the basis of its monetary contribution. However, in other cases, Canada engaged less in policy dialogue and its contribution may have been smaller than what could be expected on the basis of its monetary contribution.

Once the data collection was completed and the data were coded in line with the evaluation matrices, all data was triangulated to the extent possible. When evidence was insufficient, no firm conclusions were formulated. Throughout the data analysis and quality assurance process, due attention was paid to ensure that conclusions were based on sound and robust evidence, lessons were in line with the conclusions and recommendations were linked to the conclusions and lessons.

The sector reports served to inform the evaluation at program level. In addition, specific program level information was collected and analyzed, in particular related to the Whole of Government approach. In order to improve the readability of the final report, including the sector reports presented in Volume 2 of the Technical Report, the evaluation questions by evaluation criterion were clustered under the specific headings that were used as sub-titles in this report and the sector reports (See Table A2.2).

Data were collected and coded in line with the evaluation matrices at program and sector levels. In the data analysis process, data were triangulated in order to arrive at sound findings based on solid evidence.

A6.4 Methodological challenges and limitations

Box A6.9 describes in more detail some of the challenges faced during this evaluation and the measures taken to mitigate their negative impact on the evaluation process and outcome.

Box A6.9 Methodological challenges and mitigation measures

Risk/ challenge: Missing documents, incomplete files

Mitigation measures: The evaluation team received approximately 2,000 documents from the Afghanistan Program throughout the evaluation process. Many versions of the same document were received, often without dates, making it challenging to determine the sequence of changes. The project portfolio information was complete from the beginning of the evaluation, although not all coding was done in a way that enabled clear analysis and, therefore, some recoding was required (see Appendix 5). Given the high profile nature of this Program, there was a certain amount of classified information that the evaluation team only gained access to once it had been declassified.  This may have affected the quality of information as the declassification process often took time. In one case, some classified information became available to the evaluation team after an 'access to information' request. The Program did its best to provide access to documents and people, but faced limitations, which resulted in some delays in the transmission of documents to the evaluation team and hindered the efficiency of the process.  All crucial documents appear to have been shared.

Risk/ challenge: Rapid turnover of staff and therefore lack of institutional memory

Mitigation measures: A large number of staff was involved in the Afghanistan Program throughout the evaluation period. Turnover of Canadian staff based in Kabul and Kandahar was relatively high, which was the case for most countries and not specific to Canada. With some delay, the evaluation team was able to collect names and contact details of relevant staff (xCIDA, xDFAIT, Canadian Forces, Department of National Defense), and a large number of interviews were scheduled (see Appendix 4). The fact that CIDA provided access to various staff involved at different times and in more than one role during the evaluation period contributed positively to institutional memory and enabled learning.

Risk/ challenge: Resistance to evaluation

Mitigation measures: The team met only limited resistance to the evaluation. The Program, the main stakeholder in the evaluation, facilitated the evaluation to the extent possible. It is clear that they sought to learn lessons, and draw meaningful conclusions that would benefit their work. Most stakeholders were willing to cooperate in the evaluation, as reflected in the extensive list of interviewees (Appendix 4). Some resistance from certain stakeholders related to time constraints, which were overcome by reaching clear agreements on timing. Several interviewees expressed their satisfaction with the opportunity to reflect upon their work during the interview and to take some distance from their hectic daily operations. However, other interviewees expressed some frustration with the fact that they had been interviewed before for 'lessons learnt' exercises, but no lessons learnt were ever presented to them as these exercises were internal.

Risk/ challenge: "Fishbowl effect"Footnote 97

Mitigation measures: The team also encountered some cases of the so-called, "fishbowl effect", where there was a clear desire to stress the positive results. The team has mitigated this risk through the use of open and transparent communication methods and through the triangulation of evidence from multiple sources, including a large number of interlocutors and field visits. The evaluation team prepared detailed response sheets providing replies to each comment on draft sector reports and the draft technical report, indicating whether and how comments were addressed.

Risk/ challenge: Data availability

Mitigation measures: The evaluation team was very well aware of data availability challenges in Afghanistan, where the pressure to show short-term results and ensure visibility hindered the establishment of useful baselines and, ultimately, impeded the measurement of long-term effects. This problem was recognized and a certain number of new surveys and evaluation reports became available during the evaluation period and were used as sources of information. Nevertheless, there was an important absence of reliable information – particularly for certain provinces - that could not be entirely overcome.

Risk/ challenge: Problematic security situation in Afghanistan

Mitigation measures: The consultants were aware that they needed to plan with security in mind and ensure that alternative plans were ready in the event that there was a need to mitigate identified or emerging security risks. The consultants liaised closely with the Canadian Program Support Unit (CPSU) in the planning of field work in Afghanistan - in particular, the visits to project sites outside of Kabul: Kandahar, Herat and Jalababad. Only one planned visit to the province of Bamyan was cancelled due to weather conditions.

Risk/ challenge: Separate Dahla Dam / Arghandab Irrigation Rehabilitation (AIRP)  project evaluation

Mitigation measures: Since CIDA had commissioned an independent review of the Arghandab Irrigation Rehabilitation signature project (AIRP), it was agreed that this summative Program evaluation would rely on the findings from that project evaluation in order to avoid duplication. Findings, conclusions and lessons learned from the project evaluation were to be incorporated into this summative evaluation. It was agreed that evaluation methodologies and matrices used by both evaluations would be consistent and aligned. Despite substantial effort, this alignment was not realized. The project evaluation went ahead with very serious delays and the risks that were presented could not be sufficiently mitigated. Therefore, an alternative approach was developed to integrate findings on this important signature project in this Program evaluation on the basis of additional desk research and some data collection done by the program evaluation team. An appendix to the evaluation design was developed to address this and is presented in Appendix 2.

A6.5 Phasing of the evaluation

Figure A6.10 Phasing of the evaluation

Figure A6.10 Text Alternative

2011 — Evaluation preparation phase

  • Evaluability study (2011-2012)
  • Elaboration of Terms of Reference (2011-2012)
  • Tender procedure and contracting of the evaluation team (2012-March 2013)

April 2013 — Evaluation design phase

  • Kick-off meeting between the Developmant Evaluation Division and the team leader in Canada (April 2013)
  • Interviews main stakeholders Canada — April 2013
  • Literature review — May-June 2013
  • Preliminary portfolio analysis — May-June 2014
  • Scoping mission Kabul — July 2013
  • Draft evaluation design and work plan — July 2013
  • Final evaluation design and work plan — October 2013

October 2013 — Data collection phase

  • Interviews stakeholders Canada including face to face telephone and skype interviews — September 2013 (see Annex 2, List of interviews);
  • Document review (see Annex 3, List of documents);
  • Field mission Afghanistan — October - November 2013

November 2013 — Data analysis and reporting phase

  • Presentation of preliminary findings of the field mission to Embassy staff Kabul, November 7, 2013;
  • Presentaion of preliminary findings and conclusions to Evaluation Advisory Committee Ottawa, December 12th, 2013;
  • Elaboration of draft sector and thematic reports based triangulation of findings, review by the Development Evaluation division, December 2013 - February 2014;
  • Draft final report including comments on sector reports, March 2014;
  • June 2014 — Revised final report to be presented to the Evaluation Committee in June 2014

September 2014 — Finalization of the Technical Report

A6.6 Approach regarding the integration of findings on the Arghandab Irrigation Rehabilitation Program (AIRP) evaluation

The approach outlined in the evaluation design was to rely to an important extent on findings from the Dahla Dam/AIRP project evaluation that was commissioned by the Afghanistan Program. However, that approach could not be implemented for reasons briefly indicated in Table A6.9 above. Therefore, an alternative approach had to be developed.

The alternative approach developed for this main Signature project is based on desk research done for the summative evaluation in line with the main evaluation design, findings from interviews in Canada and Afghanistan, additional desk research and a few findings from the project evaluation on some key issues for which sufficient triangulated evidence was available on time.

This approach regarding the integration of findings on the Dahla Dam/AIRP project is comparable to the approach adopted for the other two Signature projects i.e. specific findings on these projects are presented either in the text or in specific text boxes related to the evaluation questions and the indicators defined in the evaluation matrices. Only the field work on the Dahla Dam was not explicitly focused on this Signature project, but it was also not ignored. Although it was expected that through the project evaluation substantial more information would become available than for the other projects of the sample, this expectation could not be fulfilled, but the coverage of the Dahla Dam is comparable to the other projects of the evaluation sample.

A6.7 Organization of the evaluation

The roles and responsibilities of the main actors in the summative evaluation are explained below.

Development Evaluation Division

The Development Evaluation Division was ultimately responsible for oversight of the design, implementation and dissemination of the evaluation. The Development Evaluation Division is the Technical Authority as indicated in the ToR, providing oversight, coordinating review processes and approving all deliverables. As a decision was taken in June 2014 to use the final report produced by Ecorys as the technical report, the Development Evaluation Division was also responsible for producing the final synthesis report.

Consultant

The Consultant was responsible for the detailed evaluation design, work plan, implementation of the evaluation and preparation of the final report according to high quality standards as indicated in the ToR.

Evaluation Advisory Committee

The Evaluation Advisory Committee acted as a consultative body providing guidance and recommendations to the Technical Authority. The Evaluation Advisory Committee was chaired by the Director General responsible for managing the Development Evaluation Division and included participants from the Afghanistan Development Program, stakeholders from other branches of DFATD, and the Department of National Defence. The Evaluation Advisory Committee provided comments on all deliverables.

Evaluation Peer Reviewer

An independent evaluation expert was brought in as an external evaluation peer reviewer, and was responsible for providing additional quality assurance throughout the evaluation process, including comments on all deliverables.

Departmental Development Evaluation Committee

The Development Evaluation Committee, with a majority of external members, is responsible for quality assurance of evaluation reports. This report was presented as technical report to the Development Evaluation Committee, whose members provided comments that were integrated in the final version of this report. The Committee will also be responsible for the quality assurance of the final synthesis report and will decide whether or not to recommend it for approval by the Deputy Minister of International Development.

Evaluation Team

The division of responsibilities within the evaluation team was as follows:

Appendix 7 List of documents

In addition:

Internal correspondence, e-mails, letters, aide-memoires, etc.

Appendix 8 List of people interviewed

A8.1 Ministry and Government Officials in Afghanistan

M Qarizada, Deputy Minister, Ministry of Finance
Ameen Habibi, Director General for Strategic Policy Implementation, Ministry of Finance
Mustafa Aria, Director Aid Management Division, Ministry of Finance
Ghulam Mustafa Safi, Aid Coordination Specialist, Ministry of Finance
Abdul Raheem Daud "Rahimi", Program Manager National Area-Based Development Program (NABDP), Ministry of Rural Rehabilitation and Development
Muneer Ahmad Barmak, Head, Program Management Support Unit, NABDP, Ministry of Rural Rehabilitation and Development
Ghulam Rasoul W. Rasouli, Director of Operations, National Solidarity Program (NSP), Ministry of Rural Rehabilitation and Development
Jovitta Thomas, Operations Adviser, NSP, Ministry of Rural Rehabilitation and Development
Mamoon Khawar, Donor Relations Manager NSP, Ministry of Rural Rehabilitation and Development
Noor Mohammed Arzoie, Head of Aid Coordination Unit, Ministry of Public Health
Sadia Fayeq Ayubi, Director of Reproductive Health, Ministry of Public Health
Tawab Hashemi, Health Specialist, Ministry of Public Health
Yousufzai Khoshrav, Head of Monitoring / M&E Directorate, Ministry of Public Health
Jalili Ab Maruf, Head of Evaluation / M&E Directorate, Ministry of Public Health
Mohammad Taufiq Mashal, General Director of Preventive Medicine, Ministry of Public Health
Shahla Rahmani, Provincial Polio Communication Officer, Ministry of Public Health
Abdul Salam Rasooly, EPI Manager, Eastern Region, Ministry of Public Health
Baz Mohammad Shirzad, Director of Health, Nangahar Province, Ministry of Public Health
Shukrullah Shakiv, M&E Consultant/ Gavi/ M&E Directorate, Ministry of Public Health
Musli Waheedullah, M&E Officer / Global Fund / M&E Directorate, Ministry of Public Health
Habibi Wahidullah, Chief of Paediatrics, Nangahar Regional Hospital
Kameen Wali, HMIS Officer, Nangahar province, Ministry of Public Health
Sayed Ahmad, Provincial Liaison Officer, CBE Unit. Ministry of Education
Multan Alingari, Team Leader, CBE Unit, Ministry of Education
Jalaudin Atayee, Manager for Research and Evaluation, Ministry of Education
Hamida Nizami, Director of Basic and Secondary Education, Ministry of Education
Mohammad Rahman Rahimi, Technical Advisor, CBE Unit, Ministry of Education
Abdul Haq Rahmati, Director of Academic Affairs, Ministry of Education
Khalil ur Rahman Rahmani, Provincial Liaison Officer, CBE Unit, Ministry of Education
Khowaja Mohammad Sediqi, Technical Advisor, CBE Unit, Ministry of Education
Razia Stanikzai, Senior Manager, Pre-Service Training, Ministry of Education
Arian Wassay, Director, Ministry of Education
Susan Wardak, Senior Policy Advisory & General Director of Teacher Education, Ministry of Education
Saadatullah Zaheer, Technical Advisor, CBE Unit, Ministry of Education
Fareshta Akhgar, Community Mobiliser, EQUIP, Ministry of Education
Amiri Basir, EQUIP EMIS Director, Ministry of Education
Abdul Jabbar Hazim, Provincial Social Mobiliser Supervisor, EQUIP, Ministry of Education
Seddiq Weera, EQUIP Director. Ministry of Education
Qaderi Salehi, Herat, EQUIP Director, Ministry of Education
Zia Ahmad Ahmadi, CRS Education Project Manager, Herat Teacher Training College
Wahida Baburi, CRS Education Project Officer, Herat Teacher Training College
Katy Cantrell, Head of Office, Herat Teacher Training College
Haji Ghawsoddin, CRS Education Project, Injil District Coordinator, Herat Teacher Training College
Gholam Hazrat Tanha, Director, Herat Teacher Training College
Farida Hancuzai, Principal, Dalai Girls' School, Jalalabad
Mojgan Mustafavi, Deputy Minister, Ministry of Women Affairs
Mohamed Sediq Rashid, Mine Action Coordination Centre of Afghanistan (MACCA)
Aziz Ahmad Khaled, Provincial Director of Education Kandahar
Dr. Abdul Quyoom (Pokhla), Provincial Health Director Kandahar
Dr. Qassam, Ex-provincial council member, Kandahar
M. Eshan Noorzai, Chairman Kandahar Provincial Council
Mohammed Omar Satai, Head of Joint Secretariat of Kandahar Peace Committee
Parwiz Najeeb, Chief of staff, for Kandahar Governor Office
Prof. Tooryalai Wesa, Governor of Kandahar
Rafiullah Rawan and staff, Director AICB- Kandahar office
Eng. Sher Mohammad, Director, Arghandab River Sub-basin Agency
Abdul Sattar "Baryalai and staff, Director, LKRO - Loy Kandahar Reconstruction Organisation, Kandahar
Saidullah Saeed, Financial manager, SHAO - Social and Humanitarian Assistance Organization, Kandahar
Muhammed Yaqub Sulliman, Regional Manager, Southern Region MRRD / NABDP office, Kandahar
Ehsanullah Ehsan and staff, Director, Kandahar Institute of Modern Studies of Afghan Learning and Development Organisation

A8.2 Development Partners

Joji Tokeshi, Country Director ADB
Mark Bailey, Counsellor – Development, AusAid
Semin Qasmi, Senior Program Manager, AusAid
Andrew Leigh, Deputy Head of Office DFID
Olivier Rousse, Director ECHO Afghanistan
Rocco Busco, Anna Stege, Dr Habib, EU Delegation, Team Leader Agriculture & Rural Development, Political Advisor, and Health Advisor
Katja Weigelt, Development Counsellor, Embassy of Germany
Chiara Fonghini, Program Manager, Italian Development Cooperation Office, Kabul
Azzurra Chiarini        Gender and Human Rights officer, Italian Development Cooperation Office, Kabul
Kenicho Masamaoto, Counsellor Embassy of Japan
Laetitia van Asch, Head of Development Cooperation, Netherlands Embassy
Nasrin Hoseini, Program Manager, Education and Gender (Second Secretary), Embassy of Sweden
Dr. Ken Yamashita, Director Program Coordination, Embassy of the USA
Sarah Wines, Deputy Mission Director, USAID
Edward P. Heartney, Deputy Coordinating Director for Development and Economic Affairs
Charles Swagman, Mark Michell and Allister Starr, Head USAID Kandahar and staff
Charles V. Drilling, Director Office of Economic Growth and Infrastructure, USAID
Michael Nehrbass, Deputy Director, Office of Economic Growth and Infrastructure, USAID
Freeman Daniels, Stabilization Liaison Officer, USAID
Mohammad Faiz, Acting Health Team Leader, USAID
Christopher Steel, Education and Youth Development Officer, Office of Social Sector Development, USAID

A8.3 United Nations

Aidan O'Leary, Head OCHA
Arnhild Spence, Deputy Head OCHA
Cindy Issac, Kabul Region Coordinator, OCHA Afghanistan
Shoaib Timory, Assistant Country Director, Sub-National Governance and Development Unit, UNDP
Amiri Atiqullah, Reports Officer, UNICEF
Khadija Bahram, Education Officer, Curriculum UNICEF
Catherine Panji Chamdimba, Education Specialist (CFS), UNICEF
Carmen Garrigos, Chief of Polio Eradication Unit, UNICEF
Zulfikur Ali Khan, Programme Specialist - GPE Education, UNICEF
Amina Mohammed, Chief of Field Office, UNICEF
Bo Shack, UNHCR Afghanistan Representative
Abigail Hartley, United Nations Mine Action Service
Pamela Hussain, UN Women
Rik Peeperkorn, Representative WHO
Mohammad Akram Hussain, Head of Eastern Region Office, WHO
Arshad Quddus, Team Leader PEI Afghanistan, WHO
Mehmet Akif Saatcioglu, Deputy Team Leader PEI Afghanistan, WHO
Djordje Vdovic, P4P Coordinator, WFP
Keiko Izushi, Head of Donors Relations, WFP
Marcus Prior, Deputy Head, WFP
Shafiq Yari, Programme Officer, WFP
Bob Saum, World Bank Country Director
Ditte Fallesen, Dolly Aziz and Sara Azimi, ARTF Operations Officer and staff, World Bank

A8.4 Non Governmental Organisations

Rafiullah Rawan, Kandahar office, AICB
Sima Samar, Chair AIHRC
Mohd Shafiq Nour, AIHRC - Head of Special Investigations Team, AIHRC
Qaiss Bawari, AIHRC – monitoring and investigation unit officer, AIHRC
Erik Bentzen, Director, Education, AKF Afghanistan
Farzana Bardal, National Coordinator, Education Programme, AKF Afghanistan
Khoban Kochai, National Coordinator, Health Program, AKF Afghanistan
Mohammad Dauod Khuram, National Manager, Health Program, AKF Afghanistan
Tanya Salewski, Program Manager, AKF Afghanistan
Hasina Safi, Director, Afghan Women Network
Palwasha Hassan, Founder, Afghan Women Network
Amanul Haque Chowdhury, Manager, Programme Development, BRAC
Fatema, Provincial Manager, BRAC
Golab, Provincial Manager, BRAC
Gulbuddin, Provincial Liason Officer, BRAC
Mahbubul Kabir, Coordinator BRAC International Research for Asian Countries, Sr. Research Fellow, Research and Evaluation Division
Abdul Qdaer, Project Manager, BRAC
Abdul Quyyum, Programme Manager, BRAC
Rana, Masud, Provincial Director, BRAC
Suraiya, District Manager, BRAC
Richard Paterson, international programs director, Care Canada
Alain Lapierre, Former Humanitarian Assistance coordinator, Care Canada
Jessie Thomson, Humanitarian assistance coordinator, Care Canada
Karen Moore, Interim Director, Care Afghanistan
Abdul Ghafoor Latifi, Emergency Response Coordinator, Care Afghanistan
Delawaiz Sayeda, Programme Coordinator, Humanitarian Assistance to Women in Afghanistan, Care Afghanistan
Frozan Hahmana, Monitoring and Evaluation Senior Officer, Care Afghanistan
Khawani Rashed, Program Coordinator, Humanitarian Rural Assistance Program (HRAP), Care Afghanistan
Haqmal Munib, Department Manager, HRAP, Care Afghanistan
Drew Gilmour (through Skype), Development Works Inc.
Modaser Islami, Technical Coordinator, Human Resources Development Board
Wakil Ahmad Naji, Herat Provincial Education Manager, International Rescue Committee
Nirali Mehta, BEACON - Program Director, International Rescue Committee
Bahram Barzin, Director – Operations and Acting Managing Director, Microfinance Investment Support Facility for Afghanistan (MISFA)
Shoshana Clark Stewart, Chief Executive, Turquoise Mountain
Abdul Wali, Chief Communication and Reporting Officer, Turquoise Mountain
Wazhma Frough, Women Peace and Security Research institute

A8.5 CIDA/ DFATD Development Headquarters Gatineau/ Ottawa

Adrian Walraven, CIDA Afghanistan Task Force, based in Kabul, Kandahar and Ottawa
Andrew Scyner, Former Advisor KAF (Kandahar)
Anne Lavender, CIDA Kandahar 2010-2011
Bernard Etzinger, CIDA Afghanistan Task Force, DG responsible for communications
Cheryl Urban, CIDA Afghanistan Task Force 2010-2011
Christie Skladany, Chief of staff CIDA Afghanistan Program 2011-2013
David Metcalfe, Director Kandahar Ops; Senior Director, Afghanistan, Geographic Programs Branch
Francoise Ducros, Vice President Afghanistan Task Force
Heather Cruden, CIDA Head of Aid Kabul, 2009-2011
Tracie Henriksen, Education Advisor, Ottawa
Ingrid Knutson, CIDA Head of Aid Kabul
James Melanson, CIDA Kandahar Director of Development (2009-2010) and DG of CIDA Afghanistan Program 2010-2011
Jean-Frédéric Beauchesne, CIDA Kandahar Senior Advisor Economic Growth 2008-2010, Senior advisor transition Afghanistan Task Force 2010-2011
John de Boer, CIDA Afghanistan Task Force Sector Lead Governance 2007-2011
Jonh Summerbell, CIDA Afghanistan Task Force, Senior Analyst
Lara Romaniuc, Former Program Advisor, Humanitarian Assistance (HQ)
Lawrence Peck, Team leader CIDA Afghanistan Task Force and Program
Lucas Robinson, CIDA Kandahar
Maliha Dost, Former Program Advisor, Humanitarian Assistance (HQ)
Margaret Biggs, CIDA President (2008-2013)
Michael Collins, CIDA Afghanistan Task Force, Director of Management Services 2007-2012
Michael Koros, CIDA fragile states specialist
Mojaddedi, Abdullah, Education Advisor, Ottawa
Moreno Padilla, Senior Environmental Specialist, CIDA
Nipa Banerjee, CIDA Head of Aid Kabul 2003-2006
Nicolas Lacroix, Afghanistan Program
Robert Greenhill, CIDA President (2005-2008)
Sam Millar, Director Policy Afghanistan Task Force
Stephen Salewicz, Director, International Humanitarian Assistance
Stephen Wallace, Vice President Afghanistan Task Force, Vice-President Policy
Sudeep Bhattarai, Senior Health Specialist
Vincent Raiche, Governance officer, CIDA India, Sri Lanka and Afghanistan Programme 2004-2006 CIDA Afghanistan Task Force HQ 2007-2010
Viola Cassis, Former Program Advisor, Humanitarian Assistance (HQ)
Zoe Kahn, CIDA Kandahar

A8.6 Canadian Embassy Kabul and PRT Kandahar

Abdullah Mojaheddi, Governance officer CIDA Afghanistan Task Force HQ 2006
Alia Mirza, Gender focal point CIDA Afghanistan Task Force HQ and Kabul
Geneviève Bussière, First Secretary (Health)
Catherine Coleman, Governance officer CIDA Kabul 2011-2013
Laurie Clifford, Gender specialist (consultant) Afghanistan Task Force HQ and Kabul 2004-2006 and 2011
Caroline Delaney, Governance officer CIDA Kabul 2007-2009
Claude Desilets, Deputy Head of Cooperation
Nasir Ebrahimkhail, Senior Development Officer (Health)
Genevieve Gasser, Governance officer CIDA Afghanistan Task Force HQ 2007-2—8; 2011
Mohammad Iqbal Halimi, Education Officer. Kabul
Jennifer Heys, Deputy Head of Aid
Geeta Khosla, First Secretary Health (2008-09) at Canadian Embassy in Kabul
Deborah Lyons, Canadian Ambassador, 2013-
Jennifer Miles, Gender specialist CIDA Afghanistan Task Force HQ and Kabul 2009-2011
Crystal Procyshen, Head of Cooperation
Provencher, Marie-France, Former Health Sector Staff, Afghanistan Programme / Embassy (2009-2012)
Alison Riddle, Former First Secretary - Health at Embassy of Canada, Kabul (2010-11)
Jeea Saraswati (phone interview), Gender specialist CIDA Afghanistan Task Force HQ 2007-2008
Pamela Scholey, CIDA Gender specialist involved in Afghanistan 2004-2005; 2007-2008 and 2011
Taslim Madhani, First secretary Embassy of Canada (Development)
Suzanne Quinn, Governance officer CIDA HQ and Kabul 2008-2011
Sue Wiebe, Senior Education Officer, Kabul
Ben Rowswell, Deputy Head of Mission in Kabul, ROCK in Kandahar
Elissa Golberg, Executive Director Manley Panel, ROCK 2008-2009, DG START
Ron Hoffman, Ambassador Kabul and Deputy Head of Mission Kabul
Shelley Whiting, Deputy Head of Mission Kabul, A/Ambassador Kabul

A8.6 DND and CF

Bernd Horn, Security and Defence Consultant
Dr Howard Coombs, Special Advisor to the Commander

A8.7 PCO

David Mulroney, PCO and Secretary Manley Report
Fayaz Manji, PCO

A8.8 Canadian Program Support Unit - KabulFootnote 98

Director
Education Advisor
Health Advisor
Humanitarian Advisor
Gender advisor
Development officer Governance and Human Rights
Field Monitor
Operations Manager
Mission Coordinator

A8.9 Other

Graeme Smith, Journalist and author
Janice Gross Stein, University of Toronto

A8.10 Focus Groups

Beneficiaries of the Kabul Widows Humanitarian Assistance project, Twelve Kabul Widows
Beneficiaries of the Responsive Fund for the Advancement of Women.

Appendix 9 Additional Policy and Portfolio Analysis

A 9.1 Strategic Frameworks

CIDA developed the following strategies and Results and Risk Management and Accountability Framework (RRMAFs) for the Afghanistan Program in which objectives, sector choices, aid modalities, aid channels and specific targets were defined:

A new Afghanistan Program strategy 2014-2017 was being prepared as of early 2014.
Across as well as within the noted phases, Afghanistan Development Program priorities changed. This is captured in Table A 9.1, which illustrates that while the sector priority labels changed, several of the sub-themes that were addressed remained the same.

Table A9.1 Priority sectors in the Afghanistan Program as per strategic documents and logic models
2003-2005Footnote 992006 - 2007Footnote 1002007-2008Footnote 1012008-2011Footnote 1022011 – 2014Footnote 103
Security and Rule of LawDemocratic development and effective government
Building capacity and strengthening institutions
Stabilization in Kandahar
Governance
Rule of law
Democratic institutions and processes
Public Administration Reform
Human Rights
Governance
(National institutions and democratic development)
Public Institutions
Electoral processes
Human rights
in particular women's rights
Natural resource management
Community-driven developmentFootnote 104
Enhancing the role of women and girls in society
Education of women and girls
Micro-loans
Women's rights Institution building
Social and economic development
Gender Equality
Private Sector Development
Community-based infrastructure Economic governance
Natural resources
Basic services
Economic growth
Community-based infrastructure
 
Rural livelihoods and Social protection
Increased access to social services
Sustainable rural livelihoodsFootnote 105
Agriculture, livestock and horticulture support
Micro-loans
Social and economic development
Access to Health
Access to Education
Basic services
Access to Education
Access to Health servicesFootnote 106 Eradicate Polio
Basic education and maternal, newborn, and child healthFootnote 107
Security
Human and physical safety (mine action/ mine risk education, resettlement)
Humanitarian assistance
Emergency assistance and preparedness
Mine action
Humanitarian assistance

*Note:   Sector categories over time are not always fully comparable. Inclusion in the same rows indicates at least partial overlap similarities in the respective thematic areas of programming. There is a lack of information on the period 2003-2005 as regards whether and how specific projects supported during this period corresponded with the noted priority sectors.

This table on CIDA's priorities in combination with the portfolio analysis clearly indicates the following:

These observations illustrate that most of the noted sectors constitute very broad concepts that capture a multitude of rather diverse sub-themes. Furthermore, the strategic documents provide very little information on the criteria and/or processes used to select and label sector and/or related thematic (or sub-thematic) priorities, or on how these priorities were used to inform decisions on specific projects/investments or on sector exit strategies. The documents do refer to alignment with priorities of the GIRoA and perceived comparative advantage of Canada in the priority sectors.

A 9.2 Portfolio analysis

The source of the portfolio information is DFATD's CFO Statistics Branch and the dataset has been generated from the system on 18 April 2013. Information related to fiscal year 2012-2013 is preliminary. All calculations performed by the evaluation team are based on information contained in the dataset listing the 310 initiatives that form the portfolio of the Afghanistan program. Figures exclude imputed long-term institutional support to multilateral organizations.

During the data analysis process a series of minor corrections have been made with regard to the coding and the definitions used in the dataset. Which each revision, the changes made in the dataset have been duly recorded in a special "changes" tab. For example, the KLIP program was originally classified as private sector development in relation to the principal sector of focus. It has been re-classified as multi-sector. The dataset also listed three NABDP initiatives, of which one was implemented by the International Criminal Defence Attorneys Association.  The title of this $3.1 million project was subsequently changed to Nationalization of Legal Aid Services, after checking the project codes in the CIDA database. Other changes made by the evaluation team relate to the classification of aid channels and the Kandahar focus of projects. No changes have been made with regard to the gender coding used by CIDA. 

With regard to the disbursement data it needs to be realized that the data relate to payments made in a given fiscal year. The majority of projects or initiatives record multi-year payments. Taking into account the long evaluation horizon (total of 9 Fiscal Years) and the large numbers of sectors (7), trends in disbursements per sector give a good overall indication of changes in priorities in line with changes in the strategic thrust of the program.

Figure A9.1 Annual disbursements Afghanistan Program, fiscal year 2004-05 to 2012-13 Footnote 108

Figure A9.1 Text Alternative
Comparing # projects, total budget, total disbursement, and disbursement within evaluation time period, by CIDA Branch for Afghanistan
Row Labels# projectsProject total budgetProject total disbursement (2)Total disbursements over 2004/2005 to 2012/2013 in Afghanistan
Geographic Program Branch211$1,622.35 M$1,487.11 M$1,476.77 M
Multilateral and Global Programs Branch30$232.73 M$232.29 M$60.76 M
Partnership with Canadians64$228.66 M$138.17 M$8.69 M
Strategic Policy and Performance5$0.37 M$0.25 M$0.05 M
Grand Total310$2,084.10 M$1,857.82 M$1,546.26 M

The top ten projects with highest disbursements are the following:

Table A9.2 Top Ten Projects - Afghanistan Program, 2004/05 to 2012/13, in million $
Sector of FocusName of ProjectNo. of
Initiatives
Executing AgencyPeriodTotal Dis-bursem.
Other/MiscellaneousRecurrent cost window5World Bank - ARTF2004-2013203
Democratic governance* National Solidarity Program (NSP)3World Bank - ARTF2004 - 2010148
Peace and Security Mine Action Program Afghanistan4UNMAS2004-2012100
Economic Growth Microfinance Investment Support Facility for Afghanistan (MISFA)3World Bank - ARTF2004 - 201095
Education Education Quality Improvement Program (EQUIP)Footnote 1091World Bank – ARTF2007 - ongoing92
Humanitarian AssistanceWFP Appeals6World Food Programvarious years85
Health Global Polio Eradication Initiative4WHO and UNICEF2006 -ongoing93
Economic Growth Dahla Dam/ Arghandab Irrigation Rehabilitation3SNC Lavalin International/
CADG
2008 - 201247
DemocraticFootnote 110 governance National Area Based Development Program (NABDP)3UNDP2004 - 201141
Education Girls Education Support Program (GESP)Footnote 1112BRAC and
AKF
2006-201326

Most of these projects are either implemented through the UN or via ARTF, with the exception of the Dahla Dam signature project implemented by the private sector and the Girls education Support Program implemented by NGOs. It is therefore not surprising that the International Financial Institutions and the United Nations Agencies are the two most important aid channels, each representing 38% of all disbursements. The International Financial Institutions include all ARTF funding via the World Bank.

A9.3 Aid Channels

Figure A9.2 presents the main aid channels used by the Afghanistan Program during the entire evaluation period.

Figure A9.2 Total disbursements by type of executing agency, 2004-05 to 2012-13

Figure A9.2 Text Alternative
  • International financial Institution - 38%
  • United Nations - 38%
  • Civil Society - 15%
  • Private Sector - 6%
  • Other - 3%

When the use of aid channels is analyzed over the evaluation period, a clear pattern emerges, as illustrated in Figure A7.3 In 2004-05 to 2007-08, more than 90% of all disbursements were channelled via the World Bank/ARTF (and a very small proportion of other international financial institutions) or the UN, and the share of the private sector and civil society partners were minimal (see also Appendix 5 for the distribution per aid channel per year in %). This drastically changed from 2008-09 onwards when civil society and private sector became more important partners. However, the share of private sector funding declined again from 2011-12, which is related to the Canadian transition away from the economic growth sector. In 2012-13, civil society became the most important aid channel for the first time with 37% of total disbursements.Footnote 112

Figure A9.3  Aid channels by type of executing agency, 2004-05 to 2012-13, per year

Figure A9.3 - Text Alternative
Aid channels by type of executing agency, 2004-05 to 2012-13, per year(in million dollars)
Type of Executing Agency2004/052005/062006/072007/082008/092009/102010/112011/122012/13Total disbursements over 2004/05 to 2012/13
Civil Society3.899.869.6517.7036.1339.5250.1829.4835.32231.73
Private Sector0.190.080.212.6713.0017.9837.4611.781.1384.50
United Nations50.3436.6549.7073.50123.74116.5580.5242.1018.63591.74
International Financial Institution45.0052.00117.30183.3043.8745.7034.8132.0034.50588.49
Other3.942.491.993.227.0010.3312.513.305.0149.81
Grand Total103.36101.09178.86280.39223.75230.08215.48118.6694.591,546.26

The following figures present a further breakdown of disbursements over the aid channels:

Figure A9.4 Aid channels by type of Executing Agency, 2004-05 to 20112-13 per year in %

Figure 9.4 Text Alternative
(percentage)
Type of Executing Agency2004/052005/062006/072007/082008/092009/102010/112011/122012/13Total disbursements over 2004/05 to 2012/13
Civil Society3.76%9.76%5.40%6.31%16.15%17.18%23.29%24.84%37.34%14.99%
Private Sector0.18%0.08%0.12%0.95%5.81%7.82%17.38%9.93%1.19%5.46%
United Nations48.70%36.26%27.79%26.21%55.30%50.66%37.37%35.48%19.70%38.27%
International Financial Institution43.54%51.44%65.58%65.37%19.61%19.86%16.16%26.97%36.48%38.06%
Other3.82%2.46%1.11%1.15%3.13%4.49%5.80%2.78%5.30%3.22%
Grand Total100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%

Figure A9.5 Comparison disbursements between UN and international financial institutions, per year in %

Figure 9.5 Text Alternative
(percentage)
Type of Executing Agency2004/052005/062006/072007/082008/092009/102010/112011/122012/13Total disbursements over 2004/05 to 2012/13
United Nations52.80%41.34%29.76%28.62%73.82%71.84%69.82%56.81%35.06%50.14%
International Financial Institution47.20%58.66%70.24%71.38%26.18%28.16%30.18%43.19%64.94%49.86%
Grand Total100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%100.00%

Figure A9.6 Disbursements per UN agency, per year in %

Figure 9.6 Text Alternative
(percentage)
Disbursements to the United Nations2004/052005/062006/072007/082008/092009/102010/112011/122012/13Total disbursements over 2004/05 to 2012/13
UNDP - United Nations Development Programme27.54%31.65%16.98%17.00%27.04%12.63%21.78%5.00%0.00%159.61%
UN-HABITAT - United Nations Human Settlements Programme%0.00%0.00%3.48%3.02%0.00%19.70%0.70%0.00%0.00%26.90%
UNHCR - United Nations High Commissioner for Refugees5.00%0.00%0.00%5.80%5.00%3.00%2.95%0.00%0.00%21.75%
UNICEF - United Nations Children's Fund0.00%0.00%5.35%5.13%9.95%6.30%18.26%10.00%2.01%57.00%
UNMAS - United Nationsine Action Service10.00%5.00%7.00%21.80%26.00%10.00%17.00%3.10%0.00%99.90%
WFP - World Food Programme5.00%0.00%11.90%16.80%26.20%37.35%15.00%13.00%7.52%132.77%
WHO - World Health Organization1.00%0.00%5.00%3.95%29.56%26.78%3.54%11.00%7.10%87.92%
Other UN organizations1.80%0.00%0.00%0.00%0.00%0.80%1.28%0.00%2.00%5.89%
Grand Total50.34%36.65%49.70%73.50%123.74%116.55%80.52%42.10%18.63%591.74%

A9.4 Sectors of Focus

The following figures show the division in Principal DAC Sector codes per Principal Sector of Focus:

Figure A9.7 Democratic Governance using DAC sector codes

Figure A9.7 Text Alternative
  • Democratic participation and civil society - 28%
  • Public sector policy and administrative management - 24%
  • Decentralisation and support to subnational government - 20%
  • Elections - 15%
  • Human rights - 8%
  • Other - 3%

Figure A9.8 Economic Growth using DAC sector codes

Figure A9.8 Text Alternative
  • Informal/semi-formal financial intermediaries - 30%
  • Agicultural water resources - 17%
  • Road transport - 12%
  • Democratic participation and civil society - 9%
  • Food crop production - 7%
  • Input required - 6%
  • Other - 19%

Figure A9.9 Education using DAC sector codes

Figure A9.9 Text Alternative
  • Multiple areas - education - 64%
  • Primary education - 20%
  • Education facilities and training - 5%
  • Other - 11%

Figure A9.10 Health using DAC sector codes

Figure A9.10 Text Alternative
  • Infectious disease control - 59%
  • Health policy and administrative management - 12%
  • Tuberculosis control - 11%
  • Basic health infrastructure - 6%
  • Reproductive health care - 6%
  • Other - 6%

Figure A9.11 Emergency assistance using DAC sector codes

Figure A9.11 Text Alternative
  • Emergency food aid - 57%
  • Material felief assistance and services - 15%
  • Relief co-ordination; protection and support services - 8%
  • Input required - 6%
  • Other - 14%

Figure A9.12 Peace and security using DAC sector codes

Figure A9.12 Text Alternative
  • Removal of land mines and explosive remnants of war - 82%
  • Participation in international peacekeeping operations - 12%
  • Other - 6%

A9.5 Gender coding

An overview of the gender coding of the Afghanistan Program portfolio provides a first indication of the extent to which gender equality was considered in programming. The coding specifies whether a project was designed based on a gender analysis and whether gender equality results were consistently included in its intervention logic.Footnote 113

Figure A9.13  Gender coding Afghanistan Program 2004-05 to 2012-13

Figure A9.13 Text Alternative
  • No gender equality results - 56%
  • Gender equality "integrated" initiative - 35%
  • Limited gender equality results - 6%
  • Gender equality "Specific" initiative - 3%

Appendix 10 Donor Disbursements to Afghanistan 2002-2011 (million US $), excluding multilaterals and excluding USA

Source: Islamic Republic of Afghanistan, Ministry of Finance, Development Cooperation Report 2010 and 2011.

Appendix 11 Management Response

Overarching Comments on the Summative Evaluation of Canada's Afghanistan Development Program (Fiscal years 2004-2005 to 2012-2013)

  1. This summative evaluation is a strategic milestone for the Afghanistan Development Program.  The Afghanistan Development Program is cognizant of the significance of this evaluation exercise for Canada, our development partners, the Government of Afghanistan, and the international community. The evaluation findings offer the opportunity to share lessons learned with the international donor community, while adapting to new realities in Afghanistan over the years ahead as Afghanistan enters its self-declared Transformation Decade (2015-2024).
  2. The Program recognizes the complexity and magnitude of this evaluation exercise spanning nine fiscal years, and the good work delivered by the Evaluation Team. The Afghanistan Development Program agrees with the report's conclusions, and acknowledges that over the period of the evaluation, the contextual environment for planning and implementing a development assistance program in Afghanistan within an international military mission was of the highest order of complexity and insecurity. While CIDA had limited expertise working in this context, the Program developed skills and innovative approaches over time to better support Afghan and international efforts aimed at establishing the conditions that lead to generating sustainable development.
  3. The evaluation affirms that Canada is recognized as a main development partner of Afghanistan and effectively participated as a member of the international community in policy dialogue with the Afghan government, and that the Program contributed to both short term and long term results that led to real improvements in the lives of Afghans.
  4. The evaluation presents areas where the Program should direct its focus on gender equality and human rights, and that promote synergies across and within sectors, while continuing to build upon policy dialogue, planning and programming efforts with strong leadership and support to the government to achieve concrete goals.
  5. The evaluation also helps inform the next phase of the Afghanistan Program over the coming years. Through this exercise, the Program has drawn upon key findings, lessons learned, and recommendations that will help shape the Afghanistan Development Program Strategy over the 2014-2019 period.

Canada's announced commitment to Afghanistan of $227 million over 2014-2017 will build on areas where the Program has demonstrated leadership and experience, and where there is a strong basis for continued development assistance and results, namely in education, health, capacity building for disaster risk reduction and advancing the rights and empowerment of women and girls.

Recommendation 1

Establish an institutional mechanism to capture lessons learned from the implementation of the Whole of Government approach in Afghanistan and elsewhere, to better inform future Canadian engagement in fragile states.

Commitments/measures

Partially Agree:  

The Department agrees that it is important to ensure that lessons learned in one challenging context are understood and applied as appropriate in other situations.  Rather than establish a stand-alone institutional mechanism focussed on lessons learned from Afghanistan, the Department's preference is to use established channels to achieve this objective, including relevant departmental governance committees such as Program Committee, missions in the field (which retain locally-engaged staff with a wealth of knowledge), and key departmental bureaux.  In terms of fragile states policy, the Stabilization and Reconstruction Task Force (START) Bureau is the department's focal point and, as part of its ongoing role, facilitates dialogue on fragile states policy and applies lessons learned to future engagements in situations requiring an extraordinary Canadian response.  The ADM, International Security and Political Affairs also has a dedicated role in coordinating whole-of-DFATD and whole-of-government integrated responses to major crises, bringing together, as appropriate, security, defence, development and diplomatic policy and programming responses, and building on lessons learned.

Responsible - ADM IFM

Completion date - N/A

Recommendation 2

Develop a vision for Canada's future engagement in Afghanistan, taking lessons from the implementation of the Whole of Government approach into account.

Commitments/measures

The Afghanistan Development Program agrees with this recommendation. 

While the Program certainly agrees with the Whole-of-Government approach, the Government of Canada's footprint in Afghanistan has been reduced to a few ministries. This includes the amalgamation of Canada's development agency with the foreign affairs and trade ministry.

The Program will develop a Bilateral Country Development Strategy (2014-2019) in consultation with relevant partners and other government departments, and that takes into consideration:

Responsible - ADM Asia Pacific

Completion date - The Program's completed Bilateral Country Development Strategy (2014-2019) is approved by the Minister in 2015.

Canada's vision for its engagement in Afghanistan (2014-2019) is presented to the Afghan Ministry of Finance through the annual Development Coordination Dialogue by September 2015.

Recommendation 3 : Governance

The crosscutting nature of governance should be further enhanced in the Afghanistan Program, including the strengthening of linkages between political dialogue and development policy dialogue with Afghan government partners. Programming decisions on the type of support to be provided – on-budget versus off-budget support – should be based on clear targets and directly linked to on-going political and policy dialogue.

Commitments/measures

The Afghanistan Development Program agrees with this recommendation.

The Program will:

Responsible - ADM Asia Pacific

Completion date - On an ongoing basis, the Program will engage on the implementation of the TMAF, as determined by the Government of Afghanistan and in coordination with other international partners, as DFATD's interlocutor with the Government of Afghanistan.

The Policy Dialogue Toolkit will be developed by April 2015.

On an annual basis, the program will apply the approved approach to incentivization and ensure alignment with other international donors, where possible.

On an ongoing basis, the Program will engage in ongoing Development Cooperation Dialogues with the Government of Afghanistan (Ministry of Finance).

Recommendation 4

Continue the focus on gender mainstreaming while adapting it to ensure improved responsiveness to socio-cultural values and principles, to the extent possible.

Commitments/measures

The Afghanistan Development Program agrees with this recommendation.

While the Program certainly agrees to continue with the focus on gender mainstreaming, it must be recognized that in Afghanistan, there is continued limited acceptance of women's participation in the economic, social and political spheres of society.  This limited acceptance is a barrier to effective gender mainstreaming.

In the development of the Bilateral Country Development Strategy (2014-2019), the Program will use gender equality –with a focus on rights and empowerment of women and girls – as the "integrating factor" across the Program's development assistance intervention areas. This will also entail the development of a sector-level logic model articulating our results commitments on initiatives supporting women's and girls' rights and empowerment.

The Program, together with other partners, will explore innovative approaches to adapt the objective of gender equality to the cultural reality of Afghanistan including the importance of engaging men, boys, women, girls, religious leaders and social authorities. New approaches will be based on comprehensive gender equality analysis to understand the gender dynamics, to develop applicable initiatives, and to mitigate the associated risks.

To increase the Program's Gender Equality (GE) components, regular GE training will be offered to staff at headquarters and in Kabul.

Responsible - ADM Asia Pacific

Completion date - The Program's completed Bilateral Country Development Strategy (2014-2019) is approved by the Minister in 2015.

The Program's completed Women's and Girls' Rights and Empowerment Sector Strategy (2014-2019) is approved by the Program's Senior Director in September 2015.

The Program's completed Women's and Girls' Rights and Empowerment Sector Level Logic Model (2014-2019) is approved by the by the Program's Senior Director in September 2015.

Annual Gender Equality training is delivered to Program staff at headquarters and in Kabul on an ongoing basis.

Recommendation 5

For future investment in key sectors, ensure clear strategic direction, including a realistic risk analysis and robust risk mitigation strategy:

Education – undertake the transition from a program focused primarily on access to education to one that also targets quality education with an increased focus on learning outcomes, and that facilitates students' transition through different stages of education (for example, from community-based to formal education).

Health – strengthen program focus on the right to health, social equity and the objectives defined as part of Canada's commitments to Maternal, Newborn and Child Health (MNCH).

Human Rights – strengthen the protection of human rights by increasing awareness and capacities on the part of the government and non-governmental actors through political and policy dialogue, and programming.

Humanitarian Assistance – seek opportunities to further strengthen the linkages between relief, rehabilitation and development while ensuring that humanitarian assistance continues to be delivered in line with the principles of good humanitarian donorship.

Commitments/measures

The Afghanistan Development Program agrees with this recommendation. 

In the development of a Bilateral Country Development Strategy (2014-2019), the Program will develop sector strategies that take into consideration:

Specifically:

Education: the sector strategy will continue to focus primarily on basic education, with a view of improving: access to education, quality of education, and the capacity of local systems to deliver education services. This effort will take into consideration:

Health: the sector strategy will focus on promoting women's and girls' rights through strategic investments aimed to improve their access to quality health services, in line with Canada's continued international commitments in Maternal, Newborn and Child Health and the fight against polio, and take into consideration:

Human Rights: the sector strategy will focus on leveraging the lessons learned from the Whole of Government experience and continue to refine and enhance an already operational strategic plan for Canada's involvement in protecting and promoting human rights in Afghanistan, especially in the area of the rights and empowerment of women and girls, and will take into consideration:

Humanitarian Assistance: the sector strategy will focus on  linking relief to recovery and development (LRRD), bridging the gap between the provision of emergency relief phase humanitarian assistance and to the short and longer term sustainable development phases, in consultation with DFATD's International Humanitarian Assistance Bureau, that will take into consideration:

In the context of developing and implementing the Bilateral Country Development Strategy (2014-2019), the Program will ensure its risk management and approaches through:

In addition, the Program will:

Responsible - ADM Asia Pacific

Completion date - The Program's completed Bilateral Country Development Strategy (2014-2019) is approved by the Minister in 2015.

The Program's completed Education, Health, Women's and Girls' Rights and Empowerment, and Humanitarian Assistance Sector Strategies (2014-2019) are approved by the Program's Senior Director in September 2015.

The Afghanistan Program Level Logic Model (2014-2019) and Program Level Performance Measurement Framework is aligned to DFATD's strategic development priorities and completed in 2015 and approved by June 2015 by the Asia-Pacific Bureau's Director General (Development).

The Afghanistan Program's assessment of risk through the Program-level Risk Register exercise is completed on an ongoing basis at twice-yearly intervals.

DFATD risk assessments are completed for all new projects on an ongoing basis.

Date Modified: