A Review of African Development Bank (AfDB) Program Evaluation Reporting - 2007-2012 - Report

July 2013

List of Acronyms and Abbreviations

Acknowledgments

The Development Evaluation Division wishes to thank all who have contributed to this review exercise for their valued input, their constant and generous support, and their patience. Our thanks go first to the independent team of the consulting firm Universalia Management Group, made up of Principal Evaluator Marie-Hélène Adrien, Senior Evaluator Charles Lusthaus and Consultants Halcyon Louis, Elisabetta Micaro and Emma Mason.

The Development Evaluation Division would also like to thank Vaughn Lantz, Vincent Raiche and Mariette Maillet from the Multilateral and Global Programs Branch for their valuable support. Our thanks also go to Mohamed Manai and Rakesh Nangia, representatives of the AfDB in Tunis, for their useful and practical advice to the evaluators.

From the Development Evaluation Division, we wish to thank Nancy Odeh, Evaluation Officer, for her significant contribution to refining report drafts. We also thank Andres Velez-Guerra, Team Leader, for contributing to report revisions and guiding this review to completion, and James Melanson, Evaluation Director, for overseeing the review.

Caroline Leclerc
Head of Development Evaluation

Executive Summary

Background

The AfDB was founded by signatory agreement between regional member states in 1963. It is one of three entities that together comprise the African Development Bank Group, the other two being the African Development Fund and the Nigeria Trust Fund.

As a regional multilateral development finance institution, the mission of the AfDB is to contribute to the sustainable economic development and social progress of African countries. Membership of the AfDB (as of December 2011) comprises 77 countries, of which 54 (70 percent) are regional members. The AfDB provides development assistance to its regional member countries through the public and private sectors, in the form of financial instruments, policy advice, and technical assistance. In 2011, the AfDB financed 184 operations in regional member countries at a cost of UA 5.72 billion.

Purpose and Objectives

The purpose of the review was to provide an independent, evidence-based assessment of the AfDB’s development effectiveness for use by the Government of Canada, the AfDB and other interested shareholders. However, a preliminary assessment of the AfDB’s program evaluation reporting for the period 2007- 2012 noted shortcomings in coverage and quality.

This report therefore provides forward-looking recommendations to:

1. Support the strengthening of the AfDB’s evaluation and reporting on its development effectiveness; and
2. Guide future engagement between DFATD’s^{Footnote 1} Multilateral and Global Programs Branch, and the AfDB on evaluation and development effectiveness reporting.

Methodology

A generic approach and methodology for the review of development effectiveness of multilateral organisations was developed under the guidance of the OECD-DAC Network on Development Evaluation (DAC-EVALNET). In line with this methodology, the review of the AfDB involved three distinct steps:

• Step 1: Establishing the evaluation universe in collaboration with the AfDB by identifying all program-level evaluations and development effectiveness reports^{Footnote 2} published by the AfDB between 2007 and 2012.
• Step 2: Conducting a preliminary review of AfDB program-level evaluations and development effectiveness reports through a two-stage process entailing:
  • assessing whether a given document in the universe was an evaluation or development effectiveness report, as opposed to a policy review or background document, and whether it fell within the time scope established for the review; and,
  • scoring the coverage and quality of the evaluation reports using the DAC-EVALNET Evaluation Coverage and Quality Screening Guide.
• Step 3: Using the results of the preliminary review, triangulated with MOPAN’s assessment of the institution, to classify the AfDB independent evaluation function and its development effectiveness reporting under one of three scenarios:
  • adequate;
  • not adequate but providing an adequate information base on development effectiveness; and
  • not adequate and not providing an adequate information base on development effectiveness.

Following the implementation of the above-mentioned process, it was determined that the AfDB’s program evaluation reporting is not adequate and the multilateral organisation’s evaluation function does not provide an adequate information base on overall development effectiveness. Therefore, this report presents recommendations aimed at supporting improved evaluation and development effectiveness reporting, as suggested by the OECD-DAC evaluation methodology.

Key Findings of the Preliminary Review

The AfDB evaluation function has not produced an adequate number of program-level evaluations and development effectiveness reports over the period 2007-2012.

Out of the 43 documents provided by the AfDB for the formulation of the evaluation universe, only 20 documents met all the DAC-EVALNET methodology preliminary review criteria. The remaining 23 reports were not screened in during the preliminary review for three main reasons.

First, the document’s scope was outside the selected review timeframe (6 documents). Second, while fulfilling other objectives, the document in question was not a program-level evaluation or a development effectiveness report (13 documents). Third, the document received a low quality score (4 reports) for reasons that included: the lack of a clear description of the evaluation objectives, or limitations; or a poor evaluation design.

Program-level evaluations and development effectiveness reports within the review period (2007-2012) provided inadequate coverage of the results of AfDB programming in its priority areas and its strategic objectives.

The 20 documents that met all the preliminary review criteria, together addressed all but two of the AfDB’s priority areas, with one report taking a multisector focus. Importantly, however, these documents provided inadequate coverage of the results achieved. The preliminary review has noted that AfDB program-level evaluation reporting on its priority areas and strategic objectives is largely descriptive, and focuses on AfDB programming activities without covering AfDB development results. This implies a need for the AfDB to revisit its approach to program-level evaluation, with a focus on its reporting on development results.

AfDB program-level evaluations and development effectiveness reports provide insufficient coverage of AfDB investment in its crosscutting thematic priorities.

Of the 20 reports that met all the preliminary review criteria, only five reports addressed the AfDB crosscutting thematic priorities. One report included discussion of the gender theme, and three reports addressed the environment. An additional report addressed climate change. This finding indicates insufficient coverage by the AfDB’s program-level evaluations and development effectiveness reports of investment made in priority crosscutting themes.

The usefulness of AfDB development effectiveness reporting to decision-making on the Bank’s forward-looking investments is sometimes impeded by the time lag between the year an evaluation is conducted and the year it is finalized and published.

All 43 documents included in the preliminary review universe were published during the established timeframe of January 2007 to September 2012. However, 8 reports had a significant lag between their publication and the years they reviewed. For example, a report published in 2007 evaluated programming during 1996-2004, thus implying a three-year lag between implementation and publication. The lag between implementation and finalization of AfDB evaluations reduced their usefulness in informing future programming.

Some evaluation designs in the program-level evaluation reports reviewed did not follow the DAC-EVALNET quality guidelines. This is a potential area for improvement.

The preliminary review awarded points to each report using the quality screening criteria of the DAC-EVALNET (Evaluation Coverage and Quality Screening Scoring Guide) to determine evidence of two aspects of quality: the existence of basic evaluation report characteristics; and the existence of key components required for a review of development effectiveness.

The preliminary review findings indicated that all 20 program-level evaluation reports surpassed the threshold required for basic evaluation report characteristics. However, preliminary review findings also indicate that the reviewed program-level evaluations received low scores on the criterion of appropriate evaluation design. These scores echo the 2012 self-assessment of the AfDB’s independent evaluation function, which stated that the AfDB employed a narrow range of evaluation methodologies in its project-level evaluations.

The preliminary review findings indicated that all 20 program-level evaluation reports surpassed the threshold required for basic evaluation report characteristics. However, preliminary review findings also indicate that the reviewed program-level evaluations received low scores on the criterion of appropriate evaluation design. These scores echo the 2012 self-assessment of the AfDB’s independent evaluation function, which stated that the AfDB employed a narrow range of evaluation methodologies in its project-level evaluations.

Conclusions

The findings of the preliminary review of the AfDB program-level evaluations indicate that the evaluations included in the review do not provide an adequate level of coverage of the AfDB’s program-level development results.

At the same time, it should be noted that the AfDB has demonstrated a clear commitment to addressing the challenges mentioned above. In particular, the AfDB has indicated their intention to: ‘raise the bar on quality’, ensure methodological rigour, conduct more program level evaluations, focus on timing/usefulness and strategic priorities (including crosscutting issues), and improve the quality assurance process. These intentions arise from both the 2012 self-assessment of the AfDB’s independent evaluation function, and the AfDB 2013-2017 independent evaluation strategy.

Recommendations

Strengthening its independent evaluation function should be a priority for the AfDB.

A stronger evaluation function would enable assessment of progress towards the achievement of program-level development outcomes and the capture of lessons learned to improve its investments in regional member countries. The areas of AfDB evaluation activity in which the review findings have noted particular weakness were the design of evaluations and the coverage of program-level development effectiveness results. The anticipated results of strengthening the AfDB evaluation function would be increased robustness in evaluation design through the use of appropriate methodologies, and more comprehensive reporting on development effectiveness.

The AfDB should consider engaging in a peer review of its evaluation function.

As it moves toward improving its evaluation and development effectiveness reporting, the AfDB could add value to its earlier self-assessment by engaging a peer review. This could be used by the AfDB to gain further insight into the capacity and quality of its evaluation function, and to assess the relevance, coverage, and overall quality of its development effectiveness reporting. The results of this assessment could help the AfDB increase the extent to which its program evaluations and development effectiveness reports inform AfDB strategic decision-making. Importantly, the AfDB has indicated that it plans to engage in a peer review in 2014 once the independent evaluation strategy is operational.

Canada, in concert with other donors and AfDB Board members, should support the Bank’s efforts to strengthen its evaluation function.

Canada should support the AfDB’s efforts to strengthen its evaluation function and the design of its evaluations, as outlined above. One possibility, among others, would be an annual analysis of the AfDB’s evaluation coverage to be included as part of the AfDB’s evaluation work plan. Specifically, the AfDB’s rolling work plan could provide some measure of the target for, and actual implementation of, program-level evaluations, perhaps as a percentage of total expenditure. In working with other Board members and donors, Canada should signal its readiness to support the strengthening of the AfDB’s evaluation function. Important elements would include peer review; capacity development to strengthen evaluation coverage, design, and timeliness; and strengthened program-level development effectiveness reporting.

1.0 Background

1.1 Introduction

The overall approach and methodology for the preliminary review was developed by a multidonor task group under the guidance of the OECD-DAC Network on Development Evaluation (DAC-EVALNET). In 2010, the DAC-EVALNET methodological approach was piloted on reviews of the development effectiveness of the Asian Development Bank (ADB), and the World Health Organization (WHO). In 2011-2012, the assessment methodology was applied to full reviews of the development effectiveness of the United Nations Development Programme (UNDP), and the World Food Programme (WFP).

1.2 Why Conduct this Review?

The purpose of the review was to provide an independent, evidence-based assessment of the AfDB’s development effectiveness for use by the Government of Canada, the AfDB and other interested shareholders. However, a preliminary assessment of the AfDB’s program evaluation reporting for the period 2007-2012 noted shortcomings in coverage and quality. This report therefore provides forward-looking recommendations to:

1. support the strengthening of the AfDB’s evaluation and reporting on its development effectiveness; and
2. guide future engagement between DFATD’s Multilateral and Global Programs Branch and the AfDB on evaluation and development effectiveness reporting.

The African Development Fund provides concessional loans and grants to 41 low-income African countries, and technical assistance for studies and capacity-building activities.^{Footnote 3} Discussions during the midterm review of ADF-12 focused on AfDB development effectiveness, as well as on its operational priorities, institutional effectiveness, resource allocation, and the financing capacity and sustainability of the African Development Fund. This report will therefore build on the findings of the midterm review, and may inform potential discussion of the Bank’s evaluation function.

1.3 The African Development Bank

Founded by signatory agreement between member states in 1963, the African Development Bank is one of three entities that together comprise the African Development Bank Group (AfDB), the other two being the African Development Fund and the Nigeria Trust Fund. As a regional multilateral development finance institution, the mission of the AfDB is to contribute to the sustainable economic development and social progress of African countries. Membership of the AfDB comprises 77 countries, of which 53 (69 percent) are regional member countries. The remaining 25 countries (31 percent) are non-African/ non-regional member countries. The supreme governing body of the AfDB is the Board of Governors, the members of which are ministers and high-level officials from economic and financial institutions of member countries.

The AfDB provides development assistance to its regional member countries through the public and private sectors, in the form of financial instruments, policy advice and technical assistance. The main financial instruments used in AfDB interventions in its regional member countries are:

• Project lending;
• Policy-based operations;
• Grants;
• Lines of credit;
• Equity participation; and
• Guarantees.

Contributions to enable the delivery of development assistance are provided by each of the three AfDB entities:

• the African Development Bank generates resources through capital market borrowings for the provision of non-concessional loans;
• resources from the African Development Fund derive from contributions and fund-replenishment by State participants on a 3-year basis; and
• the Nigeria Trust Fund provides resources under a signed agreement that specifies the terms of usage.

In 2011, the AfDB financed 184 operations in regional member countries at a cost of UA 5.72 billion. Table 1 provides the percentage breakdown by type of financial instrument. Figure 1 provides an overview of financial allocations to regional member countries in 2011, by area of strategic focus.

Figure 1: AfDB Sector Financing to Regional Member Countries, 2011

1.4 Strategic Objectives and Priority Areas of the AfDB

The 2008-2012 Medium–Term Strategy of the AfDB outlined the Bank Strategic Framework, comprising five strategic objectives and related priority areas for AfDB activities over this period. The strategic objectives identified in the 2008-2012 Medium–Term Strategy are presented below:

• Operational Selectivity;
• Results and Performance;
• Decentralization;
• Streamlined Internal Processes; and
• Staff, Knowledge, Financials and Partnerships as Foundation.

Under the strategic objective of Operational Selectivity, the AfDB committed itself to vertical and horizontal areas of focus, as well as crosscutting priorities and selected areas of intervention. The AfDB prioritized a vertical focus on four key areas that are distinct to the AfDB’s sectors and topics: Infrastructure; Governance; Higher Education and Vocational Training; and the Private Sector^{Footnote 5}. This was supported by a horizontal emphasis on: Regional Integration; Fragile States; and Client-responsive Engagement, with a focus on middle-income countries. In the interest of mainstreaming crosscutting themes, the thematic focusesfor the AfDB were: gender; the environment; and climate change. The AfDB further indicated that it would provide selected support to agricultural development.

In April 2013, the Bank adopted a new Ten Year Strategy (TYS) for the period 2013-2022^{Footnote 6}. It will focus on two objectives: inclusive growth, and the transition to green growth. The TYS reaffirms the Bank’s strategic focus on five core operational areas:

• Infrastructure development;
• Private sector development;
• Regional economic integration;
• Governance and accountability; and
• Skills and technology.

In implementing its ten-year strategy, and as an integral part of the two objectives, the Bank will pay particular attention to fragile states, agriculture and food security, and gender.

1.5 Self-Assessment of the AfDB’s Independent Evaluation Function

In its capacity as the premier development-finance institution in Africa, the AfDB is constantly engaged in institutional reform to maximize its development effectiveness, as well as to increase the quality of its operations. To this end, following the commissioning of a self-assessment of its independent evaluation function in 2011, the AfDB revised its independent evaluation policy and developed a new, independent evaluation strategy.

The 2012 self-assessment of the AfDB’s evaluation function was conducted five years after the implementation of the independent evaluation policy. The primary objective of the self-assessment was to identify the strengths, as well as the limitations, of the AfDB evaluation function, and to recommend improvements. Importantly, the 2012 self-assessment preceded the current preliminary review. It is illustrative of an ongoing process of assessment and reform at the AfDB to improve on institutional, as well as development effectiveness. The findings of the current preliminary review are informed by and build upon the key findings of the self-assessment.

Findings of the 2012 self-assessment identified general support across the institution for the independent evaluation function, in light of the perceived value-added of the evaluation activity of the AfDB Operations Evaluation Department. There was, however, an equal perception of the need to realign the approach to evaluation with the strategic direction of the AfDB in order to determine what has worked well, and identify areas for improvement. It will be important for the Bank to generate evaluations with increased relevance to AfDB strategic decision-making processes and effectiveness, through the application of evaluation findings and lessons.

The findings of the self-assessment have informed the 2013-2017 strategy for independent evaluation. The revised policy and strategy provide the basis for the AfDB’s rolling work program for 2013-2015, including resource allocation, within its Operations Evaluation Department.

According to the Bank, "strengthening the evaluation function is at the heart of the 2013-17 Independent Evaluation Strategy… The (revised) strategy highlights a diverse range of actions designed to strengthen the quality of evaluations including preparation of an evaluation manual, establishing quality standards and a clearer quality assurance process. These are being implemented during the course of 2013 and beyond. It also proposes more explicit coverage of the themes outlined in the Bank’s Ten Year Strategy and has increased evaluations at country, sector and thematic level."^{Footnote 7}

2.0 Methodology

This section provides an overview of the methodology used to conduct this preliminary review, which involved three steps:

• Step 1: Establishing the evaluation universe in collaboration with the AfDB by identifying all program-level evaluations^{Footnote 8} and development effectiveness reports published by the AfDB between 2007 and 2012.
• Step 2: Conducting a preliminary review of AfDB program-level evaluations and development effectiveness reports through a two stage process entailing^{Footnote 9}:
  • Assessing whether a given document in the universe was an evaluation or development effectiveness report, as opposed to a policy review or a case study^{Footnote 10} and whether it fell within the time scope established for the review; and
  • Scoring the coverage and quality of evaluation reports using the DAC-EVALNET Evaluation Coverage and Quality Screening Guide.
• Step 3: Using the results of the preliminary review, triangulated with MOPAN assessment of the institution, to classify the AfDB independent evaluation function and its development effectiveness reporting under one of three scenarios:^{Footnote 11}
  • Reporting on development effectiveness is adequate (No further action required);
  • Reporting on development effectiveness is not adequate but the multilateral organization’s evaluation function provides an adequate information base on development effectiveness (Conduct Meta-synthesis of the organizations program-level evaluations); and
  • Reporting on development effectiveness is not adequate and the multilateral organisation’s evaluation function does not provide an adequate information base on development effectiveness (support improved evaluation and development effectiveness reporting).

The following section provides specific details on the implementation of the above-mentioned approach.

2.1 Rationale

The DAC-EVALNET methodology was designed to address a gap in information about the development effectiveness of multilateral organizations. It is modest in terms of time and resource requirements, and therefore minimizes the burden on multilateral organizations for its execution. The DAC-EVALNET methodological approach involved a review of program evaluations and development effectiveness reports produced by the AfDB.

The first stage of the review involved formulating the evaluation universe by identifying all program-level reports on AfDB development effectiveness and evaluations, which were produced between 2007 and 2012. Although the DAC-EVALNET approach does not specify a timeframe for the eligibility of reports for review, all reviews conducted to date have focused on reports published during the five-year period that preceded each review.

The evaluation universe was formulated in consultation with the AfDB to ensure that it included all program-level evaluations and development effectiveness reports produced by the AfDB over the period January 2007 – September 2012. Once the evaluation universe was agreed upon individual reports were reviewed using the criteria specified in the DAC-EVALNET methodology. Reports were screened in or out depending on the extent to which they conformed to these criteria. The overall result of this preliminary review then suggested which of the three scenarios outlined below was most appropriate for the AfDB^{Footnote 12}:

Scenario A – Reporting on development effectiveness is adequate (No Further Action, Option 1).

If the preliminary review indicates that the multilateral organisation’s reporting on the development effectiveness criteria is rigorous, evidence-based and covers a significant proportion of the multilateral organisation’s investments, then the reporting may be relied on, especially if based on evaluation data. This indicates the implementation of Option 1 – Rely on the multilateral organisation reporting systems.

Scenario B –Reporting on development effectiveness is not adequate but the multilateral organization’s evaluation function provides an adequate information base on development effectiveness (Conduct a meta-synthesis of the organization’s program-level evaluations, Option 2).

If MOPAN, the Development Assistance Committee, the United Nations Evaluation Group, or the Evaluation Cooperative Group peer review of the evaluation function indicates that the evaluation function is adequate, and the preliminary review confirms that the evaluation function provides a critical mass of credible information on the development effectiveness criteria, then implement Option 2. This option requires conducting a meta-synthesis of program-level evaluations to come to conclusions about development effectiveness.

Scenario C – Multilateral Organization reporting on development effectiveness is not adequate and the multilateral organisation’s evaluation function does not provide an adequate information base on development effectiveness (Support the organization’s development effectiveness reporting, including the monitoring and evaluation function, Option 3).

If MOPAN, the Development Assistance Committee, the United Nations Evaluation Group or the Evaluation Cooperative Group peer review of the evaluation function suggest that development effectiveness reporting may be inadequate, and the preliminary review confirms that the evaluation function does not provide a critical mass of credible information on the development effectiveness criteria, implement Option 3.

Option 3 requires the implementation of actions aimed at strengthening the multilateral organization’s development results reporting. Potential actions include direct support to multilateral organization results reporting and evaluation systems (i.e. peer review of the evaluation function), and/or support to strengthen the multilateral organization’s decentralized evaluation systems.

Following the preliminary review of program-level reports on development effectiveness, and a review of the MOPAN report, the AfDB was categorized under Scenario C, suggesting implementation of Option 3. (Subsections 2.2 to 2.5 explain the rationale and present the criteria used for document selection and review, as well as the limitations of the methodological approach.)

The rationale for suggesting that the AfDB follow scenario C rests on the two criteria outlined in the DAC-OECD methodology. First, the reports included in the universe did not represent adequate coverage of programmatic expected results during the period under review. Second, the MOPAN review found “major gaps” in the implementation of evaluations, although their rating was based on planned evaluations. Both this report and the MOPAN review indicate that insufficient “actual implemented” program-level evaluations were carried out by the AfDB during the period under review.

At the same time, project completion reports and the limited number of project evaluation reports play an important role in the Bank’s management processes. However, they do not provide the breadth of evaluative evidence necessary for a rigorous meta-synthesis of development effectiveness.

2.2 Scope

Document selection for establishing the AfDB evaluation universe was guided by the DAC-EVALNET methodological guidelines for the review, and was done in consultation with the Bank.

Given that the DAC-EVALNET approach is designed for high-level assessment of development effectiveness, project evaluations are not considered because of their limitations in formulating program-level findings for use at an institutional level. Therefore, this review drew on an agreed evaluation universe of 43 reports produced by the AfDB between January 2007 and September 2012 that included the following type of documents:

1. Country Assistance Evaluations;
2. Country Sector Reviews and Case Studies^{Footnote 13};
3. Process Reviews; and
4. Sector, Thematic and Policy Reviews.

Given the small size of this evaluation universe, (compared to that available in reviews of other organisations), it was decided to assess all reports in the universe, rather than selecting a more limited sample.

2.3 Criteria

Preliminary review of the agreed AfDB universe used the four DAC-EVALNET criteria for screening: type of document, scope, coverage, and quality. This was done in two stages:

1. Determining whether a given document was an evaluation report, as opposed to a policy review or a case study; and whether it fell within the time scope established for the review.
2. Rating the coverage and quality of evaluation reports using the DAC-EVALNET Evaluation Coverage and Quality Screening Guide (Appendix II).

The four criteria used in the preliminary review are described in Subsections 2.3.1 through 2.3.4. All four criteria were used to conduct the preliminary review.

2.3.1 Type of Document

Program evaluations and development effectiveness reports provide information on development results, and can be used to reflect the development effectiveness of AfDB programming. Documents that did not meet this criterion were screened out at this stage of the preliminary assessment.

2.3.2 Scope

Two scope criteria were used. First, a given evaluation report must be published between January 2007 and September 2012. Second, the programming period covered by a given evaluation report must end after 2005 (i.e. a report covering 1999-2004 and published in 2008 would be screened out). Documents that did not meet these criteria were screened out at this stage of the preliminary assessment.

2.3.3 Coverage

The coverage criterion was subdivided into three areas: financial; AfDB priority areas; and AfDB strategic objectives. This criterion was used to inform the review analysis as opposed to screen documents during the preliminary assessment.

The review of financial coverage was aimed at assessing the percentage of AfDB total expenditure that was covered by particular evaluations. However, this objective could not be achieved given the lack of precision in the financial data provided by several evaluations reviewed.

In reviewing coverage of AfDB priority areas and strategic objectives, the preliminary review aimed to identify AfDB priority areas and/ or strategic objectives that were identified in each document. This information was then used to determine whether there was coverage of the results of AfDB programming in the priority areas and/ or against the strategic objectives.

2.3.4 Document Quality

The preliminary review awarded points to each report using the quality screening criteria of the DAC-EVALNET Evaluation Coverage and Quality Screening Scoring Guide (Annex 2). Each report was required to achieve a minimum of 25 points, out of a maximum of 40 points, based on the statement of essential evaluation characteristics:

• Purpose;
• Evaluation;
• Organization;
• Subject evaluated is clearly described;
• Scope;
• Evaluation criteria;
• Multiple lines of evidence;
• Evaluation design;
• Evaluation findings and conclusions are relevant and evidence-based;
• Evaluation limitations; and
• Evaluation conclusions.

Three criteria are particularly relevant to reviews of development effectiveness: use of multiple lines of evidence; the evaluation design; and the development of relevant, evidence-based findings and conclusions. (Criteria G – I of the Evaluation Coverage and Quality Screening Scoring Guide – Appendix II). Documents were required to achieve a minimum of 10 points on these sub-components of quality screening.

2.4 Quality Assurance

The review adhered to established control systems in order to ensure quality, consistency and transparency during the application of screening procedure and criteria. The following controls were followed:

• A lead reviewer was selected from among the team of three reviewers to provide oversight during the application of the DAC-EVALNET methodology; and
• The lead reviewer developed and facilitated a training workshop^{Footnote 14} for the team to guide the preliminary review.

2.5 MOPAN Assessment of the AfDB Evaluation Function

MOPAN indicates that major areas of the AfDB portfolio are not adequately covered by evaluations, most significantly the rapidly growing area of private sector operations. Similarly, the number of Country Assistance Evaluations carried out annually by the Bank may be insufficient. In addition, the Bank has not been able to launch a programme of rigorous, well-resourced impact evaluations.

The AfDB’s Three-Year Rolling Work Program and Budget 2011-2013 states that although the evaluation function at the Bank has been strengthened with additional human resources in recent years, additional staff is required to effectively respond to the Bank’s evaluation needs. In fact, the evaluation function’s Budget and Work Program points to an “evaluation gap”.

Given that the MOPAN assessment of the AfDB suggests quality assurance mechanisms of the evaluation function of the Bank are inadequate, and that this review found that there is not a critical mass of program level evaluations, the AfDB’s evaluation function has been categorized under Scenario C.

2.6 Limitations

There was one key limitation to conducting a development effectiveness review. It pertained to the number of documents that were available to establish the AfDB evaluation universe. In contrast to previous reviews of development effectiveness conducted using this methodology, the AfDB evaluation universe of 43 documents was small.

For the review of the development effectiveness of the UNDP, a sample of 55 documents was selected from an evaluation universe of 199 documents. For the development effectiveness review of the World Food Programme, the evaluation universe comprised 52 documents. In order to mitigate the limitation of the small size of the AfDB’s evaluation universe, the preliminary review involved an assessment of all documents included in the evaluation universe.

This limitation became more acute following the selection and rating of documents using the DAC-EVALNET criteria. This process resulted in the reduction of the evaluation universe by 47 percent. Only 20 documents of the 43 included in the universe were retained following screening based on the criteria of type of document and scope.

However, 12 of the 20 documents that were retained in the final universe focused on policy and finance, and therefore provided partial coverage of the AfDB's priority areas (sectors). In addition, 5 of the 20 documents retained in the final universe provided partial coverage of the AfDB's work, as they were joint evaluations with other development agencies.

It was not possible to mitigate this limitation as documents that were not selected did not meet the DAC-EVALNET standards, notwithstanding their importance for the AfDB in terms of providing information to manage financial risks, maintaining effective financial oversight, and responding to the views of financial rating agencies.

3.0 Results of the Preliminary Review

This section presents the results of the preliminary review of program evaluations and development effectiveness reports published by the AfDB over the period January 2007 – September 2012. The review was based on an evaluation universe of 43 AfDB reports from four reporting categories, as identified in Subsection 2.2.

3.1 Key Findings

The results of the preliminary review of the AfDB’s program evaluations and development effectiveness reports have been summarized under six key findings in Subsections 3.1.1 and 3.1.2.

3.1.1. Coverage

The AfDB evaluation function has not produced an adequate number of program-level evaluations and development effectiveness reports over the period 2007-2012.

Upon implementing the DAC-EVALNET preliminary review criteria to the evaluation universe, only 20 documents out of the aggregate 43 documents comprising the evaluation universe met all the review criteria.

The remaining 23 reports were screened out by the preliminary review for three main reasons. The first pertained to the scope of individual reports. Although all 43 documents in the universe were published during the review timeframe of January 2007 – September 2012, the scope of 6 documents (14 percent of the evaluation universe) was outside the selected review timeframe. These documents focused on a previous reporting period, for example, 1996-2004.

The second reason was that thirteen documents (30 percent of the evaluation universe) were neither program evaluations nor development effectiveness reports.

The third reason pertained to the quality of the document. Four documents (9 percent of the evaluation universe) were screened out for reasons that included the lack of a description of the evaluation’s stated objectives, omission of limitations, and/or a poor evaluation design.

Figure 2 provides an overview of documents that were screened out by the preliminary review, by type of report and/ or the reason for rejection.

Figure 2: Screened Out Documents by Type of Document/ Reason (Actual values, n=23)

Overall, less than half of the documents listed in the evaluation universe met all the DACEVALNET screening criteria. This is illustrated in Figure 3.

Figure 3: Selected versus Screened out Reports (%, n=43)^{Footnote 15}

Review findings show that the AfDB’s evaluation function has not produced a critical mass of information on AfDB development effectiveness over the review period of January 2007 – September 2012. Of the 20 documents that met the minimum criteria for the preliminary review, there was an overwhelming focus on policy-based, and financial management evaluations (55 percent of all documents in the evaluation universe).

Very few documents (three) examined the changes observed in regional member countries arising from AfDB financial assistance. Two reports (10 percent of all selected reports) focused on AfDB country assistance, and one report (5 percent of all selected reports) explored AfDB financial support to a given sector (see Figure 4). As a result, the documents that were selected did not provide an adequate representation of progress made by the AfDB in meeting its planned development outputs and outcomes.

Figure 4: Selected Documents by Area of Focus (%, n=20)^{Footnote 16}

This finding leads to categorizing the AfDB’s program-level evaluations and development effectiveness reports under Scenario C – Option 3. The review methodology suggests that the multilateral organization’s reporting on development effectiveness is not adequate, and the evaluation function does not provide an adequate information base on development effectiveness. It further suggests that the multilateral organization requires support for its development effectiveness reporting, including its evaluation function.

The review notes that the AfDB has taken steps to strengthen its evaluation function through the 2012 self-assessment of its independent evaluation function, and the resulting revision of its independent evaluation policy and strategy. The revised evaluation policy has outlined the AfDB’s commitment to improving its development effectiveness by focusing on three complementary objectives for the strengthening of its independent evaluation function, specifically: learning; accountability and the promotion of an evaluation culture across the AfDB. This review anticipates that the strengthening of the AfDB evaluation function will include the strengthening of AfDB program-level results reporting.

Program-level evaluations and development effectiveness reports within the review period (2007-2012) provided inadequate coverage of the results of AfDB programming in its priority areas and its strategic objectives.

The 20 documents that met all the preliminary review criteria together addressed all but two of the AfDB’s priority areas. In addition, while one report had a multisector focus, two reports examined issues pertaining to social and human capital, which have an implicit link to the AfDB commitment to provide selective support to health-related millennium development goals (MDGs). All 20 reports addressed the AfDB’s strategic objectives. These findings are illustrated by Figures 5 and 6.

Figure 5: AfDB Priority Areas Addressed by Reports (Actual values, n=20)^{Footnote 17}

Figure 6: AfDB Strategic Objectives Addressed by Reports (%, n=20)^{Footnote 18}

The preliminary review suggests, however, that AfDB reporting on its priority areas and strategic objectives is largely descriptive, involving an overview of AfDB programmatic activity. Significantly, the reporting provides inadequate coverage of results achieved through AfDB programming in its priority areas and under its strategic objectives. Using the DAC-EVALNET guidelines, the review indicates that the AfDB’s reports do not provide outcomes-based information on AfDB programming activity. This implies a need for the AfDB to revisit its approach to program-level evaluation reporting, as well as review its capacity to report on its development results.

AfDB program-level evaluations and development effectiveness reports provide insufficient coverage of AfDB investment in its crosscutting thematic priorities.

In its Medium-Term Strategy for 2008-2012, the AfDB committed itself to mainstreaming three thematic areas: Gender; the Environment; and Climate Change. Preliminary review findings indicate, however, relatively little evidence of the exploration of these crosscutting themes in AfDB development effectiveness reporting. Of the 20 reports that met all the preliminary review criteria, five reports (20 percent) addressed the crosscutting thematic priorities. One report included discussion around the gender thematic, and three reports addressed the environment. An additional report explored the theme of climate change. This finding is a reflection of insufficient coverage by the AfDB’s development effectiveness reporting of investment made in its priority crosscutting themes. This is crucial, as crosscutting priorities are core areas of AfDB investment requiring financial support to regional member countries. There is an equivalent need for assessment through program evaluations and/or development reports to determine the extent to which planned development outcomes are achieved in these areas.

This assessment notes that the AfDB produced a major internal report on gender in 2012, and has put new rules in place for gender mainstreaming. The AfDB also plans to conduct a review of gender mainstreaming in 2014. In the meantime, the AfDB has conducted a synthesis exercise to learn lessons on gender mainstreaming as practised by other organisations. At the time of the assessment, the AfDB submitted a report on environmental mainstreaming within the transport sector and engaged in a joint evaluation of the Climate Investment Funds (CIF).

The usefulness of AfDB development effectiveness reporting to the Bank’s forward-looking investments is sometimes impeded by the time lag between the year an evaluation is conducted and the year it is finalized and published.

All 43 documents included in the preliminary review universe were published during the established timeframe of January 2007 to September 2012. However, 8 reports had a significant lag between their publication and the years they reviewed. For example, a report published in 2007 evaluated programming during 1996-2004, thus implying a three year lag between implementation and publication. Because context and priorities change, AfDB evaluations with significant time lags between publication and the years reviewed have reduced usefulness for new program planning. This finding has been corroborated by the 2012 self-assessment of the AfDB’s independent evaluation function, which raised concerns about the timeliness of many AfDB evaluation reports. That said, there is value in the feedback provided to program managers as part of an evaluation process and prior to report publication.

Section Summary:

The findings from the preliminary review of AfDB program-level development effectiveness reports and evaluations indicate insufficient evidence to illustrate the AfDB’s development effectiveness during period 2007 to 2012.

Following the formulation of the AfDB evaluation universe of 43 program evaluations and development effectiveness reports, more than 50 percent of the reports did not meet the minimum standards established by the DAC-EVALNET guidelines.

Overall, the evaluation universe did not provide a critical mass of credible information on the AfDB’s development effectiveness. It further did not provide adequate coverage of crosscutting priorities identified by the AfDB in its 2008-2009 Medium-Term Strategy. There is an additional suggestion that development reports have decreased usefulness for AfDB planning and future investments because of time lags between evaluation activity and the publication of the evaluation report. Importantly, however, the AfDB has documented a commitment to strengthening its independent evaluation function. The review anticipates that this will lead to a strengthening of AfDB program-level results reporting.

3.1.2. Quality

Some evaluation designs in the program level evaluation reports reviewed did not follow the DAC-EVALNET quality guidelines. This is a potential area for improvement.

The preliminary review awarded points to each report using the quality screening criteria of the DAC-EVALNET Evaluation Coverage and Quality Screening Scoring Guide to determine evidence of two aspects of quality: (i) basic evaluation report characteristics; and (ii) the existence of key components required for a review of development effectiveness.

Preliminary review findings indicated that all 20 program-level evaluation reports that were retained in the universe surpassed the threshold of 25 points required to demonstrate having basic evaluation report characteristics. The score range was 27 – 36 points out of a maximum of 40 points, equivalent to a percentage score of 68 – 90 percent.

In terms of the existence of report characteristics that are required for a review of development effectiveness, documents received high scores on two of the three screening criteria: ‘Multiple lines of evidence’; and ‘Evaluation findings and conclusions are relevant and evidence-based’. Documents received 3 or 4 points out of a total of 4 points for each criterion.

Points attained for ‘Evaluations are well-designed’ were significantly lower, and ranged from one to five, out of a maximum of five attainable points. Of the 20 documents reviewed, only one received the maximum number of points attainable for a well-designed evaluation. These scores echo the 2012 self-assessment of the AfDB’s independent evaluation function, which stated that the AfDB employed a narrow range of evaluation methodologies in its project-level evaluations.

This finding provides evidence of AfDB attention to the quality of its development effectiveness reporting, as illustrated by the moderate to high scores awarded to reports that were screened for quality. There is an equally strong indication, however, of the need for a strengthening of the evaluation design process.

A reference in this regard would be the "DAC Guidelines and Reference Series: Quality Standards for Development Evaluation". According to the DAC guidelines, the core elements of a good evaluation design that are to be used to produce well-articulated evaluations include:

• An explicit theory for the achievement of objectives and results;
• Specification of the level of results achieved, with emphasis on outputs, outcomes, and impact;
• Baseline data (quantitative or qualitative) on conditions that existed prior to program implementation;
• Comparison of conditions after program delivery to those that existed before program implementation.

4.0 Conclusions

The purpose of the preliminary review was to provide an independent, evidence-based assessment of the development effectiveness of the AfDB, for use by the Government of Canada, the AfDB and other interested stakeholders. It was based on a universe of 43 AfDB program-level reports, comprising program-level evaluations and development effectiveness reports that were produced over the review period of January 2007 – September 2012.

The preliminary review of the AfDB’s program-level development effectiveness and evaluation reports involved the application of the DAC-EVALNET criteria to assess quality and coverage. The findings indicate that the evaluations included in the review do not provide an adequate level of coverage of the AfDB’s development results.

This leads to the following conclusions:

1. The application of the DAC-EVALNET criteria to the preliminary review of the AfDB evaluation universe has shown that the AfDB has not produced a critical mass of information for the assessment of development effectiveness. Through the provision of forward-looking recommendations, this review can support the AfDB in improving evaluation and development effectiveness reporting in the future.
2. Overall, AfDB coverage of its priority areas and strategic objectives in program-level reporting is inadequate. This is seen in its descriptive accounts of programmatic activity and a lack of details on development results. As well, there is a dearth of evidence on its priority crosscutting themes in program evaluations and development effectiveness reports. Further work is required to provide adequate outcome-based reporting for the Bank’s priority areas, strategic objectives, and crosscutting themes.
3. The utility of some of the AfDB evaluations included in the universe of this review has been decreased by the length of time that elapsed between the evaluation activity and the year of report publication. The time lag has implied a decreasing usefulness of AfDB’s evaluations, given internal institutional changes, as well as contextual developments at the level of regional member countries. This leads to the conclusion that some of the AfDB’s evaluations included in the universe of this review are limited in their relevance to AfDB strategic decision-making processes.
4. AfDB program-level evaluations have been hindered by the narrow range of methodologies employed. The use of methodologies that are not conducive to program-level evaluations provide an unreliable overview of progress made by the AfDB towards its planned development results. While the AfDB has met the DAC-EVALNET standards of quality for the production of evaluation reports, this has not been matched by well-articulated evaluation designs. This implies that there is need to strengthen AfDB capacity for designing its program-level evaluations.

5.0 Recommendations

These recommendations have been developed to assist the AfDB in developing a critical mass of program-level development effectiveness reports and evaluations, which would in future be amenable to the application of the DAC-EVALNET methodology for meta-synthesis of evaluation results. Recommendations are also provided to support CIDA’s future engagement with the AfDB, with emphasis on the strengthening of the AfDB’s evaluation function.

The recommendations build upon the results of this review, the MOPAN assessment of the Bank, the 2012 self-assessment of the AfDB’s evaluation function, and the AfDB 2013-2017 independent evaluation strategy. The latter has outlined the AfDB’s commitment to ‘raising the bar on quality’, to ensure methodological rigour, as well as improving the quality assurance process. Similarly, the need for a periodic peer review of the AfDB’s independent evaluation function was emphasized by the 2012 self-assessment, which recommended that the AfDB should consider engaging in a peer review once every 5 years.

5.1 Recommendations for the AfDB

Strengthening its independent evaluation function should be a priority for the AfDB.

A stronger evaluation function would enable the AfDB to accurately assess progress towards the achievement of program-level development outcomes, and capture lessons learned to improve its investments in regional member countries.

Based on the findings and conclusions of the preliminary review, the AfDB’s evaluation function has not produced the critical mass of reports required to facilitate a review of AfDB development effectiveness. The areas of AfDB evaluation activity in which the review findings have noted particular weaknesses were the design of evaluations and coverage of program-level development effectiveness results. The strengthening of the evaluation function will allow the AfDB to effectively address each of these areas. The anticipated results of this process are increased robustness in evaluation design through the use of appropriate evaluation methodologies, and more comprehensive reporting on development effectiveness.

The AfDB should consider engaging in a peer review of its evaluation function.

In order to improve its evaluations and maintain the DAC-EVALNET standards for conducting evaluations and reporting on development effectiveness, the AfDB should engage in a peer review of its evaluation function. A peer review is markedly different from a self-assessment. It would be of use to the AfDB for assessing the capacity and quality of its evaluation function. Findings from the peer review can be further used by the AfDB to assess the relevance, comprehensiveness, and overall quality of its development effectiveness reporting. The results of this assessment would be of value to the AfDB in determining the extent to which its program evaluations and development effectiveness reports can be used to inform AfDB strategic decision-making.

In addition to benefiting from advice for the strengthening of its evaluation function, the AfDB can use findings from the peer review to develop best practices for dissemination and learning. This is in accordance with the AfDB’s commitment to ‘using knowledge management’^{Footnote 19} as outlined in its independent evaluation strategy for 2013-2017.

5.2 Recommendations for DFATD

Canada, in concert with other donors and AfDB Board members, should support the Bank’s efforts to strengthen its evaluation function.

Canada should support the AfDB’s efforts to strengthen its evaluation function and the design of its evaluations, as outlined above. One possibility, among others, would be an annual analysis of the AfDB’s evaluation coverage to be included as part of the AfDB’s evaluation work plan. For example, the AfDB’s rolling work plan could provide some measure of the target for, and actual implementation of, program level evaluation, perhaps as a percentage of total expenditure.

In working with other Board members and donors, Canada should consider among others supporting the strengthening of the AfDB’s evaluation function. Important elements would include: peer review; capacity development to strengthen evaluation coverage, design, and timeliness; and strengthened program-level development effectiveness reporting.

Annex 1: Evaluation Universe

No 1

• Year: 2012
• Programming Unit: High Level Evaluations Division
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Fostering Regional Integration in Africa: An Evaluation of the Bank’sMultinational Operations, 2000-2010; Synthesis report
• Pages: 52
• Hyperlink: 2000-2010: Fostering Regional Integration in Africa – An Evaluation of the Bank’s Multinational Operations: Approach Paper : Portfolio Review Paper
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 2

• Hyperlink: 2000-2010: Fostering Regional Integration in Africa – An Evaluation of the Bank’s Multinational Operations: Approach Paper
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is an Approach Paper.

No 3

• Hyperlink: 2000-2010: Policy and strategy Review Paper
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is a Policy and Strategy Review Paper.

No 4

• Hyperlink: 2000-2010: Case study 1: Arusha Namanga Athi River Road Development Project
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is a 1-page Case Study.

No 5

• Hyperlink: 2000-2010: Case study 2. Manantali Energy Project (Mauritania, Mali & Senegal)
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is a 1-page Case Study.

No 6

• Hyperlink: 2000-2010: Case study 3: Sasol Gas Project
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is a 1-page Case Study.

No 7

• Year: 2012
• Programming Unit: Operations Evaluation Department
• Type: Corporate Evaluations
• Evaluation Title: Evaluation of the Assistance of the African Development Bank to Fragile States
• Pages: 44
• Hyperlink: Evaluation of the Assistance of the African Development Bank to Fragile States
• Results of Preliminary Review: Screened out: This is a summary report, referring to case studies and other data collection tools.

No 8

• Year: 2012
• Programming Unit: Collaborative effort between AfDB, Sida and Danida,
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Joint Evaluation of Public Financial Management Reform : Burkina Faso, Ghana and Malawi 2001–2010; Final Synthesis Report
• Pages: 126
• Hyperlink: Joint Evaluation of Public Financial Management Reform : Burkina Faso, Ghana and Malawi 2001–2010; Final Synthesis Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 9

• Year: 2012
• Programming Unit: Collaborative effort between AfDB, Sida and Danida,
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Public Financial Management Reform in Malawi : Final Country Case Study Report
• Pages: 149
• Hyperlink: Evaluation of Public Financial Management Reform in Malawi : Final Country Case Study Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 10

• Year: 2012
• Programming Unit: Collaborative effort between AfDB, Sida and Danida,
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Public Financial Management Reform in Ghana : Final Country Case Study Report
• Pages: 153
• Hyperlink: Evaluation of Public Financial Management Reform in Ghana : Final Country Case Study Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 11

• Year: 2012
• Programming Unit: Collaborative effort between AfDB, Sida and Danida,
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Public Financial Management Reform in Burkina Faso : Final Country Case Study Report
• Pages: 182
• Hyperlink: Evaluation of Public Financial Management Reform in Burkina Faso : Final Country Case Study Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 12

• Year: 2012
• Programming Unit: Operations Evaluation Department
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Mainstreaming Gender Equality: A Road to Results or a Road to Nowhere? An Evaluation Synthesis
• Pages: 52
• Hyperlink: Mainstreaming Gender Equality: A Road to Results or a Road to Nowhere? An Evaluation Synthesis
• Results of Preliminary Review: Screened out: This report is not an evaluation. It is a review of best practices.

No 13

• Year: 2011
• Programming Unit: Bank-wide Task Force
• Type: Corporate Evaluations
• Evaluation Title: Mid-Term Review of the Bank’s Medium Term Strategy for 2008 - 2012
• Pages: 31
• Hyperlink: Mid-Term Review of the Bank’s Medium Term Strategy for 2008 - 2012
• Results of Preliminary Review: Screened out: This report is a review of the AfDB’s own strategy.

No 14

• Year: 2011
• Programming Unit: Operations Evaluation Department
• Type: Corporate Evaluations
• Evaluation Title: Evaluation of the Paris Declaration Implementation at the African Development Bank
• Pages: 39
• Hyperlink: Evaluation of Paris Declaration Implementation at the African Development Bank
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 15

• Year: 2011
• Programming Unit: Operations Evaluation Department
• Type: Country Assistance Evaluation
• Evaluation Title: Gabon – Country Assistance Evaluation 1996 - 2008
• Pages: 72
• Hyperlink: Gabon - Country Assistance Evaluation 1996-2008
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 16

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Tanzania
• Pages: 82
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999-2009 - Country Case Study: Tanzania
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 17

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Sierra Leone
• Pages: 71
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999-2009 Country Case Study: Sierra Leone
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 18

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Morocco
• Pages: 90
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999–2009 - Country Case Study: Morocco
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 19

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Rwanda
• Pages: 70
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999-2009 - Country Case Study: Rwanda
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 20

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Ethiopia
• Pages: 102
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999–2009 - Country Case-Study: Ethiopia
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 21

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Burkina Faso
• Pages: 89
• Hyperlink: Evaluation of Policy Based Operations in the African Development Bank, 1999-2009 - Country Case Study: Burkina Faso
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 22

• Year: 2011
• Programming Unit: Operations Evaluation Department
• Type: Country Sector Reviews and Case Studies
• Evaluation Title: Ghana – Mali – An Evaluation of the AfDB Assistance – Agricultural Water Management 1990 - 2010
• Pages: 48
• Hyperlink: Ghana-Mali - An Evaluation of the AfDB Assistance - Agricultural Water Management 1990-2010
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 23

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 – Institutional and Policy Review
• Pages: 89
• Hyperlink: Evaluation of Policy Based operations in the African Development Bank, 1999-2009 Institutional and Policy Review
• Results of Preliminary Review: Screened out: This report is a review of AfDB operations.

No 24

• Year: 2011
• Programming Unit: High Level Evaluations Division
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Evaluation of Policy Based Operations in the AfDB (1999 – 2009)
• Pages: 45
• Hyperlink: Evaluation of Policy Based Operations in the AfDB (1999-2009)
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 25

• Year: 2010
• Programming Unit: High Level Evaluations Division
• Type: Country Assistance Evaluation
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Egypt Financial Sector Reform Programme
• Pages: 16
• Hyperlink: Evaluation of Policy Based Lending in the African Development Bank, 1999-2009 - Case Study: Egypt Financial Sector Reform Programme
• Results of Preliminary Review: Screened out: This report is a case study of a larger evaluation.

No 26

• Year: 2010
• Programming Unit: High Level Evaluations Division
• Type: Country Assistance Evaluation
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 Country Case Study: Botswana Economic Diversification Support Loan
• Pages: 18
• Hyperlink: Evaluation of Policy Based Lending in the African Development Bank, 1999-2009 - Case Study: Botswana Economic Diversification Support Loan
• Results of Preliminary Review: Screened out: This report is a case study of a larger evaluation.

No 27

• Year: 2010
• Programming Unit: Collaborative effort between AfDB, Danida, DFID and Sida, and conducted by the Overseas Development Institute (ODI)
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Evaluation of Donor Support to Public Financial Management Reform in Developing Countries – Analytical study of quantitative cross-country evidence – Final Report
• Pages: 84
• Hyperlink: Evaluation of Donor Support to Public Financial Management (PFM) Reform in Developing Countries - Analytical study of quantitative cross-country evidence - Final Report
• Results of Preliminary Review: Screened out: This report is a joint evaluation that did not assess the work of the AfDB.

No 28

• Year: 2010
• Programming Unit: Operations Evaluation Department
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Evaluation of Policy Based Operations in the African Development Bank, 1999 – 2009 – Review of Literature and Experience
• Pages: 18
• Hyperlink: Evaluation of Policy Based operations in the African Development Bank, 1999-2009 - Review of Literature and Experience
• Results of Preliminary Review: Screened out: This report is a literature review.

No 29

• Year: 2010
• Programming Unit: Operations Evaluation Department
• Type: Evaluation Report Summary
• Evaluation Title: Independent Assessment of the Quality at entry of ADF 2005 – 2008: Operations and Strategies – Final Summary Report
• Pages: 49
• Hyperlink: Independent Assessment of the Quality at entry of ADF 2005-2008 Operations and Strategies - Final Summary Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 30

• Year: 2010
• Programming Unit: Operations Evaluation Department (in collaboration with the International Fund for Agricultural Development)
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Joint Evaluation of the Agriculture and Rural Development Policies and Operations in Africa of the African Development Bank and the International Fund for Agricultural Development – Final Report
• Pages: 128
• Hyperlink: Joint Evaluation of the Agriculture and Rural Development Policies and Operations in Africa of the African Development Bank and the International Fund for Agricultural Development - Final Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 31

• Year: 2010
• Programming Unit: Operations Evaluation Department
• Type: Corporate Evaluation
• Evaluation Title: Project Supervision at the African Development Bank 2001-2008 – An Independent Evaluation – Final Report
• Pages: 52
• Hyperlink: Project Supervision at the African Development Bank 2001-2008 – An Independent Evaluation – Final Report
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 32

• Year: 2009
• Programming Unit: Independent Evaluation Group
• Type: Country Assistance Evaluation
• Evaluation Title: 2001-2007 – Uganda – Country Assistance Evaluation
• Pages: 174
• Hyperlink: 2001-2007 - Uganda - Country Assistance Evaluation
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 33

• Year: 2009
• Programming Unit: Operations Evaluation Department
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Independent Evaluation of the Decentralisation Strategy and Process at the African Development Bank
• Pages: 52
• Hyperlink: Independent Evaluation of the Decentralisation Strategy and Process at the African Development Bank
• Results of Preliminary Review: Accepted: This report met all 4 DAC-EVALNET screening criteria

No 34

• Year: 2009
• Programming Unit: Human Development Department
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Assessment of the African Development Bank’s Assistance to the Prevention and Control of HIV-AIDS 1997-2006
• Pages: 26
• Hyperlink: Health - Assessment of the African Development Bank's Assist to the Prevention and Control of HIV-AIDS 1997-2006
• Results of Preliminary Review: Screened out: This report was low on quality as regards evaluation design.

No 35

• Year: 2009
• Programming Unit: Operations Evaluation Department
• Type: Sector, Thematic and Policy Reviews
• Evaluation Title: Joint Evaluation of the Agriculture and Rural Development Policies and Operations in Africa of the African Development Bank and the International Fund for Agricultural Development – Summary Report
• Pages: 43
• Hyperlink: Joint Evaluation of the Agriculture and Rural Development Policies and Operations in Africa of the African Development Bank and the International Fund for Agricultural Development - Summary Report
• Results of Preliminary Review: Screened out: This report was low on quality as regards evaluation design.

No 36

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country Assistance Evaluation
• Evaluation Title: Ethiopia – Country Assistance Evaluation 1996 - 2007
• Pages: 90
• Hyperlink: Ethiopia – Country Assistance Evaluation 1996 - 2007
• Results of Preliminary Review: Screened out: This report was low on quality as regards evaluation design.

No 37

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Corporate Evaluation
• Evaluation Title: Evaluation of the Joint Africa Institute
• Pages: 59
• Hyperlink: Evaluation of the Joint Africa Institute
• Results of Preliminary Review: Screened out: This report was low on quality, as regards evaluation design.

No 38

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Ethiopia - Country Assistance Evaluation - Multisecteur 1992-2004
• Pages: 43
• Hyperlink: Ethiopia - Review of Bank Group Assistance to the Multi-Sector
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

No 39

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Ethiopia - Country Assistance Evaluation - Agricultural Sector and Rural Development 1993-2004
• Pages: 51
• Hyperlink: Ethiopia - Review of Bank Group Assistance to the Agriculture and Rural Development Sector
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

No 40

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Cameroon - Country Assistance Evaluation - social 1996 - 2004
• Pages: 55
• Hyperlink: Cameroon - Evaluation of Bank Group Assistance to the Social Sector 1996-2004
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

No 41

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Cameroon - Country Assistance Evaluation - Transport 1996 - 2004
• Pages: 54
• Hyperlink: Cameroon - Evaluation of Bank Group Assistance to the Transport 1996-2004
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

No 42

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Cameroon - Country Assistance Evaluation - Agriculture and Rural Development 1996 - 2004
• Pages: 75
• Hyperlink: Cameroon - Evaluation of Bank Group Assistance to the Agriculture and Rural Development Sector 1996-2004
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

No 43

• Year: 2008
• Programming Unit: Operations Evaluation Department
• Type: Country sector Review and Case study
• Evaluation Title: Cameroon - Country Assistance Evaluation - Multisector 1996 - 2004
• Pages: 50
• Hyperlink: Cameroon - Review of the Bank Assistance to the Multisector, 1996 – 2004
• Results of Preliminary Review: Screened out: This report was external to the evaluation (assessment) scope.

Annex 2: Evaluation Coverage and Quality Screening Scoring Guide

Evaluation Title:

Reviewer:

Coverage¹:

Financial²:

Comments

Annex 3: Corporate Documents Reviewed

African Development Bank Group. (October 2012). Independent Evaluation Strategy 2013 – 2017. Tunis: African Development Bank Group – Operations Evaluation Department.

African Development Bank Group. (n.d.). Medium-Term Strategy 2008-2012. Tunis: African Development Bank Group

African Development Fund. (September 2012). Overview Note: ADF-12 Mid-Term Review, September 2012, Praia, Cape Verde.

Canadian International Development Agency. (April 2012). Development Effectiveness Review of the United Nations Development Programme (UNDP) – 2005 – 2011: Synthesis Report. Gatineau: CIDA

Multilateral Organisation Performance Assessment Network (2012). Assessment of Organisational Effectiveness and Reporting on Development Results. African Development Bank (AfDB). Volume I and II, December 2012.

OECD-DAC Network on Development Evaluation. (June 2012). Assessing the Development Effectiveness of Multilateral Organizations: Guidance on the Methodological Approach. Paris: OECD-DAC Network on Development Evaluation.

PERRIN, Burt. (July 2012). Self-Assessment of the African Development Bank Group Independent Evaluation Function: Final Report. African Development Fund. ADF/ BD/ WP/ 2012/ 84/ Add.1

Annex 4: African Development Bank – Comments on the Review of AfDB Program Evaluation Reporting

The following comments are provided by the Independent Operations Evaluation Department of the African Development Bank. August 2013.

The Independent Operations Evaluation Department (OPEV) of the African Development Bank (AfDB or Bank) welcomes CIDA’s Preliminary Review of AfDB’s Development Effectiveness Reporting. As a development institution the AfDB is committed to continuously improving its development effectiveness and welcomes all external assessments as useful means to understand better areas where it could further improve performance.

The AfDB has recognised in its Strategy 2013-2022 the important role evaluation plays for improving its operational performance and committed to strengthen this function. The implementation of OPEV’s new Strategy has already produced visible initial results in 2013 in terms of quality and coverage of evaluations, but any ambitious change programme takes time before it yields full results. Against this background, OPEV finds in CIDA’s review elements of confirmation of its own diagnosis that led to the design and implementation of its new Strategy. We feel, however, its overall conclusion fails to capture the dynamics already in place to address existing challenges, including broader reforms at Bank level.

OPEV’s comments are organised under three main categories: (i) the current context for independent evaluation in the AfDB; (ii) the scope and methodology of the CIDA review; and (iii) the recommendations.

1. The current context for independent evaluation in the AfDB

CIDA’s review was conducted after a detailed “self-assessment” of the AfDB’s independent evaluation function. The 2012 self-assessment was followed by a new Independent Evaluation Policy and separate Strategy for Independent Evaluation (2013-17). This strategic direction has in turn has been reflected in OPEV’s three-year rolling work program. Thus, the past year marks a step change in independent evaluation at the AfDB. Implementation of the new Strategy is now underway and is designed to help OPEV to address key challenges identified in the self-assessment. Features most relevant to the CIDA review include:

• A shift in the types of evaluations conducted. As part of a wider rebalancing of learning and accountability functions of independent evaluation, OPEV plans to reduce the number of free-standing project level evaluations, in favour or more thematic, sector and cluster evaluations. An increase in the number of country strategy evaluations is already underway in 2013. These will be an important tool in capturing information regarding development effectiveness at country level. Nevertheless, evaluations of individual projects will remain important building blocks, just as projects are the main building blocks of what the Bank does. In addition, OPEV intends to continue to prepare institutional or corporate evaluations, as there is need and demand for them. In terms of the subject matter covered, the Strategy commits OPEV to a more explicit alignment with the Bank’s strategic priorities under the new Ten Year Strategy. This has already been followed through in the 2013-15 rolling work program, approved by the Committee on Development Effectiveness in 2012.
• An increased emphasis on quality and timeliness. The Strategy emphasises the importance of evaluation quality and timeliness to ensure evaluation products are useful and have an impact. To ensure all evaluations meet the same quality standards, OPEV is undertaking several steps; including preparation of an evaluation manual, which will include good practices and clear quality standards for key stages of the evaluation process. The quality standards will help OPEV to continue to make good use of multiple methods and data sources while also ensuring better articulation of the overall evaluation design. The timeliness of evaluations is improving through consultative planning and more robust management of evaluation processes.
• A mission focused on development effectiveness. In the new Policy and Strategy the department’s mission is explicitly focused on supporting the Bank’s development effectiveness. It sets out how evaluations which support accountability, learning and evaluation culture will all support that overall aim.
• Evaluations that have influence. The Strategy emphasises the importance of knowledge management and the follow-up of evaluation recommendations – through a management action reporting mechanism (MARM), which is currently under development.

As well as a step-change in independent evaluation, the Bank more broadly has redoubled its efforts to improve its development results reporting. Annual development effectiveness reviews, at continental, sector and recently country level are also produced, although these were not included in CIDA’s documentary review. The Bank’s corporate results framework is also being revised to align with the new Strategy 2013-2022.

These transitions are underway. As always the AfDB welcomes the support and engagement of development partners such as CIDA to strengthen further its ability to report on development effectiveness, either through independent evaluation or otherwise.

2. Scope and methodology

Overall, OPEV agrees that independent evaluation should make a stronger contribution to reporting on development results. Nevertheless, some issues regarding the scope and application of the methodology in the CIDA review are worth highlighting since they may affect the overall conclusions of the review. These issues were raised with CIDA when a first draft of the report was shared with OPEV, but still remain concerns in the final draft. In particular:

• A large share of evaluations produced by OPEV was not included. In particular, project level evaluations were excluded.^{Footnote 20} However, project level evaluations have been a crucial part of the work of the independent evaluation function over the period under review and they look closely at results.^{Footnote 21} The 2012 DAC methodological guidance for these multilateral reviews does not stipulate that project level evaluations should be excluded and in some other reviews the project level evaluations were included.^{Footnote 22} Given the Bank is a project based organization and given that project evaluations were a major component of evaluation work conducted during the period under review, additional relevant information about development results may have been missed.
• In addition, some important program level evaluations have been screened out. For example, the evaluation of the Bank’s assistance to Fragile States was not included on the grounds that the reviewer thought it a “synthesis” of other evaluations. In fact it is a summary report, which brings together different components, such as case studies, literature review and so on. Most OPEV reports are such summaries, with supporting documentation available from component methods.^{Footnote 23} It is also important to note that the DAC guidelines were adjusted in 2012 to add a second minimum benchmark. Using the original benchmark only, as in the case of most of the other reviews, all of the OPEV evaluations that were accepted for screening easily surpassed the benchmark. The majority also met the additional new benchmark.^{Footnote 24} Finally, it is important to understand that background documents, such as approach papers, although important to review as part of the background or design of an evaluation, will not meet review criteria given their nature. In conclusion and going forward, we suggest more careful application of the screening process in future reviews.
• Finally, it should be noted that the review is focused only on evaluations published by OPEV. Other reports on development results, such as the annual development effectiveness review (under the responsibility of the Quality Assurance and Results Department) have been important products for the Bank in the last few years. These reports are not included. The 2012 OECD DAC methodological guidance for these reviews indicates that independent program evaluation reports need not be the only source of information.

3. Recommendations

The Bank welcomes the support of all development partners for strengthening its independent evaluation function. In late 2012 the new evaluation Policy and Strategy were discussed by the Board’s Committee on Development Effectiveness, to which OPEV reports. The Strategy in particular sets out how the independent evaluation function of the Bank can be strengthened; it has been widely endorsed and in 2013 was formally approved by the Bank’s Board.

Recommendation to the Bank: Strengthening its independent evaluation function should be a priority for AfDB.

Response: Strengthening the evaluation function is at the heart of the Independent Evaluation Strategy 2013-2017. OPEV voluntarily launched a detailed self-assessment to diagnose weaknesses and identify strengths and opportunities. On this base of solid evidence OPEV revised its Independent Evaluation Policy and prepared a first Independent Evaluation Strategy. The Strategy highlights actions to strengthen the quality of evaluations including preparation of an evaluation manual, establishing quality standards and a more systematic quality assurance process. These are being implemented during the course of 2013 and beyond. It also proposes more explicit coverage of the themes outlined in the Bank’s Ten Year Strategy and has increased evaluations at country, sector and thematic level. Support from Canada to the implementation of OPEV’s Strategy is most welcome.

Recommendation to the Bank: The AfDB should consider engaging a peer review of its evaluation function.

Response: The peer review process is already planned to take place within the next two years. It is expected that the peer review will form the basis for the next set of reforms for OPEV.

Recommendation to CIDA: CIDA, in concert with other donors and AfDB Board members, should support the Bank’s efforts to strengthen its evaluation function.

Response: Support is most effective when aligned with the Bank’s own Strategy and Policy for independent evaluation. Discussion on entry points for CIDA is very welcome; discussions with other interested partners are on-going. Regarding the specific suggestion to assess the coverage of OPEV reports, coverage is looked at through the process of developing the rolling work program; which involves consultation within the Bank and with its stakeholders, including Board members. Canada’s involvement in this process through its Board membership is most welcome.

Annex 5: Management Response

The Preliminary Review of the African Development Bank (AfDB) Development Effectiveness Reporting prepared by CIDA’s Evaluation Division, provides an assessment of the Bank’s program-level evaluation function.

In 2011, the AfDB financed 184 operations in regional member countries at a cost of UA 5.72 billion. The AfDB was founded by signatory agreement between regional member states in 1963. It is one of three entities that together comprise the African Development Bank Group, the other two being the African Development Fund and the Nigeria Trust Fund. As a regional multilateral development finance institution, the mission of the AfDB is to contribute to the sustainable economic development and social progress of African countries. Membership of the AfDB comprises 77 countries, of which 54 are regional member countries. The AfDB provides development assistance to its regional member countries through the public and private sectors, in the form of financial instruments, policy advice, and technical assistance.

Canada’s financial support to the Bank consists of: (i) contributions to the Bank’s capital through share purchases; (ii) voluntary support to specific trust funds; and, (iii) voluntary support to the African Development Fund – the Bank’s concessionary funding window. The African Development Bank continues to be a key partner for CIDA in helping to achieve poverty reduction in the African region.

The Review conducted a meta-synthesis of the African Development Bank’s Operations Evaluation Office program evaluations between 2007 and 2012, and did not include a review of project-level or trust fund evaluations. Based on the quantity and quality of program-level evaluations, it was determined that the AfDB’s reporting on development effectiveness is not adequate and that its evaluation function does not provide an adequate information base on development effectiveness. Therefore, the report presents recommendations aimed at supporting improved evaluation and development effectiveness reporting, as suggested by the OECD-DAC evaluation methodology.

The Review’s conclusions are broadly consistent with the views expressed in the 2012 MOPAN assessment of the Bank, which concluded that the AfDB’s Evaluation Unit does not appear to have a formal process for reviewing the quality of evaluations, and that the frameworks and systems it uses to manage for, and report on, organization-wide results need further improvement.

CIDA accepts the Preliminary Review’s recommendation for improving the African Development Bank’s evaluation function, while noting that there have been some improvements since the review period. In particular, the AfDB has initiated a process to enhance its evaluation function through ensuring methodological rigor and improving the quality assurance process, as proposed in the 2011 self-assessment of the AfDB’s evaluation function, and the AfDB 2013-2017 independent evaluation strategy.

CIDA’s interventions to address the recommendations will focus on the following approach: 1) Issues related to improving the Bank’s evaluation function will figure prominently in Canada’s interventions in the upcoming meetings for the replenishment of the African Development Fund; 2) The issue will also be raised at the Bank’s Board of Directors in the context of the approval of the Three-Year Rolling Evaluation Work Programme; and 3) CIDA will assist with capacity development, based on opportunities identified by the Bank. This could be done, for example, through expertise provided via Evaluation Division standing offers.

Recommendations

1. Canada, in concert with other donors and AfDB Board members, should support the Bank’s efforts to strengthen its evaluation function.

Canada should support the AfDB’s efforts to strengthen its evaluation function and the design of its evaluations, as outlined above. One possibility, among others, would be an annual analysis of the AfDB’s evaluation coverage to be included as part of the AfDB’s evaluation work plan. For example, the AfDB’s rolling work plan could provide some measure of the target for, and actual implementation of, program level evaluation, perhaps as a percentage of total expenditure.

In working with other Board members and donors, Canada should signal its readiness to support the strengthening of the AfDB’s evaluation function. Important elements would include: peer review; capacity development to strengthen evaluation coverage, design, and timeliness; and strengthened program-level development effectiveness reporting.

Commitments/measures

Agree

1.1 CIDA, along with other donors, will urge the African Development Bank (AfDB) at the upcoming African Development Fund (AfDF XIII) replenishment meetings to strengthen its evaluation function. The AfDF XIII negotiations are scheduled to conclude in the Fall of 2013.

Responsible: CIDA Multilateral and Global Programs Branch IFI Division

Completion date: March 2014

Status: Ongoing. Was raised at the February 21-22, 2013 meeting in Tunis.

Commitments/measures

1.2 The Bank has indicated that evaluation coverage is already considered in the process of developing its Three Year Rolling Work Programme, which is adjusted annually. This is also considered within the Bank’s Independent Evaluation Strategy.

Responsible: CIDA Multilateral and Global Programs Branch IFI Division

Completion date: Completed

Status: The Three-Year Rolling Work Programme.

Commitments/measures

1.3 In working with other Board members, and in line with the Bank’s evaluation strategy, CIDA will assist with capacity development, based on opportunities identified by the Bank. This could be done, for example, through expertise provided via Evaluation Division standing offers.

Completion date: March 2014