Archived information

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Bolivia Country Program Evaluation - 2005-2010 - Synthesis Report

Foreign Affairs, Trade and Development Canada (DFATD)

January 2014

Table of Contents

Acknowledgments

The Development Evaluation Division wishes to thank all who contributed to this evaluation for their valued input, their constant and generous support, and their patience.

Our thanks go to the team that carried out the evaluation.  It was led by Philippe Bâcle of Baastel and included three experts from the same firm (Dean Pallen – expert in sustainable economic growth; Catherine Gander – expert in governance and gender equality; and Alain Lafontaine – expert in the environment) and three Bolivia experts (Bertha Pooley – expert in health; Aida Ferreyra Villarroel – expert in gender equality; and Fernando Aramayo – expert in governance).

The Development Evaluation Division would also like to thank the management and staff of the Bolivia Country Program for their valuable support.

From the Development Evaluation Division, we wish to thank Frantz Pierre-Jérôme, Evaluation Manager and Sarah Gibson, Senior Editor, for their contribution to the synthesis report, and Michelle Guertin, Evaluation Team Leader, for guiding the evaluation to completion.

James Melanson
Head of Development Evaluation

List of Abbreviations

BCP
Bolivia Country Program
CEAA
Canadian Environmental Assessment Act
CIDA
Canadian International Development Agency
CDPF
Country Development Programming Framework
DFAIT
Department of Foreign Affairs and International Trade
DFATD
Foreign Affairs, Trade and Development Canada
NDP
National Development Plan
SEG
Sustainable Economic Growth
USAID
United States Agency for International Development

Executive Summary

1.1 Bolivia Country ProgramFootnote 1

This evaluation assesses the Bolivia Country Program’s performance between 2005 and 2010 as part of the former Canadian International Development Agency (CIDA), amalgamated as of July of 2013, with Foreign Affairs, Trade and Development Canada.  The total funding disbursed by the Agency in Bolivia for the period was $119.66 million, spread across 233 projects, and involving: the Geographic Programs Branch which contributed $71 million (59% of the total); Partnerships with Canadians Branch having contributed 40%, (its most significant country in the Americas and globally over some of the review period); and Multilateral and Global Programs Branch contributing the remaining 1%.  Sectors of focus were democratic governance, health, and economic growth.

This evaluation responds to the Government of Canada’s Financial Administration Act and Treasury Board Evaluation Policy (2009) that require evaluation of all program expenditures every five years.  It aims to assess results achievement, inform decision-making, draw lessons learned, and improve program performance.  Evaluation criteria included relevance, effectiveness, efficiency, economy, coherence, and sustainability, with attention to cross-cutting themes of gender equality and environmental sustainability.  A sample of twenty-four projects that were in operation between 2005 and 2010 was drawn, encompassing 50% of total disbursements with a proportionate representation of funding from the three program branches.

1.2 Results

The evaluation finds good performance and contribution to reducing poverty in Bolivia.  The program built on previous investments and responded to emerging priorities to promote development in basic health care, democratic governance and sustainable economic growth (SEG).  The program’s on-going policy dialogue allowed it to respond effectively to changing policy environments in Bolivia and at CIDA.  Maintaining relevance to Bolivian needs increased chances for sustainability.

Notably, the program’s ability to respond to Bolivia’s changing priorities positioned it, and Canadian non-governmental organizations and their counterparts, to contribute to poverty reduction in Bolivia.  The hydrocarbons project was notable in this regard.

1.3 Conclusions

The evaluation provides evidence that the program helped improve living conditions and standards in the key programming sectors of health, governance, and SEG.  Factors contributing to success included a focus on building consistent and long-term relationships with Bolivian counterparts, alignment of development programming under Bolivian priorities, and coordination with other donors.

Policy dialogue proved effective in facilitating results across all sectors of priority.  In governance and in gender equality, a participatory approach allowed the program to secure government, donor and civil society organization participation.  Canadian interventions contributed directly to the strengthening of important national institutions, including the Auditor General, the National Electoral Court, and the Ombudsman Office.

Based on evidence from the sample of projects, key interviews, and relevant documentation, the following conclusions can be drawn on the evaluation criteria:

Relevance: While navigating changes to political leadership in Bolivia and to corporate orientation at CIDA, the program remained relevant to Canadian strategic priorities and Bolivian needs.  It delivered aid across priority sectors according to the 2003-2007 Country Development Programming Framework and the revised 2009 Country Program Strategy.  A major strength was the focus on building consistent, long-term relationships with key partners and beneficiaries through a highly participatory process.

Effectiveness: There were good development results achieved over the review period, particularly in SEG where the hydrocarbons project contributed to the country’s ability to benefit and redistribute revenues by establishing a more transparent and efficient tax and royalty collection system.  The program also contributed to better health services for the most vulnerable, and facilitated decentralization of services and strengthened governance and democratic institutions.

Sustainability: Encouragement of ownership by Bolivian stakeholders and other donors was a good strategy to help sustain some of the program’s investments in the face of limited local administrative capacity.  In the hydrocarbons sector there was an improvement in institutional capacity and sustainability. However, investments in health were rated as less sustainable due to the difficulties of developing strong local administration.

Gender Equality: During the early part of the review period, integration of gender equality across the portfolio was minimal given that the local gender specialist was focused on managing CIDA’s Gender Equality Fund.  In 2010, Holland became the lead on the Gender Equality Fund, which allowed the gender specialist to improve the integration of gender equality as a crosscutting issue in the program.  There were important contributions from the gender-focused programming, such as increased political participation by women, strengthened capacity of women’s organizations, and improved protection of women’s rights.  The program also played a leadership role in promoting policy dialogue on gender equality with other donors, Bolivian government partners and non-governmental organizations.

Environmental Sustainability: The evaluation demonstrated that in Bolivia, applying the Canadian Environmental Assessment Act was not enough to ensure full integration of environmental considerations across the program.  A case in point was water and sanitation where the focus was more on securing water supply than in promoting conservation and integrated management of water resources.  The program did nonetheless provide some good examples of promoting and integrating the environment, particularly in the SEG sector.  Overall there was room for improvement in the mainstreaming of environmental issues in the country program and projects, as recommended by the strategic environmental assessment conducted in 2009.

Coherence: The program demonstrated good examples of external coherence with other donor programs through involvement in multiple pooled funds and by participating effectively in a key multi-donor coordination group.  The evaluation also found relatively good internal coherence between the Geographic Program Branch and Partnership with Canadians Branch on some key projects, although a more systematic approach to coordination with civil society organizations would have been useful.

Efficiency: Based on document review including the 2011 Americas Program benchmarking exercise, the program performed favourably in terms of cost-efficiency, when compared to similar CIDA programs in other middle-income countries.  As well, there was an appropriate blend of larger pooled fund investments with relatively lower overhead cost ratios, and smaller but innovative remote area health interventions with relatively higher overhead cost ratios.  However, several limitations, also evident in other Country Programs, affected efficiency during the evaluation period, including high staff turnover, delays in project approval and limited delegation of spending authority.

Management Principles: The program demonstrated strong evidence of adherence to the principles of the Paris Declaration on Aid Effectiveness (ownership, harmonisation and alignment).  Policy dialogue was also a strength, facilitated through involvement in a number of pooled funds.  Even with shifting political circumstances in Bolivia, and new articulation of corporate priorities at CIDA, general alignment of program interventions with Bolivian priorities was maintained.

Performance Management: At the project level, results-based management principles, monitoring, evaluation and risk management practices were generally applied systematically.  There was a program level performance management framework in place, but it did not provide a sufficient overview of planned results with measurable indicators, and baseline data was not established, which limited the ability to articulate broader achievements and results.  Knowledge management, communication and dissemination of information about the program’s works and achievements could have been stronger.

1.4 Recommendations

A number of recommendations arise from the evaluation findings and conclusions:

1.0 Introduction

1.1 Objectives of the Evaluation

The findings and conclusions of this evaluation of the Canadian International Development Agency (CIDA)’s Bolivia Country Program (BCP)Footnote 2 are based on evidence gathered in 2012.  The evaluation supports accountability and learning, and has three objectives:

1.2 Country Context

Despite modest economic progress and its wealth in oil, natural gas, and minerals, Bolivia remains one of South America’s poorest countries and one of the world’s most unequal societies. According to the World Bank, Bolivia qualifies as a lower middle-income country with a gross national income per capita of $4,510 in 2010Footnote 3.  Continuing high poverty levels mean nearly 65% of its population lives under the national poverty line, and 84% of its rural population lives in extreme poverty.  In 2011, the country ranked 108 out of 187 on the Human Development Index developed by the United Nations Development Programme.Footnote 4

Over the period under review here, Bolivians radically restructured the country’s political economy. In 2006, an overwhelming majority of Bolivians voted for Evo Morales, the country’s first indigenous Aymarian to be elected President.  The new government outlined an economic and political vision for Bolivia that rejected the “free market economic model” encouraged by the World Bank and the International Monetary Fund.

The government’s new vision called for a mixed economic model built upon state management of the basic sectors of the economy.  The 2006 laws to nationalize natural resources, hydrocarbons and the mining sector were aimed at increasing government revenues and redistributing wealth.  The 2006 National Development Plan (NDP) provided a five-year overarching framework for change built upon four pillars:

To diminish the influence of donors on internal policy, the government passed a 2007 law regulating international cooperation and foreign funding.  The Ministry for Development Planning and its Vice-Ministry of Public Investment and Foreign Financing were charged with the responsibility of coordinating international technical and financial cooperation.  The Constitution of 2009 further defined the role of the state in the economy and society and devolved powers to sub-national governments and to indigenous communities to facilitate the implementation of the NDP.

Nevertheless, the spirit of national consensus dissipated over the period under review as political tensions climbed and sporadic violent protests erupted.  Progress was slow and the government did not meet the competing demands of all its voters.  A national debate ensued over the administration and redistribution of resources in the nationalized sectors of the economy.  Foreign interests complained about the nationalization law while political opposition groups contended that weaknesses in the law still left Bolivia’s natural resources vulnerable to exploitation by foreign corporations.

1.3 International Cooperation in Bolivia

The 2006-2011 NDP and the 2007 international cooperation laws in Bolivia changed the operating environment for CIDA’s Bolivia Country Program.  It responded by allying with other donors and the Bolivian government to form the Group of Partners for Development, a policy forum to promote Bolivia’s leadership in donor harmonization and alignment according to the principles of the 2005 Paris Declaration on Aid Effectiveness.

However, the socialist and nationalistic approach to development outlined in the NDP and the 2007 foreign-aid law complicated the relationships between the Bolivian government and Western donors. A chill in relations ensued.Footnote 5 The absolute amount of donor funding to Bolivia dropped significantly during the first years of the Morales government.  At the same time, it became particularly critical of the United States Agency for International Development (USAID) for its alleged funding of political opposition groups as part of its aid program.  In 2008, the US Drug Enforcement Agency was expelled from Bolivia, and in 2009, USAID was also asked to suspend its governance program.  Nonetheless, the United States remained the lead donor to Bolivia between 2005 and 2010 and overall funding from other donors increased again from 2008 to 2010, but generally did not reach pre-2006 levels.

Between 2005 and 2010, the top ten bilateral donors disbursed a total of US$2.7 billion to support development projects in Bolivia, averaged at US$454 million a year.  The principal donors included the United States (28%), Spain (17%) and Japan (11%), followed by Germany, Netherlands, Denmark, Sweden, Canada (4%), Switzerland and France.  In recent years, new donors have appeared, such as Venezuela, Brazil, China, and Cuba, all promoting a socialist agenda compatible with that of the Morales administration.  Major donors have coalesced around initiatives related to hydrocarbons, public health, governance, economic growth, education, capacity building and the environment.

1.4 CIDA Program

Canada has a long history of providing financial and technical assistance to Bolivia, beginning in 1967, one year before the official creation of the former CIDA.  Given the country’s chronic poverty and inequality, Bolivia was selected as one of CIDA’s nine countries of focus in 2003, and retained that status through reviews in 2005 and 2009.

CIDA’s programming in Bolivia for the period covered by this evaluation (2005 to 2010) was guided by the 2003-2007 Country Development Programming Framework (CDPF), the 2007 Canadian Government Americas Strategy, the 2009 Country Program Strategy, and other strategic programming documents.  It was also framed in line with Bolivia’s NDP.  Funding was provided through three channels: government to government; multilateral organisations and global programs; and civil society organisations.

The objective of CIDA’s assistance during this period was to reduce poverty and enhance living standards by helping to meet basic human needs and to consolidate Bolivia’s economic and political reforms.  This was pursued through efforts to improve access and capacity in the health and water and sanitation sectors, and to modernize the state’s regulatory capacity, public sector efficiency, and democratic practices.  In addition, CIDA programming addressed the integration of gender equality and environmental sustainability (as crosscutting issues), as required by the Agency’s policies.Footnote 6

In 2009, with the articulation of CIDA’s Bolivia Country Program Strategy, the overall programming objective stayed the same but the sectors of focus shifted to:

During the 2005-2010 period, CIDA’s disbursements to Bolivia totalled $119.66 million, covering 233 projects and making it one of the largest country programs funded by the Agency in Latin America.  Between fiscal years 2004-2005 and 2009-2010, disbursements for projects worth more than $250,000 totalled $111.58 million, across 64 projects.  More than half of the assistance was through the bilateral country program, one-third through partnerships with Canadian civil society organizationsFootnote 7, and a smaller fraction through multilateral and global programming,Footnote 8 as presented in Figure 1.  The sectoral breakdown of disbursements is illustrated in Figure 2, with democratic governance and health accounting for almost two thirds of the funds. Funding was disbursed mostly through civil society (61%) and the United Nations (20%), as illustrated in Figure 3.

Figure 1: Disbursements to Bolivia according to programming channels between fiscal years 2004-2005 to 2009-2010

Disbursements to Bolivia according to programming channels between fiscal years 2004-2005 to 2009-2010
Figure 1 Text Alternative
Disbursements to Bolivia according to programming channels between fiscal years 2004-2005 to 2009-2010
Programming ChannelDisbursements between fiscal years 2004-2005 to 2009-2010 (in millions)Percentage of Total
Bilateral programming$70.5859%
Multilateral and global programming$1.381%
Partnerships with Canadians programming$47.7140%

Figure 2:Disbursements to Bolivia according to programming sectors from fiscal years 2004-2005 to 2009-2010

Disbursements to Bolivia according to programming sectors from fiscal years 2004-2005 to 2009-2010
Figure 2 Text Alternative
Disbursements to Bolivia according to programming sectors from fiscal years 2004-2005 to 2009-2010

Programming Sector
Disbursements from fiscal years 2004-2005 to 2009-2010 (in millions)Percentage of Total
Democratic governance$42.2539%
Health$34.4930%
Economic Growth$23.4120%
Other or multisector$13.2011%

Figure 3: Disbursements to Bolivia by type of executing agency from fiscal years 2004-2005 to 2009-2010

Disbursements to Bolivia by type of executing agency from fiscal years 2004-2005 to 2009-2010
Figure 3 Text Alternative
Disbursements to Bolivia by type of executing agency from fiscal years 2004-2005 to 2009-2010
Type of Executing AgencyDisbursements from fiscal years 2004-2005 to 2009-2010 (in millions)Percentage of Total
Civil Society$72.6661%
United Nations$24.4920%
Foreign Government$8.087%
Canadian Government$7.496%
Other multilateral organisation$3.743%
Multiple partners (local funds)$3.213%

Building on the priorities of the 2009 Country Program Strategy, the 2010-2015 CDPF focused on economic growth, governance, and children and youth.

2.0 Evaluation Methodology

2.1 Scope

The methodology for country program evaluation assesses overall program performance, as well as results in sectors and crosscutting themes based on a representative sample of investments.

The evaluation team chose projects from among all those in operation, whether just beginning, in progress, or sun setting, and for which sufficient data existed.  The sampled projects were intended to be representative of the whole program in terms of:

In the end, a sample of twenty-four projects was selected, representing 50% of total program disbursements for the five-year period of this evaluation (See Annex 1 for the list of projects in the sample).  Ten represented health sector programming; eight, governance; and five sustainable economic growth (SEG).  Thirteen projects were funded through bilateral programming, ten through partnerships with Canadian civil society organizationsFootnote 9, and one through multilateral institutions and global programs.

2.2 Evaluation Questions and Criteria

Development results are examined by asking what was achieved by the program during the evaluation period.  Program management effectiveness is assessed by asking how and why the intended results were achieved, or not.  The evaluation assesses these points based on eight criteria: core ones recommended by Canada’s Treasury Board; and additional ones from the Organisation for Economic Co-operation and Development’s Development Assistance Committee.

Development Results

What was achieved by the program during the evaluation period?

Management Factors

How and why were the intended results achieved or not?

2.3 Data Gathering Methods and Analysis

Data was obtained at CIDA headquarters, and from Canadian stakeholders, support agencies, non-governmental organizations, and monitors involved in implementing the program.  Supported by three Bolivian experts, the team also collected data in Bolivia during a three-week mission in September-October 2012.

Several lines of evidence were gathered, ensuring triangulation of findings:

The data analysis focused on (see Annex 4 for the evaluation matrix):

2.4 Evaluation Challenges and Limitations

The evaluation team faced some constraints and limitations in collecting and analyzing data.  These were deemed manageable and do not detract from the overall validity of the findings, conclusions and recommendations.

2.5 Data Availability and Reliability

To overcome these constraints, the evaluation team expanded its research at CIDA - triangulating and comparing lines of evidence, and validating them with the largest possible number of contacts within and outside CIDA to create a reliable picture of the program’s performance.

2.6 Canadian Contribution and Attribution

The results reported in the present evaluation may be attributed to Canada where the Agency has been the sole executive party of the development interventions.  In many instances, however, Canada acted as a partner in development interventions.  Only Canada’s contribution to results can be recognized; direct attribution to Canada of the results achieved is not feasible.

In the context of this evaluation, given the large number of development interventions in which Canada is a partner, the focus is placed on Canada’s contribution.

3.0 Findings for Development Results Criteria

The program’s three sectors achieved predominantly “highly satisfactory” results across all five evaluation criteria measuring development results as demonstrated in the following sections (see Annex 5 for project scores by sector).

3.1 Relevance—Highly Satisfactory

The program scored well on relevance because it committed to stakeholder participation, and focused on multiple needs as articulated by the most vulnerable.

For the period under review (2005-2010), CIDA’s Bolivia Program rated in the highly satisfactory range for its relevance to Bolivian, Canadian and Agency development priorities and to the needs of the Bolivian people (summarised by sector in the table below).

Figure 4: Relevance Scores by Programming Sector

Relevance Scores by Programming Sector
Figure 4 Text Alternative
Relevance Scores by Programming Sector
Programming sectorScore
Sustainable Economic Growth4.1 – highly satisfactory
Health4.8 – highly satisfactory
Governance4.4 – highly satisfactory
A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

3.1.1. Relevance to shifting Bolivian priorities.

The program’s long-standing commitment to involving stakeholders at the design and implementation stages allowed it to navigate the political changes in Bolivia while maintaining relevant programming.

The evaluation confirms that the 2003-2007 CDPF in place at the outset of the review period reflected both Agency and Bolivian aims.  The programming was in line with the 2005 Paris Declaration on Aid Effectiveness.  The program’s sectors of concentration reflected three corporate priorities: health; water and sanitation; and modernization of the state, including public sector reforms.  The plan also promoted the crosscutting issues of gender equality and environmental sustainability.  Further, it channelled most of its disbursements to Bolivia through institutional pooled funds (also called basket funds), which required a greater focus on Bolivians’ own capacity to manage and implement.

The CDPF responded to Bolivian priorities and reflected the 2001 Poverty Reduction Strategy that called for more efficient, transparent, and accountable institutions, more professionalism in the public service, judicial reform to reduce corruption, a strengthening of municipal governments and a general enhancement of civil society participation.  It also built on a history of CIDA investment in health benefitting vulnerable Bolivians.

The high relevance of this programming stemmed from a practice of participatory project design and of working with partners including other donors, Ministries, community leaders, non-governmental organizations and municipal authorities.  As a result, many parts of the program proved resilient during the 2006 political realignment in Bolivia, and the 2006 and 2009 elections of the Morales government on a “socialistic-indigenous” platform.

The new administration’s agenda had implications for the delivery of development assistance in the country.  The 2006 National Development Policy provided new guiding principles based on ideas of social inclusiveness, and economic decentralization and diversification.  Also, the 2007 law to govern international cooperation in the country introduced new standards and procedures.  For donors, there followed a period of adjustment between 2006 and 2008 as tensions rose over perceptions about the possible dilution of Bolivian development plans by western donors.

Throughout this period, CIDA maintained the confidence of the Bolivian Government.  Bilateral discussions led to the ratification of initiatives begun under the 2003-2007 CDPF.  The program moved forward with investments in health and sustainable economic development.  It also retooled some of its existing governance sector programming, (designed in collaboration with the outgoing political establishment) to better reflect the new political landscape.

Sustainable Economic Growth: Evaluation evidence suggests that support of non-governmental organizations to promote entrepreneurship and to support micro finance contributed to small-scale economic growth.  Likewise, support to reform of the hydrocarbon industry helped increase revenues and facilitated the government’s efforts at wealth redistribution.

Health: Most of the health projects in this evaluation sample were developed during the transition period (2006-2009)Footnote 11, and reflected the benefits of a participatory design process.  The program’s major health investment, to support the Ministry of Health and Sport, arose out of consultations with the government and aligned explicitly with the NDP.  The Program to Support the Health System project and its components responded to Bolivian desires for universal access to quality health care, environmentally sensitive health initiatives, nutrition, and the full ownership of health services.

The programs’ water and sanitation projects also aligned well with Bolivian priorities set out in Bolivia’s National Water Resource Strategy, the National Watershed Plan and the National Desertification Plan.  The Zero Malnutrition Program reflected Bolivia’s Health Sector Development Plan, and the Micronutrient Initiative’s projects fit well with Bolivia’s Strategic Nutritional Plan.  The projects focused on delivering comprehensive health care by (1) designing indigenous-sensitive health care models; and, (2) offering training to deliver services, nutrition, and water-and-sanitation to remote regions.

Stakeholders agreed that CIDA was “very relevant and crucial to improve(ing) Bolivia’s health indicators and (for) align(ing) interventions with the international community.”

Governance: The ability of governance sector programming to adjust to the new regime also reflected its highly participatory design process.  Three governance sector projects included in the evaluation sample predated the Morales election.  All remained relevant afterward.  The Strategic Governance Mechanism supported four institutions: the Ombudsman’s Office, National Statistic Institute, National Electoral Court, and Auditor General’s Office—identified jointly with the Bolivia Vice Ministry of Public Investment and Foreign Financing.  When the 2009 Constitution broadened the responsibilities of these institutions, the program only increased its relevance.  The Andean Regional Gender Equality Fund (Phase three), designed to increase women’s role in decision-making, maintained relevance before and after the 2006 election.  The fund was designed to complement the gender policies of the previous government and support decentralization, also central to the new platform.  Promoting a more pro-active approach to gender equality, the Government created a new National Women’s Machinery under the Ministry of Justice.  A National Plan for Equality of Opportunities set the government’s priorities in promotion of women’s rights in land ownership, access to education, representation and participation in public office, and prevention of gender-based violence.

The program also retooled some of its programming to emphasize contributions to the Bolivian aim of increased representation of indigenous peoples.  It focused on promoting human rights and access to justice in collaboration with the Ombudsman’s Office and Public Defender’s Office; and elector participation and access to municipal and national government, with the National Electoral Court, the Association of Women Councillors of Bolivia and the Federation of Municipal Associations.

3.1.2. Relevance to realigned Agency priorities

The 2009 Country Strategy continued many of the general themes outlined in the 2003-2007 CDPF, readjusting emphasis to take account of emerging Canadian strategic priorities.

The Government of Canada’s 2007 Americas Strategy emphasised economic prosperity, democratic governance, and security and stability.  In 2009, CIDA refocused corporate priorities on food security, economic growth, and children and youth.  Consequently, the program reconfigured its investments so that i) governance became a crosscutting theme (along with gender equality and environmental sustainability); ii) longer-term health and water and sanitation programming contributed to children and youth; and, (3) the hydrocarbons project emphasized its contribution to sustainable economic growth while continuing to contribute to governance.

3.1.3. Relevance to a range of beneficiary needs

Innovative and culturally appropriate health care projects in remote and less populated areas served a population overlooked by the Bolivian government and other donors.  Governance and sustainable economic development programming addressed needs of national institutions, respecting the government’s socialist principles.

3.2 Effectiveness—Highly Satisfactory

There were good development results achieved over the review period, particularly in SEG.

The effectiveness of the Bolivia Country Program over the period under evaluation was rated as highly satisfactory and is summarised by sector in the figure below. The program demonstrated progress towards and achievement of stated project, program, and corporate objectives, particularly those related to the reduction of poverty.

Figure 5: Effectiveness Scores by Programming Sector

Effectiveness Scores by Programming Sector
Figure 5 Text Alternative
Effectiveness Scores by Programming Sector
Programming sectorScore
Sustainable Economic Growth4.3 – highly satisfactory
Health4.2 – highly satisfactory
Governance4.1 – highly satisfactory
A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

3.2.1. Sustainable Economic Growth programming contributed to poverty reduction

The hydrocarbons project evolved over the five years under review to achieve results in governance and sustainable economic growth, securing modest gains in reducing Bolivian poverty.

The project helped strengthen national institutions relevant to the hydrocarbon industry and provided guidance around natural gas pricing for export to Brazil and Argentina.  This “helped greatly,” according to interviewed stakeholders, in seeing the average annual revenue from the hydrocarbon sector increase to US$ 2.1 billion, between 2006 and 2011, from US$322 million, between 2000 and 2005.  The increased revenue contributed to the reduction of the country’s dependency on development aid funds to 17% in 2010, down from 53% in 2005.

The hydrocarbon project also contributed to the country’s ability to redistribute hydrocarbon revenues.  It helped establish a more transparent and efficient tax and royalty collection system, supported by training from the Canadian Revenue Agency in strategic planning, legal contracts and tax collection, auditing, and fraud investigation.  The redistribution of increased revenue helped put the country on track to meet the Millennium Development Goal of reducing by half the number of people living on less than a dollar a day.Footnote 12

The program also contributed to small-scale economic growth by supporting Canadian and Bolivian non-governmental organizations. The Société de Coopération pour le Développement International project supported the emergence of a new and highly viable economic activity—oregano production in high altitude, semi-arid conditions.  Nearly 1000 small-scale farmers in eight municipalities currently participate.  Those interviewed for this evaluation spoke of using the extra income generated to pay for children’s education, health care, and for increasing consumption.  The project also invested in further developing local markets, including increased production and sale of organic eggs.

The Uniterra project promoted the concept of Fair Trade to help local producers gain access to world markets on favorable terms.  It trained 30 organizations in Fair Trade and the principles of economic solidarity, and 80 women-leaders in the management of microenterprises.  It also certified (with a private Bolivian institution) 160 productive organizations.  The project supported trade fairs, conferences, showrooms, promotional campaigns, and the opening of markets in Germany, Switzerland, Italy, Spain and Sweden.  It improved the ability of artisans to defend their interests and engage in political dialogue.

The participation in the work of these non-governmental organizations at a strategic level by the Canadian Executive Services Overseas’ Volunteer Cooperation program,Footnote 13 amplified the gains across the whole sector.  The program helped the textile business, supported by a non-governmental organization, to establish a highly profitable store, Coconut.  As well, the Canadian Executive Services Overseas is helping companies to develop their export capacity, in cooperation with the Trade Facilitation Office of Ottawa.

3.2.2. Health programming contributed to enhanced public policy and delivery of services to the most vulnerable

The program’s culturally relevant health projects contributed to policy development in Bolivia and the provision of increased healthcare in remote regions and to marginalized peoples in Beni and Pando.

Several projects seeded or provided templates for health-related policy developments, including Bolivia’s 2001 Basic Sanitation Development Plan, the National Food Security Plan and the Nutrition Policy.

The Support for the Ministry of Health and Sport project and its component projects performed well.  A formal demographic and health survey in 2013 will generate comprehensive data to support conclusions about its effectiveness.  In the interim, this evaluation found evidence of accessible healthcare services and improvements to the capacity of the Ministry of Health and Sports and to Departmental Health Service. Indicators for the Project to Improve the Coverage and Quality of Health Care show an increase in the number of children below the age of two vaccinated (from 50% to 70%); a 10% increase in services delivered by skilled workers in three departments; and the skills of over 50% of the health personnel updated or upgraded.  In Oruro Department, almost 80% of facilities were updated to be user friendly and culturally sensitive.

Other projects such as the National Fund for Productive and Social Investment, which responded to infrastructure and cultural needs amplified the Program to Support the Health System outcomes.  .  However, there was evidence to suggest that infrastructure design did not always take the opinions of end users or local health personnel into account.

The evaluation also confirms that the program’s health sector projects contributed to tangible outputs and outcomes for the intended beneficiaries:

3.2.3. Governance programming supported decentralization and capacity building.

The Municipal Partnership Program built on achievements of a previous phase, and facilitated decentralization, the transfer of revenues to municipal governments, and mechanisms for dispute resolution between federal and local levels of government.  It provided advice incorporated into the 2005 Presidential Decree on Hydrocarbon Revenue, encouraged increased constitutional powers for municipalities, women’s participation in local governance, and support for local economic development.

The Strategic Governance Mechanism focused on the National Electoral Court, the Bolivian National Institute of Statistics, and the Auditor General’s Office to increase transparency and capacity for public policy-making.  The Defensoria del Pueblo project contributed to the consolidation of the Ombudsman of Bolivia as the “leading institution on human rights”.  Bolivian opinion polls revealed that Bolivians considered this institution “to be the (...) closest to citizens and the most committed to defending human rights.” The Office increased its presence throughout the country, empowered citizens, promoted its conflict resolution services, increased the number of investigations into complaints by 68%, and improved civic education about human rights.  However, much remains to be done to promote indigenous rights and to fully extend the institution’s presence into rural areas.  In addition, the political flux in Bolivia between 2001 and 2006 meant a high turnover in staff and continuing limited capacity of the judicial system.

The National Electoral Court, which sought to increase participation and integration in the electoral process, particularly for indigenous people, contributed to the following results:Footnote 14

The Trilateral Criminal Defence project aimed to improve access to justice for impoverished and indigenous persons.  Implemented by the Quebec Bar and the Public Defender’s Offices in Chile and Bolivia, it helped improve the capacity of Bolivian public defenders to protect the rights of accused persons, increased public awareness of rights, and improved the capacity of legal aid offices.  In all, 91 lawyers were trained to protect the rights of the accused, including 43 women and several indigenous professionals.

3.3 Sustainability—Satisfactory

While local administrative capacity was often modest, strong buy-in from Bolivian stakeholders helped to ensure longer-term implementation.

CIDA’s Bolivia program, for the duration of the period under review (2005-2010) was rated in the satisfactory range for ensuring the sustainability of its programming. There were challenges to creating sustainable governance programming, while other sectors attained highly satisfactory ratings (summarised in the figure below).

Figure 6: Sustainability Scores by Programming Sector

Sustainability Scores by Programming Sector
Figure 6 Text Alternative
Sustainability Scores by Programming Sector
Programming sectorScore
Sustainable Economic Growth4.2 – highly satisfactory
Health4.1 – highly satisfactory
Governance3.3 – satisfactory

A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

3.3.1. The program cultivated support in Bolivia for the long-term extension of health and governance projects.

The evaluation confirms on one hand that government capacity was not sufficient to guarantee the long-term effects of the program’s investments.  It confirms on the other hand that some key results presented relatively strong sustainability factors and would remain after CIDA’s interventions ended.  The program achieved satisfactory scores by cultivating the partnership of the Bolivian government in health and governance programming and encouraging continued commitment to programs even after external funding ceased. Sustainability was encouraged at the local level by transferring responsibilities/ownership to, and securing the commitment of, some municipal and departmental governments.

Health: Health programming in general presented relatively strong sustainability factors.  Indicators of the high probability of lasting effects of the program’s investments are documented in project mid-term and final reports.  The sustainability of multi-sectoral interventions in the health sector was supported by the approval of the National Food and Nutrition Policy.  In projects such as the Program to Support the Health System and Malnutrition Zero, methodologies were scaled up, institutional personnel upgraded their skills, government institutions were empowered and gained leadership, and municipalities adopted budgets for health activities.  One area for improvement would be the development of sustainability plans from the beginning, during the design phase. Other examples of sustainability in health include:

Governance: In the governance sector, certain project design factors add to the long-term potential of the program’s investments.  Among these were building on long-standing relationships with key Bolivian institutions, (the Bolivian National Institute of Statistics, the Ombudsman’s Office and the National Electoral Court), aligning funding priorities with the institutions’ core mandates, and taking a long-term outlook.

As well the use of pooled funding arrangements encouraged local ownership and therefore sustainability.

3.3.2. Improvement needed in government capacity to guarantee sustainability of projects in all three sectors.

Sustainable Economic Development: New laws and standards for exploration, technical development and management are needed to solidify the outcomes of the hydrocarbon project.  Government funding and other financial incentives to support small-scale economic activity (rural entrepreneurs, cooperatives and other associations) are very limited.  For example, in relation to Fair Trade there have been many declarations but little policy and direction.  Overall, however, the program’s contributions to Bolivia’s reduced reliance on foreign assistance appear to promise more nationally driven sustainable development.

Governance: Most of the program’s governance partners changed their structures, mandates and even staff upon the installation of the Morales administration. This eroded institutional capacity building in which the program had invested over the previous decade.  Also, the politicization of foreign aid delivery in Bolivia following the 2007 foreign aid laws introduced government controls that threatened the autonomy and credibility of initiatives.  Many mid-size institutional partners remained quite dependent on international funding which became less available as several donors reduced funding to governance projects in Bolivia.  The relatively short-term, small-scale projects supported through CIDA’s responsive governance funds (Responsive Fund for Local Governance, Canadian Executive Services Overseas and Gender Equality Fund) also tended to be “one off” activities which partners lacked the technical or financial capacity to sustain.

Health: Similar challenges also affected health sector sustainability:

(1) Multiple changes and staff turnover reduced the sustainability of achievements in capacity development in the central government;

(2) The institutional memory of health interventions tended to reside in extra-national organizations such as UNICEF instead of government institutions; a formal and systematic approach to institutionalize that memory in relevant actors at the central and departmental levels could help overcome this challenge; and,

(3) Some projects did not develop the administrative and financial skills of local actors (e.g. the Nutrition Unit and Departmental Health Service personnel).  Short-term consultants carried out several activities without adequate knowledge or ability to transfer skills, which reduced technical sustainability.

3.4 Crosscutting Theme of Gender Equality – Positive Results

The program was slow to promote gender equality at the design phase, but achieved satisfactory and highly satisfactory results. The promotion of gender equality in sustainable economic development was particularly strong.

For the duration of the period under review (2005-2010), CIDA’s Bolivia program achieved mixed ratings for the effectiveness of its integration of gender as a crosscutting theme.  Ratings by sector are summarised in the figure below.Footnote 15

Figure 7: Gender Equality (Crosscutting Theme) Scores by Programming Sector

Gender Equality (Crosscutting Theme) Scores by Programming Sector
Figure 7 Text Alternative
Gender Equality (Crosscutting Theme) Scores by Programming Sector
Programming sectorScore
Sustainable Economic Growth4.2 – highly satisfactory
Health3.6 – satisfactory
Governance4.2 – highly satisfactory

A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

3.4.1. The program was slow to integrate gender equality at the design phase.

Overall, the evaluation found that through its gender-focused programming, Canada contributed to important results such as increased political participation by women.  It made gender integration part of policy dialogue among stakeholders, strengthened capacity of women’s organizations, and improved protection of women’s rights.  However, evaluation evidence also noted that in the early phase of the period under review, the program was slow to integrate gender equality at the design phase of most of its projects.  As of 2008, only a few projects had an explicit gender strategy and a gender committee.  In 2010, a mid-term evaluation of the Strategic Governance Mechanism in BoliviaFootnote 16 recommended several steps for CIDA to better integrate gender equality at the institutional and program levels.  The program responded with a detailed plan to reinforce the gender perspective in its programming with notable examples of gender integration in bilateral programming and partnership projects.

In addition, through several pooled-funded projects, the program played a leadership role in promoting policy dialogue on gender with other donors and Bolivian government partners.  A 2011-2012 Peer Review of CIDA’s performance in gender integration confirmed progress achieved in this area.  The Peer-Review also recognized that the program could do more to integrate gender as a crosscutting issue.  

3.4.2. Most projects across all three sectors achieved concrete gains toward gender equality.

The present evaluation affirms that satisfactory and highly satisfactory scores reflect the achievement of concrete results.

Sustainable Economic Growth projects (the five projects factored into the sample) were by far the most effective at systematically promoting gender equality, receiving highly satisfactory scores.  The projects included activities that targeted women by promoting economic and political empowerment and professional development.  One project in particular, created 17 women-led and 22 mixed community groups that built capacity in sustainable agriculture and natural resource management.  Many women received training as entrepreneurs and community leaders.  Three projects (World University Service of Canada - Centre for International Studies and Cooperation, Canadian Executive Services Overseas, Société de coopération pour le développement international) in particular trained women entrepreneurs in producing textiles, handicrafts and oregano.  The projects also provided equal opportunity training for all workers (women included).

The success of crosscutting gender equality programming in the Hydrocarbon project is encapsulated by one manager: “Women hold less than 10% of technical or professional jobs in the oil and gas industry, so we’ve still got a long way to go.  But, we have made some progress in changing sexist attitudes and stereotypes, and in giving women more equal opportunity.  The CIDA project helped open the door for women to enter the hydrocarbon sector in Bolivia.”

Health sector programming was the weakest at promoting gender equality.  Only two of the ten projects in the evaluation sample received scores of “highly satisfactory” while the remaining eight earned scores of “satisfactory.”  Stakeholders interviewed for the evaluation suggested that the program “missed opportunities.”

Governance: During the review period, some governance sector programming integrated gender equality effectively.  Of the six governance projects reviewed, four projects received a score of “highly satisfactory” and two “satisfactory.”  Three of the initiatives that performed well were pooled funds with institutional partners.  Among pooled-fund donors, CIDA was the one that most consistently promoted gender mainstreaming in institutional policies and programs.  The program also provided technical assistance and policy advice to support partners in integrating gender equality.

The satisfactory projects helped enable indigenous women to participate in elections and access social services and promoted gender equality in water committees, while the highly satisfactory projects achieved broader results.  One of the four highest-scoring projects, (Municipal Partnership Program with the Federal of Canadian Municipalities) promoted the participation of women councillors in municipal government.  The other three contributed to pooled funds with institutional partners and set a standard for gender equality programming.  That was particularly the case when CIDA and the Netherlands designed and launched the Women’s Participation in the Economy Fund ($20 million) to continue gender-focused programming after the Gender Equality Fund ended.

The Andean Regional Gender Equality Fund, phase three, achieved results in 27 projects in seven Bolivian departments and over seventy municipalities.  It facilitated community support of women’s rights and promoted gender equality on the national level, training women to participate in the country’s decentralization process.  Concrete outputs and outcomes include:

The Gender Equality Basket Fund was not however renewed due to limited capacity of the Bolivian Vice-Ministry of Equality of Opportunities.

Stakeholders interviewed reported “we did not talk much about gender equality until CIDA raised it.  CIDA helped put gender on our agenda.”

3.5 Crosscutting Theme of Environment—Satisfactory

The program promoted environmental sustainability at the design phase, but lacked the technical expertise to realise its aims.

The program made progress toward achieving environmental sustainability. All three programming sectors, achieved scores in the “satisfactory” range and the program complied with the Canadian Environment Assessment Act (CEAA). Sustainable economic development programming scored 3.9; health, 4.0; and, governance, 3.4 (see figure below).

Figure 8: Environmental Sustainability (Crosscutting Theme) Scores by Programming Sector

Environmental Sustainability (Crosscutting Theme) Scores by Programming Sector
Figure 8 Text Alternative
Environmental Sustainability (Crosscutting Theme) Scores by Programming Sector
Programming sectorScore
Sustainable Economic Growth3.9 –satisfactory
Health4.0 – satisfactory
Governance3.4 –satisfactory

A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

3.5.1. The program promoted environmental sustainability at the design phase.

The program improved (over the five years of the evaluation period) the integration of environmental issues at the planning stage, and went beyond compliance with the Canadian Environmental Assessment Act (CEAA) in implementing health projects.  The Country Program Framework that ended in 2007 planned for several environment-related results and identified several indicators for monitoring.  The Country Program Strategy of 2009 launched a strategic environmental assessment.  The exercise indicated that for the 2010-2015 period, the program could expect to achieve some positive environmental impacts and few negative ones.  However, more work needs to be done to fully integrate environmental considerations across sectors.  For example, in water and sanitation, the focus was more on securing water supply than on promoting conservation and integrated management of water resources.  In governance, effort was mostly related to compliance with the CEAA.  SEG was the sector in which environment integration was most proactively pursued.

3.5.2. The program lacked the expertise to follow through with implementation

The evaluation demonstrated that the program was in compliance with the Canadian Environmental Assessment Act.  All screenings and forms were cleared as per provisions of the Act and contributed to some environmental projects, resulting in the program’s modest “satisfactory” score.  However, much more remains to be done to improve mainstreaming of environmental sustainability as concluded in the 2009 Strategic Environmental Assessment.  The program lacked specialist support, at headquarters and in the field.  The program also did not play an active role in policy dialogue on environmental issues as other donors have and is not very involved in capacity building of partners.

3.5.3. The program achieved some results in environmental sustainability in all three sectors

Despite the foregoing limitations, attention to crosscutting programming to support environmental sustainability yielded some results.

Sustainable economic growth: Environmental sustainability programming in the sustainable economic growth sector achieved satisfactory ratings.  The Hydrocarbon Project did not contribute any unwanted environmental impacts and it supported energy development practices and funded the Socio-environmental Monitors Training Program.  The projects proactively integrated environmental concerns as outlined below:

Health: Two of ten health projects promoted environmental sustainability and acknowledged environmental concerns as a determinant to children’s health and nutrition.  The Water and Sanitation Committee and two projects supporting UNICEF (water and sanitation project (PACSAS) and Save the Children (5-year project)), promoted environmentally sustainable water usage and health practices.  The University of Calgary water resources management project identified several priority areas in order to promote environmental sustainability, and the Université du Québec à Montréal (UQAM) Eco-développement communautaire project promoted several ways of addressing environmental sustainability in Bolivia.

Governance sector projects complied with environmental standards, but only two elaborated specific environmental components.  Notably, however, the Canadian Executive Services Overseas program sponsored a Canadian adviser to help municipal governments strengthen solid waste management systems and the Municipal Partnership Program included environmental analysis and results.

4.0 Findings for Management Factors Criteria

The program achieved “highly satisfactory” and “satisfactory” results across all four evaluation criteria measuring management factors.

Figure 9: Overview of Management Factors Scores at the Program Level

Overview of Management Factors Scores at the Program Level
Figure 9 Text Alternative
Overview of Management Factors Scores at the Program Level
Management FactorScore
Coherence4.2 – highly satisfactory
Efficiency3.9 – satisfactory
Management Principles4.2 – highly satisfactory
Performance Management4.0 – satisfactory

A score between 4.1 and 5.0 is considered highly satisfactory
A score between 3.1 and 4.0 is considered satisfactory.

4.1 Coherence – Highly Satisfactory

The program demonstrated good examples of external coherence with other donor programs through its involvement in multiple pooled funds and by participating in a key multi-donor coordination group, and internal coherence within CIDA, particularly between the Geographic Program Branch and the Partnership with Canadians Branch.

4.1.1. CIDA worked well with its partners to complement and optimize their relative strengths

The program facilitated external coherence as a founding and active member of the Group of Partners for Development, a group of 23 multilateral and bilateral donors formed in 2006.  Through its leadership, the program supported the group by mediating between donors, non-government organizations and governments, and by contributing to resolution of potential conflicts. The program chaired of Group’s committees, including the Gender Equality Committee convened to coordinate gender projects.  It also participated in six other committees: water and sanitation; health; public administration; decentralization; non-governmental organizations; and gender equality. The following examples of external coherence resulted:

By the end of the review period, the program had earned high commendations from stakeholders for its leadership and contributions to the Group of Partners for Development.

4.1.2. Internal coherence

There were some notable examples of coherence between the Geographic Program Branch and the Partnership with Canadians Branch projects in Bolivia, including:

While there was not initially a systematic mechanism to promote inter-branch coherence, the recent constitution of the Coordination of Canadian Civil Society Organizations in Bolivia (Coordinación de las Organizaciones de la Sociedad Civil Canadienses en Bolivia) does move in this direction. 

4.2 Efficiency - Positive Results

The program’s efficiency was rated as satisfactory, demonstrating some successes and areas for improvement.

4.2.1. The program maintained generally satisfactory levels of cost efficiency

The 2011 benchmarking exercise of CIDA’s Americas Branch programs on Agency programs in some middle-income countries such as, Peru, Vietnam, Indonesia and ColombiaFootnote 17 revealed the Bolivia program’s relative success in achieving cost efficiency in comparison to similar programs.  Documentation reviews and interviews carried out during the evaluation process confirmed Bolivia’s favourable performance in terms of cost-efficiency and efficient use of financial resources.

The evaluation, however, identified some limitations affecting program efficiency including:

It is worth noting that the program agreed to deliver health services in some remote areas that were difficult to access, less populated, and had limited administrative capacity at the department and municipal levels.  Even though working in these conditions proved to be costlier per beneficiary, it contributed to an overall and rational division of labour among donors.  Other donors praised this approach.

4.2.2. The degree of decentralization affected efficiency

CIDA’s Aid Effectiveness Action Plan 2009-2013 aimed to increase the effectiveness of the development cooperation program.  However, during the review period restrictions on the level of decentralised approval authority, (limited to $100,000), affected the ability of the program to plan and deliver in efficient timeframes with partners and other donors in Bolivia.

4.2.3. Human resources management affected efficiency

Stakeholders praised the leadership, open-mindedness and flexibility demonstrated by the field team, assisted by experienced analysts at the Program Support Unit.

There was however relatively high turnover during the review period.  Three heads of aid managed the program in the field between 2005 and 2010.  The number of consultants and staff at the Program Support Unit fluctuated given changes in the program during the period, when there was also uncertainty due to lack of a long-term contract.

Management Principles—Highly Satisfactory

The program showed strong adherence to the 2005 Paris Declaration principles but its promotion of local ownership fluctuated during the evaluation period.

4.2.4. Successful promotion of ownership

The 2003-2007 CDPF in effect during the review period resulted from a two-year consultative process with Bolivians, Canadians and the international aid community where Bolivians played a leading role in identifying areas of intervention.  It was based on thorough analysis of Bolivian needs, priorities, capacities, Canadian value-added, and other donor programming.

During the development of the 2009 CPS and 2010-2015 CDPF, the program appeared less committed to the spirit of Paris Declaration principles.  The 2009 CPS reflected the Agency’s new thematic priorities, and while within the country’s overall development needs, did not emerge explicitly from consultation with the Bolivians.  CIDA chose two thematic priorities (health and sustainable economic development).  Likewise in 2011, CIDA selected health as the sole sector of intervention.

At the sectoral and project levels, the degree of local ownership varied.  The governance program tended to have a high degree of local appropriation.  Projects funded through responsive local funds generally had high local ownership because they were fully designed and implemented by Bolivian partners.  Pooled funds built the capacity of institutional partners to plan, implement, and monitor their strategic plans and priorities.

Despite repeated efforts by CIDA, health sector projects tended to have limited ownership by the Government of Bolivia, reflected in low absorptive capacity and administrative problems.

The program promoted ownership in several SEG projects, for example developing the capacity of the Ministry of Hydrocarbon and Energy and Yacimientos Petroliferos Fiscales Bolivianos to prepare and implement their sectoral strategies.  

4.2.5. Alignment was a challenge given the changing Bolivian and CIDA priorities

Alignment with Bolivian development priorities was a challenge, mainly because national plans were weak and Government of Bolivia’s priorities kept changing.  The 2006-2011 NDP was very broad, and was more a vision than an operational development plan with clear guidelines.  CIDA also re-articulated its own priorities during the evaluation period.  Despite these limitations, individual projects were generally aligned with Bolivian partner priorities.  When CIDA modified the programming sectors of priority, the program consulted with the Government of Bolivia in order to ensure alignment with their priorities for those sectors.

The level of alignment depended to some degree on the delivery channel.  For example, many Sustainable Economic Growth projects managed by Canadian and Bolivian partners tended to be designed and delivered locally.  Given that these actors did not have strong working relations with the Government of Bolivia, and funding came most of the time from independent sources (donors, international non-governmental organizations) they were less aligned with Government priorities.

4.2.6. The program was relatively successful in participating in policy dialogue

CIDA was one of the founders of the Group of Partners for Development in 2006 and played an active role in the group during the review period.  Stakeholder interviews revealed appreciation for the initiative in promoting harmonization and policy dialogue.  Interviewees recognized CIDA as a leader in gender equality and a strong contributor to policy dialogue, particularly in the health sector.

“Canada tried to remove bottlenecks in relations between the government and donors.  It helped the (Group of Partners for Development) mediate conflicts.  For example, when Canada was the leader of the Group’s (non-governmental organization) subgroup, it played a very constructive role in resolving the issue of tax treatment of non-governmental organizations.  Canada took that issue to the very high levels of government, and got the dialogue started.  Canada showed real leadership and political acumen.” -Donor representative

“Canada helped establish dialogue and improve relations with the Bolivian government at a very difficult time.  Canada helped close the distance that had existed between donors and the government for a few years.  The Head of Aid had innate diplomatic skills, and his leadership was very appreciated here.” - Donor representative.

However, the program’s credibility in the field suffered from changes in CIDA country priorities, the lack of long-term funding, budget cuts, and long project approval times. For example, for nearly two years, CIDA negotiated with other donors a large-scale water and sanitation project that did not receive final approval. In another instance, the program participated in multi-donor discussions to develop a program supporting decentralization, and then had to withdraw following a funding decrease in the governance sector.

4.3 Performance Management—Challenges Noted

Performance management was generally applied systematically at the project level but weaknesses were observed at the program level limiting the program’s ability to communicate results.

4.3.1. The program became an advocate for Results Based Management

After implementing CIDA’s Result-based Management policy in 2008, the program became a strong advocate for building results-based approaches into its projects.  It provided training and technical assistance to partners in Bolivian government and civil society in implementing and executing results-based management, and in monitoring and evaluation.

In the health sector in particular, results-based monitoring trickled down to the departmental and municipal levels.  Local health partners (Departmental Health Services) appreciated the training and technical assistance they received in results-based management.

The Performance Measurement Framework (PMF) for 2003-2007 was developed at a time when CIDA was experimenting with different models.  The format used did not provide a clear overview of planned results with measurable indicators.  In addition, baseline data was not established, thus it was difficult to monitor and evaluate results at the program level.  Results could not be assessed at the impact level, as is often the case.

Nevertheless, results-based management principles and practices were applied quite systematically at the project level.  Most logic models followed results-based management principles, although some outcome statements were too ambitious and lacked realistic indicators.  The program often monitored results of bilateral funding to pooled funds with a system harmonized with that of partner institutions.  These systems proved effective in promoting CIDA’s results-based management values.

CIDA reporting requirements were not consistent from one project to the other.  The quality of project reports was generally good, although some focused more on activities and outputs than on results achieved.

4.3.2. Demonstrated improvement of monitoring and evaluation

At the project level, monitoring and evaluation became more systematic after 2007, based on a recommendation from the 2007 Country Program Evaluation.  Some projects had long-term monitoring strategies and were being tracked by the program and the Program Support Unit.  Eleven of the 24 projects (or project components) in the sample underwent independent evaluations.  The evaluation reports that were available were generally of sufficient quality to derive significant and relevant results, and presented qualitative results in an adequate manner.  Demonstrating concrete results remained a challenge, however, especially for governance-related projects.

4.3.3. The program fostered effective risk management at the project level

The 2007 Country Program Evaluation recommended that the program should place “greater effort on risk analysis and contingency planning and on periodic review of the project design to ensure that the project continues to be supported by the government and continues to be sustainable”.  The program took some steps in response.  The 2010-2015 CDPF incorporated more regular risk analysis and mitigation into the Performance Management Framework.  Risk mitigation strategies were also updated annually in project performance reports. 

Risk management at the country-program level, however, had some shortcomings:

At the project level, risks were identified and mitigated quite effectively.  For example, the program used a risk management tool for small pilot projects, which tested new partners before they got approval to receive larger bilateral support.

4.3.4. Information and knowledge management – an important challenge

Data collection proved to be a challenge during the evaluation.  The program generated and accumulated an impressive quantity of useful information, data and documents.  However, this knowledge was not well organized or easily accessible.

The program did not have an effective knowledge management system and did not effectively communicate or disseminate information about its works and achievements.  There was no corporate process to analyze or systemize lessons learned.  Many stakeholders reported that the Agency has some good stories to tell about its work in Bolivia but they had difficulty finding basic information about the Bolivia Country Program’s strategy and policies in Bolivia.

5.0 Conclusions

The evaluation provides evidence that the program helped improve living conditions and standards in the key programming sectors of health, governance, and SEG.  Factors contributing to success include a focus on building consistent and long-term relationships with Bolivian counterparts, and alignment of development programming under Bolivian priorities with other donors.

Policy dialogue, in particular, proved effective in facilitating results across all sectors of priority.  In governance and in gender equality, a participatory approach allowed the program to negotiate the scope of strategic technical assistance and helped to secure government, donor and civil society organization participation.  Canadian interventions contributed directly to the strengthening of important national institutions, including the Auditor General, the National Electoral Court, and the Ombudsman.

On the basis of evidence from the sample of projects, key interviews, and relevant documentation, the following conclusions can be drawn on the evaluation criteria:

Relevance: While navigating changes to political leadership in Bolivia and corporate orientation at CIDA, the program remained relevant to Canadian strategic priorities and Bolivian needs.  It delivered aid across priority sectors according to the 2003-2007 Country Development Programming Framework and the revised 2009 Country Program Strategy.  A major strength was the focus on building consistent, long-term relationships with key partners and beneficiaries through a highly participatory process.

Effectiveness: There were good development results achieved over the review period, particularly in SEG where the hydrocarbons project contributed to the country’s ability to benefit and redistribute revenues by establishing a more transparent and efficient tax and royalty collection system.  The program also contributed to better health services for the most vulnerable, and facilitated decentralization of services and strengthened governance and democratic institutions. 

Sustainability: Encouragement of ownership by Bolivian stakeholders and other donors built on Bolivia’s capacity to sustain some of the program’s investments.  This facilitated improvement in long-term institutional capacity, particularly in the hydrocarbons sector.  However, program investments in governance and health were rated as less sustainable due to the difficulties of developing strong local administration, particularly in the health sector.

Gender Equality: During the early part of the review period, integration of gender equality across the portfolio was minimal given that the local gender specialist was focused on managing CIDA’s Gender Equality Fund.  In 2010, Holland became the lead on the Gender Equality Fund, which allowed the gender specialist to improve the integration of gender equality as a crosscutting issue in the program.  The program did contribute to important results through its gender-focused programming, such as increased political participation by women, strengthened capacity of women’s organizations, and improved protection of women’s rights.  The program also played a leadership role in promoting policy dialogue on gender equality with other donors, Bolivian government partners and non-governmental organizations.

Environmental Sustainability: The evaluation demonstrated that in Bolivia, applying the Canadian Environmental Assessment Act was not enough to ensure full integration of environmental considerations across the program.  A case in point was water and sanitation where the focus was more on securing water supply than in promoting conservation and integrated management of water resources.  The program did nonetheless provide some good examples of promoting and integrating the environment, particularly in the SEG sector.  There was still room for improvement in the mainstreaming of environmental issues in the country program and projects, as recommended by the strategic environmental assessment conducted in 2009. 

Coherence: The program demonstrated good examples of external coherence with other donor programs through involvement in multiple pooled funds and by participating effectively in a key multi-donor coordination group.  The evaluation also found relatively good internal coherence between the Geographic Program Branch and Partnership with Canadians Branch on some key projects, although a more systematic approach to coordination with civil society organizations would have been useful.

Efficiency: Based on document review including the 2011 Americas Program benchmarking exercise, the program performed favourably in terms of cost-efficiency, when compared to similar CIDA programs in other middle-income countries.  As well, there was an appropriate blend of larger pooled fund investments with relatively lower overhead cost ratios, and smaller but innovative remote area health interventions with relatively higher overhead cost ratios.  However, several limitations, also evident in other Country Programs, affected efficiency during the evaluation period, including high staff turnover, delays in project approval and limited delegation of spending authority.

Management Principles: The program demonstrated strong evidence of adherence to the principles of the Paris Declaration on Aid Effectiveness (ownership, harmonisation and alignment).  Policy dialogue was also a strength, facilitated through involvement in a number of pooled funds.  Even with shifting political circumstances in Bolivia and new articulation of corporate priorities at CIDA, general alignment of program interventions with Bolivian priorities was maintained.

Performance Management: At the project level, results-based management principles, monitoring, evaluation and risk management practices were generally applied systematically.  There was also a program level performance management framework in place, but it did not provide a sufficient overview of planned results with measurable indicators, and baseline data was not established, which limited the ability to articulate broader achievements and results. Knowledge management, communication and dissemination of information about the program’s works and achievements could have been stronger.

6.0 Recommendations

A number of recommendations arise from the evaluation findings and conclusions:

Annex 1: Detailed Project Sample

Democratic Governance Sector
Project NumberBranchProject TitleStart YearEnd YearInvestment typeDisbursements to Bolivia
Disbursements – Democratic governance projects in sample: $20.01 M (44% of total)
Total disbursements – Democratic governance projects (2005-2010): $45.64 M
A020910018GeographicTrilateral Criminal Defence/Barreau Qc20012010Projects$0.32 M
A032846001GeographicPeacebuilding Initiative with CNE20052007Projects$5.50 M
A030740001GeographicCESO-Bolivia20012007Projects$1.21 M
A030422001GeographicInstitutional Support Defensor del Pueblo20002007Program-based approaches$1.23 M
A032217001GeographicStrategic Governance Mechanism (SGM) – Basket Fund20032013Program-based approaches$11.34 M
A033735PRGPartnershipMunicipal Partnership Program20072010Programs$0.41 M
Health Sector
Project NumberBranchProject TitleStart YearEnd YearInvestment typeDisbursements to Bolivia
Disbursements – Health projects in sample: $22.73 M (60% of total)
Total disbursements – Health projects (2005-2010): $37.70 M
A031442001GeographicPASS-Support for the Ministry of Health and Sport (MSD)20012013Program-based approaches$1.50 M
A031442004GeographicPASS - National Fund for Productive and Social Investment (FPS)20012013Programs$3.41 M
A032896001GeographicPACSAS: Support to UNICEF with Sweden and Netherlands20062011Program-based approaches$3.33 M
A033957001GeographicMSD Budgetary Support and Evaluation20082014Program-based approaches$2.20 M
A033957002GeographicSupport to the Malnutrition Zero Program: MI Micronutrient Component20082014Programs$2.72 M
A033957003GeographicUNICEF – UN Joint Program to Support the Malnutrition Zero Program20082014Programs$4.00 M
S063421001PartnershipUQAM-Écodéveloppement communautaire20052013Projects$1.05 M
S061266PRGPartnershipAUCC-UPCD Program Tier 2 – Final Phase20012012Programs$0.30 M
S062525PRGPartnershipSave the Children 2006-09 - 5 Years Prog20042011Programs$1.87 M
S062900PRGPartnershipPlan International Canada20052010Programs$2.36 M
Economic Growth Sector
Project NumberBranchProject TitleStart YearEnd YearInvestment typeDisbursements to Bolivia
Disbursements – Economic Growth projects in sample: $15.76 M (68% of total)
Total disbursements – Economic Growth projects (2005-2010): $23.12 M
A030420001GeographicBolivia Hydrocarbon Regulatory Assist.20022014Projects$7.48 M
S063681PRGPartnershipProgramme Coop- 2007-201220072012Programs$1.32 M
S061681PRGPartnershipContribution du programme/Program Contrb20022006Programs$0.81 M
S062447PRG
S064520PRG
PartnershipCESO – Program Support 2004-2009
CESO 2009-2014 Volunteer Cooperation Prg
2004
2009
2009
2014
Programs$0.87 M
$0.25 M
S062445PRG
S064494PRG
PartnershipJoint WUSC-CECI Program Support
WUSC/CECI 2009-14 Building on Success
2003
2009
2009
2014
Programs$4.23 M
$0.81 M
Other or Multisector
Project NumberBranchProject TitleStart YearEnd YearInvestment typeDisbursements to BoliviaSector
Disbursements – Other or Multisector projects in sample: $1.77 M (13% of total)
Total disbursements – Other or Multisector projects (2005-2010): $13.20 M
M012532001MultilateralBOLIVIA FLOODS 2007 / WFP EMOP10616.020072008Projects$0.50 MEmergency assistance
A032504001GeographicGender Equality Basket Fund - Bolivia20042009Program-based approaches$0.02 MMultisector
A032298001GeographicAndean Region Gender Equality Fund20042010Local Funds$1.25 MMultisector

Annex 2: List of Documents Reviewed

General Documents

CIDA

Scoping Mission Report.

COCAB

Government of Bolivia

Other Bilateral and Multilateral Donors

Sector/Programming Area Documents

Democratic Governance
Health
Economic Growth

Crosscutting Issues Documents

Gender Equality
Environment

Other Documents

Project Documents by Sector

Democratic Governance

Project: A020910018 – Trilateral Criminal Defence/Barreau Qc

Planning Documents

Financial Reports

Reports

Project: A030421001 – Public Sector Reform (Bolivia)

Planning Documents

Financial Reports

Project: A030740001 – CESO Bolivia

Planning Documents

Reports

Project: A032846001 – Peacebuilding Initiative with CNE

Planning Documents

Financial Reports

Project: A030422001 – Institutional Support Defensor del Pueblo

Planning Documents

Financial Reports

Project: A032217001 – Strategic Governance Mechanism (SGM)

Planning Documents

Financial Reports

Reports

Project: A033735PRG – Municipal Partnership Program 2007-2010

Planning Documents

Reports

Health

Project: A032896001 - PACSAS

Planning Documents

Reports

Evaluations

Project: A031442001 – MSD (PASS)

Planning Documents

Financial Reports

Project: A031442004 – FPS (PASS)

Planning Documents

Financial Reports

Project: A033079001 – Grant to PAHO-Bolivia

Planning Documents

Project:  A033957001 – MSD Budgetary Support and Evaluation

Planning Documents

Reports

Project: A033957002 – MI Micronutrient Component

Planning Documents

Reports

Project: A033957003 – UNICEF Joint Program to Support the Malnutrition Zero Program

PlanningDocuments

Financial Reports

Reports

Project: S063421001 – UQAM – Écodéveloppement communautaire

Planning Documents

Reports

Project: S061266PRG – AUCC-UPCD Program Tier 2

Planning Documents

Financial Reports

Reports

Project: S062525PRG – Save the Children 2006-09

Planning Documents

Financial Reports

Reports

Project: S062900PRG – Plan International Canada 2005-10

Reports

Economic Growth

Project: A030420001 – Bolivia Hydrocarbon Regulatory Assist.

Planning Documents

Reports

Project: A021625001 – Bolivia Public Sector Reform Mining Comm

Planning Documents

Financial Reports

Reports

Project: S063681PRG – Programme Coop 2007-2012

Project: S061681PRG – Contribution du programme/Program Contribution

Project: S064520PRG – CESO 2009-2014 Volunteer Cooperation Prg

Project: S062447PRG – CESO Program Support 2004-2009

Project: S064494PRG – WUSC-CECI Program Support

Project: S062445PRG – Joint WUSC-CECI Program Support

Gender Equality

Project: A032298001 – Andean Region Gender Equality Fund

Planning Documents

Financial Reports

Reports

Project: A032504001 – Gender Equality Basket Fund- Bolivia

Planning Documents

Reports

Environment
Emergency Assistance

Project: M012532001 – Bolivia Floods 2007/WFP EMOP 10616.0

Planning Documents

Reports

Project: M012803001 – Bolivia Floods WFP Flash Appeal 2008

Planning Documents

Reports

Annex 3: List of Persons Interviewed

CIDA - HQ and Bolivia

Name: Agnes Skornicz
Position: International Development Project Advisor (EGS)
Organization: CIDA

Name: Alberto Palacios-Hardy
Position: Former Head of Aid in La Paz
Organization: CIDA

Name: Amelie Pruneau
Position: A/Manager, Youth Participation (ETS)
Organization: CIDA

Name: Andrew Scyner
Position: Head of Aid, La Paz
Organization: CIDA – La Paz

Name: Anne-Marie Hodgson
Position: Program Policy Analyst
Organization: CIDA

Name: Chantal Boucher
Position: Former Project Officer – Bolivia, Peru
Organization: CIDA

Name: Diane Harper
Position: Director (SPCS)
Organization: CIDA

Name: Eileen Kilgour
Position:
Organization: CIDA

Name: Frances Cosstick
Position: International Development Project Advisor (HS)
Organization: CIDA

Name: Frantz Pierre-Jerome
Position: Evaluation Manager
Organization: CIDA

Name: Hélène Giroux
Position: Director General
Organization: CIDA

Name: Janet Ferreira
Position: International Development Project Advisor (ES)
Organization: CIDA

Name: Jennifer Thompson
Position: Former PWCB project officer for CLWR
Organization: CIDA

Name: Jim Sutherland
Position: Former BCP manager
Organization: CIDA

Name: John Lok
Position: First Secretary, Cooperation - Governance
Organization: CIDA – La Paz

Name: Jonathan Laine
Position: Program Manager, Bolivia Program
Organization: CIDA

Name: Lilly Nicholls
Position: Director
Organization: CIDA

Name: Lise Filiatrault
Position: Regional Director General (BMV)
Organization: CIDA

Name: Louise-Marie Thomassin
Position: International Development Projects Advisor
Organization: CIDA

Name: Luc St-Laurent
Position: First Secretary, Cooperation - Health
Organization: CIDA – La Paz

Name: Maria Javenia-Mackenzie
Position: International Development Project Advisor (PIES)
Organization: CIDA

Name: Marie-Eve Castonguay
Position: International Development Project Advisor (CGHRS)
Organization: CIDA

Name: Michelle Guertin
Position: Evaluation Team Leader
Organization: CIDA

Name: Minh Tien Nguyen
Position: International Development Project Advisor (ESS)
Organization: CIDA

Name: Moreno Padilla
Position: Senior Environment Specialist
Organization: CIDA

Name: Paul Ragusa
Position: Oil and Gas Specialist
Organization: CIDA

Name: Pierre J. Tremblay
Position: Evaluation manager
Organization: CIDA

Name: Rachel Bruneau
Position: Senior Development Officer
Organization: CIDA

Name: Rémi Turmel
Position: International Development Project Advisor (VIS)
Organization: CIDA

Name: Steve Jaltema
Position: Former BCP officer in La Paz
Organization: CIDA

Name: Darren Rogers
Position: former BCP officer
Organization: CIDA

Name: Susan Learoyd
Position: Program Policy Analyst
Organization: CIDA

Name: Vicky Laramee
Position: International Development Project Advisor (VIS)
Organization: CIDA

Unidad de Apoyo de Servicios de la Cooperacion Canadiense (UASCC)

Name: Carmen Estepa
Position: Health Specialist
Organization: UASCC

Name: Eliana Gallardo
Position: Gender Specialist
Organization: UASCC

Name: Gustavo Bracamonte
Position: Director
Organization: UASCC

Name: Mike Guerra
Position: Systems Administrator
Organization: UASCC

Name: Urzula Montes de Oca
Position: Governance Specialist
Organization: UASCC

Government of Bolivia representatives & Parastatal Organizations

Name: Ana María Aguilar
Position: Former responsible of the National Nutritional Council
Organization: Ministry of Health and Sports

Name: Eduardo Alarcón,
Position: Vice Minister, Energy & Production
Organization: Ministry of Hydrocarbons and Energy

Name: Enrique Torrico
Position: Chief of Technical Norms and Community Development
Organization: Ministerio de Agua y Saneamiento Básico

Name: Frank Molina
Position: Vice Minister, Energy Development
Organization: Ministry of Hydrocarbons and Energy

Name: Gustavo Zárate
Position: Director, Energy Development
Organization: Ministry of Hydrocarbons and Energy

Name: Guy Vargas
Position: Responsible of Planning Unit
Organization: Ministry of Health and Sports

Name: Jaime Villanueva Cardozo
Position: National Climate Change Program Coordinator
Organization: Ministry of Environment and Water

Name: Jorge Quiroga
Position: Gas Pricing Specialist
Organization: Ministry of Hydrocarbons and Energy

Name: Juan Pablo Cardozo.
Position: Vice Minister
Organization: Ministry of Environment and Water

Name: Leslie La Torre
Position: Coordinator of Fortalessa Project
Organization: Ministry of Health and Sports

Name: Hortensia Jimenez
Position: Vice Minister, Electricity & Alternate Energy
Organization: Minister of Hydrocarbons and Energy

Name: Maria Cristina Arellano
Position: Former Director General of Social and Environmental Management
Organization: Minister of Hydrocarbons and Energy

Name: Martin Maturano
Position: Vice Minister
Organization: Ministry of Health

Name: Mr. Jorge Rojas
Position: Technical Advisor APG Natural Resources
Organization: Minister of Natural Resources

Name: Mrs. Margot Ayala
Position: Director General Energy & Production),
Organization: Ministry of Hydrocarbons and Energy

Name: Mr. Nelson Bartolo
Position: Manager, APG Natural Resources
Organization: Ministry of Hydrocarbons and Energy

Name: Mrs. Norka Gonzales
Position: Manager Royalties, ex Intern)
Organization: Ministry of Hydrocarbons and Energy

Name: Isabel Chopitea
Position: Former Director General
Organization: Vice Ministry of Hydrocarbons and Energy

Name: Adriana Viera
Position: BHAP Consultant Industrial Engineer responsible for National Energy Balance
Organization: Working for Gov through BHAP project

Name: Jorge Leytón
Position: Economic Modeling Specialist Consultant
Organization: Working for Gov through BHAP project

Name: Xavier Eduardo Barriga
Position: Director General of Social and Environmental Management
Organization: Working for Gov through BHAP project

Name: Carlos Villegas
Position: President
Organization: Yacimientos Petrolíferos Fiscales Bolivianos (YPFB)

Name: Donald Osinga
Position: General Manager - Exploration
Organization: YPFB

Name: Erika Defillipis
Position: Former GSH Supervisor
Organization: YPFB

Name: Luis Carlos Sanchez
Organization: Director of Exploration & Promotion
Position: YPFB

Name: María Luisa Auza
Position: Director of Planning
Organization: YPFB

Name: Mr. Fernando Salazar
Position: VP Contracts Administration & “Fiscalization”
Organization: YPFB

Name: Mr. Javier Fernández
Position: General Manager Planning & Finance
Organization: YPFB

Name: Mrs. Leila Mokrani
Position: Advisor to YPFB Andina, Former Ministry Director
Organization: YPFB

Name: Zulema Espejo
Position: Director of International Affairs
Organization: YPFB

Name: Blanca Laguna
Position: International Relations Director
Organization: Ombudsman’s Office

Name: Rielma Mencias
Position: Former Ombudsperson
Organization: Ombudsman’s Office

Name: Waldo Albarracín
Position: Former Ombudsman
Organization: Ombudsman’s Office

Name: Armando Paredes
Position: Management Director
Organization: National Statistics Institute

Name: Willy Miranda
Position: Statistics Officer
Organization: National Statistics Institute

Name: Martha Oviedo
Position: Former Director
Organization: National Statistics Institute

Name: Silvia Terrazas
Position: Former Technical Officer
Organization: National Statistics Institute

Name: Antonio Costas
Position: Former President of CNE; now Director of SEGIP
Organization: National Electoral Court

Name: Salvador Romero
Position: Former President of CNE
Organization: National Electoral Court

Name: Pavel Perez
Position: Former CIDA Project Coordinator
Organization: Auditor General’s Office

Name: Jorge Leyton
Position: Judicial Director
Organization: Public Defender’s Office

Name: Alberto Condori
Position: Former CIDA Project Coordinator
Organization: Public Defender’s Office

Name: Irma Campos
Position: Former Director of gender unit
Organization: Vice-Ministry of Equal Opportunities

Donor Representatives

Name: Sergio Martin-Moreno
Position: Coordinador general
Organization: Spanish Agency for International Development Cooperation

Name: Adam Behrendt
Organization: Especialista desarrollo social
Organization: World Bank

Name: Mai Le
Position: Coordinator
Organization: COCAB

Name: Julio Pedroza
Position: Belgium Cooperation, Technical Advisor
Organization: Belgian Development Agency

Name: Alain Schmitz
Position: First Secretary
Organization: Embajada de Bélgica

Name: Lari-Henrik Worsof
Position: Head of Aid
Organization: Embajada Dinamarca

Name: Jerome Dubois-Nercent
Position: Head of Aid
Organization: Embajada Francia

Name: Alejandra Márquez
Position: Expert on Emancipation
Organization: Netherlands Embassy

Name: Anke van Dam
Position: Head of Cooperation
Organization: Netherlands Embassy

Name: Janet Trujillo
Position: Environment Expert
Organization: Netherlands Embassy

Name: Ann Stodberg
Position: Business Development
Organization: Embajada Suecia

Name: Isabel Ascarrunz
Position:
Position: Water & Sanitation and Environment Expert
Organization: Embajada Suecia

Name: Michel Thieren
Position: Representative
Organization: PAHO/WHO

Name: Christian Jette
Position: Democratic Governance Coordinator
Organization: UNDP

Name: Victor Hugo Bacarreza
Position: -
Organization: UNDP

Name: María de Angeles Loayza
Position: former CIDA Project Director
Organization: UNDP

Name: Rocio Chain
Position: Environment and Climate Change Programme Officer
Organization: UNDP

Name: Irma Peredo
Position: PACSAS Coordinator
Organization: UNICEF

Name: Marcolui Corsi
Position: Representative
Organization: UNICEF

Name: Rosario Quiroga
Position: Chief – Health Sector
Organization: UNICEF

Name: Isamaria Wallendcht
Position: Programme Officer
Organization: European Union

Name: Lawrence Odle
Position: -
Organization: USAID

Name: Wayne Nilsestem
Position: Director
Organization: USAID

Name: Stephanie Bellot
Position: -
Position: Organization de Planificacion y Desarrollo

Name: Navil Agrament
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Harley Rodríguez
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Jaime Garron
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Juan Carlos Soria
Position: La Paz Monitor
Organization: WFP

Name: Paolo Mattei
Position: Representative
Organization: WFP

Name: Sergio Torres
Position: Head of Programme
Organization: WFP

Bolivian Non-Government Organizations and Civil Society Organizations, Universities, Municipalities

Name: Maria Eugenia Rojas
Position: former Executive Director
Organization: Asociación de Concejalas de Bolivia

Name: Bertha Acarapi
Position: former Municipal Councillor in El Alto
Organization: Asociación de Concejalas de Bolivia

Name: Ana María Aranibar
Position: Director
Organization: Cumbre del Sajana

Name: Zonia Fabiani
Position: Director
Organization: Centro Yanapasiñani

Name: Dino Palacios
Position: former Director
Organization: Federación de Asociaciones Municipales de Bolivia

Name: Alicia Canaviri
Position: Executive Director from 2005 to 20010
Organization: CDIMA

Name: Teresa Condori
Position: Current Executive Director
Organization: CEDIMA

Name: Orlando Velásquez
Position: President – Río Pilcomayo Defence Council
Organization: CODERIP/CLWR

Name: Aquilino Villca
Position: Municipal Mayor
Organization: Curahuara de Carangas

Name: Alexia Escobar
Position: Coordinator
Organization: FCI Bolivia

Name: Fanny Zambrana
Position: Consultant for “Fundación para el Desarrollo”
Organization: FUNDES/CLWR

Name: Franklin Gonzales
Position: Consultant for “Fundación para el Desarrollo”
Organization: FUNDES/CLWR

Name: Andrés Condori
Position: Pichacani community promoter, Commercial Secretariat of the Producers Association of APROARTE PACHAMAMA
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ely Borda
Position: Chief financial Adminstrator (Jefatura Administrativa financiera)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Francisco Montalvo
Position: Community Promoter of Ura Rodeo
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Martha Puma
Position: Community Promoter of Chimpa rodeo
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Patricia Rivero
Position: Project officials (Jefatura de Planificación e Investigación)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ramiro Morales
Position: Project Technical Expert “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism).
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ricardo Guzman
Position: Project Representative “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism).
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Teodora Mamani
Position: Project Technical Expert “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: José Cepeda
Position: Project Representative
Organization: PRODEC/CLWR

Name: Nicolás Barrios
Position: Facilitator “Proyecto de Desarrollo Campesino”
Organization: PRODEC/CLWR

Name: Olga Carlos
Position: Facilitator “Proyecto de Desarrollo Campesino”
Organization: PRODEC/CLWR

Name: Paulina Sánchez
Position: Facilitator
Organization: PRODEC/CLWR

Name: Aura Teresa Barba
Position: National Director of the Project ECOMINGA
Organization: Universidad Gabriel René Moreno

Name: Gabriela Flores
Position: Student of the Master Degree Module
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Julio Torres
Position: Coordinator of the AUCC Project from the Universidad Mayo Real y Pontificia de San Francisco Xavier
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Ronald Zapata
Position: Student of the Master Degree Module
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Lidia Zenzano
Position: Special Project Coordinator
Organization: Universidad Nur

Name: Mr. Antonio Cabrera
Position: Former Associate Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Mr. Freddy Blackutt
Position: Associate Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Mr. Jorge Nunez
Position: Former Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Offman Blanco
Position: Dean Faculty Integral De Chaco
Organization: UAGRM

Name: -
Position: Presidenta
Organization: Movimiento de Economía Social y Comercio Justo de Bolivia

Canadian Non-Government Organizations and Implementing Agencies

Name: Lucie Laplante
Position: Former CIDA Project Coordinator
Organization: Quebec Bar

Name: Antoinette Chibi
Position: Agente de programa

Organization: AUCC

Name: Jen Avaz
Position: Manager UPCD

Organization: AUCC

Name: Angela Herbas
Position: Office Manager Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Bernarda Sarué.
Position: Community specilaist for environment and social monitors training programme of the Faculty Integral de Chaco
Organization: BHAP

Name: Jim Roy
Position: Canadian Advisor on Exploration Economic Model Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Nasser Akhtar
Position: Resident Project Coordinator - Bolivia Canada Hydrocarbon Project BCHP
Organization: BHAP

Name: Dr. Percy Garcia
Position: Canadian Advisor on Socio-Economic issues Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Verónica Guzmán
Position: Gender Consultant
Organization: BHRAP

Name: Mary Pullen
Position: Project Manager
Organization: Canadian Crossroads

Name: Cecilia Requeña
Position: Former Bolivian Project Director
Organization: CESO

Name: Erik Meier
Position: Former Bolivian Project Director
Organization: CESO

Name: Fernando Gamon
Position: Project Manager
Organization: CLWR

Name: Limbert Paredes
Position: Business Manager Environmental Services Association of Engineers Potosí
Organization: CLWR

Name: Luisa Fernanda Velasco
Position: Program Manager
Organization: CUSO

Name: Mai Le
Position: Cooperante – COCAB
Organization: CUSO

Name: Francesco Gatta
Position: Nuevas iniciativas
Organization: CARE

Name: Aleja Apaza
Position: President of Traditional medicine
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Dr. Franco Rocha
Position: Hospital doctor
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Gragoria Calle
Position: Traditional midwife
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Jose Luis Castro
Position: Tamani
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Justina Condori
Position: (Mamathalla: autoridad originaria)
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Ruth Bolaños,
Position: Coordinador of the Project Un aguayo para un Parto sin Riesgo
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Sebastian Colque
Position: Tamani: Local authority
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Sofía Porco
Position: Health councillor
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Teodocia Chuquichambi
Position: (Sulka: Autoridad Originaria)
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Carlos Ramallo
Position: Bolivian representative
Organization: CESO

Name: Saley Eid
Position: Program manager
Organization: Child Fund

Name: Wendy McFarren
Position: Director
Organization: Child Fund

Name: Jhonny López
Position: Exectutive Director
Organization: CIES

Name: Roxana Rios
Position: Responsable educative
Organization: CIES

Name: Wilma Quinteros
Position: President
Organization: Movimiento de Economía Social y Comercio Justo de Bolivia

Name: Christian Tremblay
Position: Representative
Organization: Oxfam Quebec

Name: Sangita Patel
Position: Programme Manager
Organization: Plan Canada

Name: Daniel Rojas
Position: Asesor de programas salud y desarrollo infantil
Organization: Plan International

Name: Eda Quispe
Position: Health Coordinator of “Camina” Project
Organization: Plan International

Name: Gustavo Tapia
Position: Research and M/E Coordinator
Organization: Plan International

Name: Ignacio Canaviri
Position: Coordinator of “Camina” Project in Sica Sica Municipality
Organization: Plan International

Name: Ximena Ostria
Position: Gerente nacional de programas
Organization: Plan International

Name: Ana Amador
Position: Coordinadora salud
Organization: Samaritan´s Purse

Name: Alejandra Villafuerte
Position: Program Manager
Organization: Save the Children

Name: Carlos Villaroel
Position: Cochabamba Manager
Organization: Save the Children

Name: Luis Ramirez M
Position: Executive Director
Organization: Save the children

Name: Marlen Mondaca
Position:
Organization: Save the Children

Name: Natalie Folz
Position:
Organization: Save the Children

Name: Philippe Demers
Position: Project Coordinator
Organization: Socodevi

Name: Roberto Muñoz
Position: Technical Specialist and site manager for oregano project
Organization: Socodevi

Name: Sebastian Valdivieso
Position: Project Director
Organization: Socodevi

Name: Eduardo Alfaro
Position: National Bolivian Project Manager
Organization: Uniterra en Bolivia

Name: Rita Cano
Position: Program Officer
Organization: Uniterra en Bolivia

Name: Aura Teresa Barba
Position: Teacher
Organization: Universidad Autónoma « Gabriel René Moreno »

Name: Frida Villarreal
Position: Coordinator
Organization: Université du Québec à Montréal (UQAM)

Name: Lucie Sauvé
Position: Project Director
Organization: Université du Québec à Montréal (UQAM)

Name: Mr. David Bethune
Position: Canadian project Director
Organization: University of Calgary

Name: Katherine Murillo
Position: Program Manager, MPED Americas
Organization: Federation of Canadian Municipalities (FCM)

Beneficiaries

Name: Vicente Ojeda Velasquez
Position: President of the CHW
Organization: Asociación de Redes de Responsables Comunitarios de Salud

Name: Carla Reque
Position: Female Intern in 2011-2012 (now ministry employee)
Organization: BHAP

Name: Laura Quinteros
Position: Female Intern in 2012-2013
Organization: BHAP

Name: Carla Derpic
Position: Female Intern in 2011-2012 now ministry employee
Organization: BHAP

Name: Virginia Rendon:
Position: Intern in 2007-2008 became the Director at the Ministry, currently working in YPFB Planning Department.
Organization: BHAP

Name: 50 Students and teachers
Position: Participants and teachers enrolled in the environment-social monitoring programme Camrini
Organization: BHAP

Name: 15 Secondary school girls
Position: Students in fifth and sixth year and four current college students and former POPL, Participants in the Gender Equality Program for Schoolgirls to encourage career selection in Science and Technology
Organization: BHAP

Name: Bertha Gutiérrez
Position: Professor at Isaac Maldonado College participant in POPL
Organization: BHAP

Name: Erika Barrientos
Position: Second semester Petroleum engineer student and graduate of the POPL program
Organization: BHAP

Name: Giovanna Atila
Position: 8th semester university student in petroleum engineering and graduate of the POPL program
Organization: BHAP

Name: Jaime Leaño
Position: Professor at Augusto Villazón College and participant in the POPL program
Organization: BHAP

Name: Marlene Fernández
Position: Professor at Isaac Maldonado college and participant in the POPL program
Organization: BHAP

Name: Mary Elisa Justiniano
Position: Professor at Monseñor Salvatierra college and participant in the POPL
Organization: BHAP

Name: Selenne Cabrera
Position: Second semester university student studying petroleum engineer and graduate of the POPL program
Organization: BHAP

Name: Josefina Nina
Position: Mother with a recently born child
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Representative of Inca Pampa
Position: Women’s association of Ecological producers Inca Pampa “ADEMPROEINPA”
Organization: CLWR

Name: Representative of La Mendoza
Position: Producers from La Mendoza Women’s Association “AMUPROLAMEN”
Organization: CLWR

Name: Representative of Poco Poco
Position: Producers from the Valle de Poco Women’s Association “AMPROVPP”
Organization: CLWR

Name: Representative of Soroma
Position: Soromo Producers Women’s Association “AMUPRODEV”
Organization: CLWR

Name: Representative of Sotomayor
Position: Women’s Association of Vallas Sotomayor “ADEMVAS”
Organization: CLWR

Name: Representative of Tasa Pampa
Position: Association of Women Producers Tasa Pampa “AMPTP”
Organization: CLWR

Name: Representative of Uyuni
Position: Vallunas Uyuneñas Association “AVAU”
Organization: CLWR

Name: Representative of Viña Pampa
Position: Association of Women Producers and Handcraft producers Viña Pampa “APAVIP”
Organization: CLWR

Name: Representative of Viña Quemada
Position: Association of women entrepreneurs of Viña Quemada “AMEVIQ”
Organization: CLWR

Name: Andres Torres
Position: President of the Agrarian Union of Peasants for Aroma Province
Organization: Municipality of Sica Sica

Name: Arminda Barrios Quispe
Position: Vice-President of the CHW Association of Community Representatives of Health Networks
Organization: Municipality of Sica Sica

Name: Erika Tintaya
Position: Doctor of the Conani Health Centre
Organization: Municipality of Sica Sica

Name: Felix Calle
Position: Municipal Health Representative
Organization: Municipality of Sica Sica

Name: Franklin Flores
Position: President of the Municipal Council
Organization: Municipality of Sica Sica

Name: Hilaria Herrera
Position: Consejala – Health Commission
Organization: Municipality of Sica Sica

Name: Maria Elena Callejas
Position: Worker at the Conani Health Centre
Organization: Municipality of Sica Sica

Name: Vilma Colque Villanueva
Position: Vice-president of the Municipal Council- Tourism and Education Commission
Organization: Municipality of Sica Sica

Name: Walter Mayzo Alandia
Position: Mayor
Organization: Municipality of Sica Sica

Name: Zulma Velasquez
Position: Health Centre Conani
Organization: Municipality of Sica Sica

Name: Director
Position: CA de la Coop de Tomina
Organization: Socodevi

Name: Farmer 1 Raul
Position: Participant in the oregano project
Organization: Socodevi

Name: Farmer 2 Javier
Position: Participant in the oregano project
Organization: Socodevi

Name: Farmer 3 Juan Luis
Position: Participant in the oregano project
Organization: Socodevi

Annex 4: Evaluation Matrix

1. Relevance

1.1 Consideration of Bolivia’s development priorities (National Development Plan)

Question: To what extent are CIDA investment choices and related results relevant to Bolivia’s NDP with respect to priorities and development policies?

Indicator: Degree of compatibility of the BCP’s priorities and Bolivia’s priorities and development policies mentioned in the NDP and PRSP.

Question: To what extent are CIDA investment choices in Bolivia and related results relevant to the achievement of Millennium Development Goals (MDGs)?

Indicator:

Question: To what extent is CIDA involved in the adequate sectors given Bolivia’s priorities?

Indicator: Degree of compatibility of the BCP’s priorities and Bolivia’s strategic orientations and priority sectors mentioned in the NDP

Question: To what extent were CIDA’s choices of investments strategic within the selected sectors?

Indicator: Degree of alignment between the NDP and CIDA’s investment choices

Question: To what extent were changes in the Bolivian context taken in account by adequate changes in the BCP?

Indicator: Type of adjustments brought to Canadian programming in light of the evolution of the context in Bolivia (socio-economic, political, environmental and technological)

Question: With the collaboration of other donors, to what extent has CIDA ensured the development of a framework to monitor results of the NDP, in order to ensure that its programmatic choices be strategic in view of the objectives and priorities, including poverty reduction?

Indicator:

Question: To what extent does CIDA participate in the monitoring of global development results on the basis of the NDP and does CIDA adopt programmatic decisions based on this analysis?

Indicator: Type of adjustments brought to Canadian programming in light of the strengths and limits with respect to the achievement of objectives and results of the NDP

Question: To what extent is CIDA perceived as adding value in the Bolivian development context? How?

Indicator:

Data sources:

Data collection methods:

1.2 Strategic priorities, Canadian and CIDA policies

Question: Is the BCP aligned with Canadian priorities and policies for South America and Bolivia?

Indicator:

Question:

Indicator:

Question: Are the BCP approaches, initiatives and expected results in line with the comparative advantages (expertise and lessons learned from past experiences) of CIDA in Bolivia?

Indicator: Degree of compatibility of the BCP’s approaches, initiatives, and expected results with expertise and lessons learned from past experiences of CIDA in Bolivia

Data sources:

Question: How does gender equality figure in the BCP? To what extent do the approaches, initiatives and results support the objectives of CIDA’s gender equality policy (1999)?

Indicator:

Question: To what extent were CIDA investment choices and results relevant to environmental sustainability?

Indicator:

Data sources:

Data collection methods:

2. Effectiveness (development results in terms of poverty reduction)

2.1 Planned versus actual results of the program

Question: Planned versus actual results of the program.  What were the results achieved versus those planned within the sample of selected projects?

Indicator:

Question: What were the main success factors or limiting factors that affected results achievement?

Indicator:

Question: What were the main key results achieved regarding the three strategic priorities of the BCP?

Indicator:

Question: To what extent did CIDA develop a gender equality strategy and effectively promote gender equality results through its investments?

Indicator:

Question: To what extent did CIDA develop an environment strategy and effectively promote environmental sustainability through its investments?

Indicator:

Question: Has the selection of delivery mechanisms contributed to achieve planned results in the targeted program areas:

Indicator:

Data sources:

Data collection methods:

2.2 BCP contribution to assist in reducing poverty in Bolivia through institutional development

Question: Was the program equipped with sectoral strategies within strategic objectives with a view to ensuring sound articulation of its investments towards achievement of poverty reduction results?

Indicator:

Question: To what extent was the program strategy designed to address the needs of the poorest populations?

Indicator:

Question: Are the program results in line with the objectives, results and indicators identified in the Bolivian PRSP and the NDP?

Indicator: Degree of compatibility of program strategy objectives with the objectives, results and indicators identified in the Bolivian PRSP and NDP

Question: What are the results achieved in terms of improvement of quality, equity and efficiency of the Bolivian health and water and sanitation sector, and what are the methods in place to measure these results?

Indicator:

Question: What were the results achieved in terms of the democratization and public sector reform and what are the methods in place to measure these results?

Indicator:

Question: What were the results achieved with respect to the inclusive management of strategic sectors such as mining and hydrocarbons, and what are the methods in place to measure these results?

Indicator:

Data sources:

Data collection methods:

3. Sustainability (sustainability of results – degree of dependency on development assistance)

3.1 Time required to generate sustainable results

Question: Is the time allocated for project delivery and achievement of results adequate and sufficient to ensure sustainability of results?

Indicator:

Question: To what extent have risks associated with sustainability of results been adequately identified, analysed and managed?

Indicator:

Data sources:

Data collection methods:

3.2 Absorptive capacity and appropriation by Bolivian partners

Question: What is the degree of dependency of the GoB vis-à-vis donor contributions?

Indicator:

Question: Which partners (public sector—state and decentralized—, civil society and private sector) have performed in such a way as to contribute optimally to sustainability of results?

Indicator:

Question: To what extent is there local ownership to sustain results achieved?

Indicator:

Question: Generally speaking, have the partners planned enough financial resources to cover recurring costs?

Indicator:

Data sources:

Data collection methods:

4. Crosscutting issues (integration of priority themes: gender equality and environment)

4.1 Integration of crosscutting themes in the BCP

Question: To what extent has the BCP and the CDPF strategically and operationally integrated crosscutting themes?

Indicator:

Question: Has the program planned for sufficient resources to ensure operationalization and monitor integration of crosscutting themes?

Indicator:

Data sources:

Data collection methods:

4.2 Planning, monitoring and evaluation processes of crosscutting themes within sample of selected projects

Question: Were concrete strategies to integrate CCI developed at the project level and did the narrative project reports systematically account for results?

Indicator:

Question: What results did the strategies to integrate CCI in project implementation strategies generate, by sector and delivery mechanism?

Indicator:

Question: To what extent has the capacity of the GoB and civil society to take into account CCI been enhanced?

Indicator: Level of awareness of CCI among representatives from the GoB and civil society

Data sources:

Data collection methods:

5. Coherence

5.1 Internal coherence of the program (content and duration of intervention)

Question: To what extent does the BCP constitute the result of an adequate coordination with other international initiatives in Bolivia?

Indicator:

Question: What are the concrete means and measures adopted by CIDA to ensure coherence and optimization of synergies between the sectors/strategic orientations of the BCP?

Indicator:

Question: What are the concrete means and measures adopted by CIDA to ensure coherence and optimization of synergies between the projects within each program area?

Indicator:

Question: What are the concrete means and measures adopted by CIDA to ensure complementarities and synergies between the different funding mechanisms—bilateral (directive, responsive, PBAs), multilateral and partnerships?

Indicator:

Question: Does the distribution of interventions and investments (i.e. distribution of investments by strategic objective) adequately reflect, in a balanced manner, planned results for both strategic objectives of the BCP?

Indicator:

Data sources:

Data collection methods:

5.2 Program coherence with Canada objectives in Bolivia

Question: Does the program implement actions (e.g. policy dialogue, integrated approach framework) to support synergy between different Canadian actors in Bolivia?

Indicator:

Data sources:

Data collection methods:

5.3 Policy dialogue to support programming

Question: Is the policy dialogue at CIDA sufficient to efficiently promote Canadian priorities (human rights, democracy, transparency, gender equality, environment, civil society participation, etc.)?

Indicator:

Question: Does the CIDA policy dialogue with and influence on the GoB create an enabling political context for the development of local networks and partnerships and the strengthening of national and/or local institutions in the BCP targeted sectors?

Indicator: Number, type and quality of CIDA communications with the GoB to create an enabling political context for the development of local networks and partnerships and the strengthening of national and/or local institutions

Data sources:

Data collection methods:

6. Efficiency (management efficiency and capacity to monitor, evaluate and draw lessons learned from programming)

6.1 Management and optimization of the cost-effectiveness ratio

Question: Has there been any delay or gap between the period when a project was approved and the period of its full implementation?

Indicator:

Question: To what extent does the communication framework between CIDA headquarters and the field allow more punctual decision making to ensure the production of results and respect program calendar?

Indicator:

Question: What effect has the shift to program approach and budgetary support had on efficiency of the BCP delivery?

Indicator:

Data sources:

Data collection methods:

6.2 Efficiency of resource allocation and utilization

Question: To what extent did the BCP have access to appropriate human and technical resources to support management in the field and at headquarters?

Indicator:

Question: To what extent did the projects have access to appropriate financial resources to achieve results?

Indicator: Correlation between financial resources disbursed and achievement of results

Data sources:

Data collection methods:

6.3. Cost-effectiveness ratio of project results (management expenses versus operational expenses)

Question: To what extent is the cost-effectiveness ratio between management expenses and operational expenses reasonable and comparable to other donor programs in Bolivia?

Indicator: Adequacy of management expenses versus operational expenses

Question: To what extent has the shift to program approach and budgetary support improved the efficiency of project implementation in terms of resources and time allocated to management?

Indicator:

Data sources:

Data collection methods:

7. Management principles (aid effectiveness)

7.1 Ownership/buy-in

Question: To what extent is the BCP the result of local demand?

Indicator:

Question: To what extent have the projects been conceived to transfer execution leadership to the local partners and to maximize local ownership?

Indicator:

Data sources:

Data collection methods:

7.2 Alignment/harmonization

Question: To what extent do the local institutions control the strategies, management and execution of the BCP?

Indicator:

Question: To what extent have the Bolivian financial institutions been strengthened at the national and local levels?

Indicator:

Question: To what extent have the capacity building initiatives for the GoB been coordinated and harmonized with those of other donors in Bolivia?

Indicator: Number of CIDA BCP capacity building initiatives coordinated and harmonized with those of other donors in Bolivia

Data sources:

Data collection methods:

Question: To what extent has the CIDA BCP been implemented through concerted mechanisms with other donors in Bolivia?

Indicator:

Data sources:

Data collection methods:

8. Performance management (management results including monitoring and evaluation)

8.1 RBM as per CIDA guidelines

Question: To what extent were Canadian cooperation interventions in Bolivia based on past acquired knowledge and lessons learned and to what extent did CIDA try and acquire new knowledge and lessons learned through the BCP?

Indicator:

Question: To what extent have risks associated with Canadian cooperation interventions in Bolivia been adequate?

Indicator:

Data sources:

Data collection methods:

8.2 M&E mechanisms in place to ensure that a RBM approach was used during CIDA BCP implementation

Question: To what extent was the CIDA BCP PMF used to monitor BCP results?

Indicator:

Question: Were the CIDA BCP M&E mechanisms adequate at the project and program levels and were the results of these mechanisms used in decision making?

Indicator:

Question: To what extent has CIDA participated in joint monitoring/evaluation initiatives and respected the harmonization principle in its performance management in Bolivia?

Indicator:

Question: To what extent has CIDA participated in initiatives aimed at building local M&E capacity in Bolivia?

Indicator: Number of M&E initiatives aimed at building local M&E capacity in Bolivia sponsored by CIDA.

Data sources:

Data collection methods:

Annex 5: Program Assessment/Project Assessments

Summary of Scores for Sectors/Crosscutting Issues, by Project

Health
ProjectRelevanceEffectivenessSustainabilityGenderEnvironmentFootnote 18CoherenceEfficiencyManagement principlesPerformance managementAverage
Rating Scale: Very Unsatisfactory (0 to 1); Unsatisfactory (1.1 to 2); Moderately Satisfactory (2.1 to 3); Satisfactory (3.1 to 4); Highly satisfactory (4.1 to 5); Not demonstrated (ND).
Support for the Ministry of Health and Sport PASS-MSD (A031442001)5.04.04.13.5-4.54.04.04.04.1
National Fund for Productive and Social Investment PASS-FPS (A031442004)4.53.54.53.03.54.04.54.04.54.0
Save the Children 2006-09 5 year program SCC (S062525PRG)5.05.04.54.04.04.54.04.54.04.4
Support to the Malnutrition Zero Program: MI Micronutrient Component (A033957002)4.53.03.03.0-2.53.03.03.03.1
Support to UNICEF with Sweden and Netherlands PACSAS (A032896001)4.54.54.13.13.14.04.13.53.53.8
Écodéveloppement communautaire UQAM (S063421001)4.54.54.14.14.14.14.53.54.14.2
UNICEF-UN Joint Program to Support the Malnutrition Zero Program (A033957003)5.04.54.54.04.55.04.05.04.04.5
Plan-International (S062900PRG)5.04.54.54.54.15.04.54.54.54.6
MSD Budgetary Support and Evaluation (A033957001)5.04.05.04.04.54.53.53.53.54.2
AUCC_UPCD Program Tier 2 – Final Phase (S061266PRG)4.54.13.03.14.04.14.13.53.53.8
Average Score4.84.24.13.64.04.24.03.93.94.1
Governance
ProjectRelevanceEffectivenessSustainabilityGenderEnvironmentFootnote 19CoherenceEfficiencyManagement principlesPerformance managementAverage
Rating Scale: Very Unsatisfactory (0 to 1); Unsatisfactory (1.1 to 2); Moderately Satisfactory (2.1 to 3); Satisfactory (3.1 to 4); Highly satisfactory (4.1 to 5); Not demonstrated (ND).
CESO - BOLIVIA (A030740001)4.14.03.53.53.14.03.54.54.03.8
Trilateral Criminal Defence /Barreau du Québec (A020910018)4.13.53.03.0-4.03.53.14.03.5
Municipal Partnership Program 2007-2010 (A033735PRG)4.54.53.54.54.04.54.14.54.04.2
Strategic Governance Mechanism SGM (A032217001)4.54.53.54.5-5.04.05.04.14.4
Institutional Support Defensor del Pueblo (A030422001)4.54.54.04.1-5.04.15.03.54.3
Peacebuilding initiative with CNE (A032846001)4.14.13.04.1-4.54.04.54.14.1
Andean Regional Gender Equality Fund III FIG3 (A032298001)4.54.53.55.03.03.54.14.53.54.0
Gender Equality Basket Fund Bolivia BF (A032504001)4.53.52.55.0-4.12.54.53.53.8
Average Score4.44.13.34.23.44.33.74.53.84.0
Sustainable Economic Growth
ProjectRelevanceEffectivenessSustainabilityGenderEnvironmentFootnote 20CoherenceEfficiencyManagement principlesPerformance managementAverage
Rating Scale: Very Unsatisfactory (0 to 1); Unsatisfactory (1.1 to 2); Moderately Satisfactory (2.1 to 3); Satisfactory (3.1 to 4); Highly satisfactory (4.1 to 5); Not demonstrated (ND).
Contribution du programme/Program Contribution (S061681PRG)4.04.54.54.54.54.14.14.54.14.3
Bolivia Canada Hydrocarbon Project (A030420001)4.54.54.54.53.14.04.04.14.14.1
WUSC/CECI 2009-14 Building on Success (S064494PRG)4.04.14.04.14.04.14.14.14.54.1
CESO 2009-2014 Volunteer Cooperation Prg (S064520PRG)4.04.14.04.03.54.04.14.04.14.0
Programme Coop 2007-2012 (S063681PRG)4.04.54.14.04.54.14.04.04.04.1
Average Score4.14.34.24.23.94.14.14.14.24.1

Annex 6: Management Response

Introduction

Overall, the Bolivia Country Program Evaluation (2005-2010) is a useful assessment of Canada’s development Program in Bolivia. It relies on information from project and progress reports as well as from extensive interviews with key informants, who were involved in decision-making and implementation of the Program. Importantly, the report assesses Program effectiveness during a period of significant change – i.e. the emergence of a new critical political and social environment in Bolivia and the redefinition of development priorities in Canada.

The report positively assesses (a) the understanding, leadership and participation of key stakeholders (national and international) involved in the processes that governed Program planning, implementation and monitoring, and (b) how well the objectives of the Program represented the interest, needs and aspirations of the communities and institutions involved. The report concludes that most of the observed projects and interventions were relevant to Canadian and Bolivian strategic priorities, and produced positive outcomes, but there is still a need to improve long-term institutional capacity to achieve sustainable investments (e.g. health and governance interventions).

With respect to the Program’s strategic capacity building approach, the evaluators’ understanding of the limited capacity of the Bolivian public service in retaining expertise and knowledge during a period (2006-10), is appreciated. The evaluators valued the efforts made by the Program and its partners, to increase the institutional and technical capacity of local authorities and institutions (e.g. municipalities) in support of the decentralization process led by the national government. The evaluators also recognized that the Program played a leadership role in promoting policy dialogue on gender equality and contributed to important gender equality results through its programming, such as increasing political participation by women, strengthening capacity of women’s organizations and improving protection of women’s rights.

Due to program budget reductions, the Bolivia Program annual budget is declining from $18M in 2011-2012 to $9M by 2014-2015. This reduced bilateral annual budget is roughly equivalent now to the annual Canadian civil society programming in Bolivia supported by the Partnerships for Development Innovation Branch (PDIB).

Given the importance of PDIB in the country, and since the creation in 2011 of the Coordination Group for Canadian Organizations in Bolivia (COCAB), the Program has intensified harmonization and coordination between its Bilateral and Partnership streams. This effort has enriched the Program’s assessment of development priorities in the country and its understanding of the vital role played by local governments (municipalities) in rural areas. It has also resulted in new partnerships between Bilateral and Partnership initiatives that serve to strengthen Canada’s aid effectiveness in Bolivia. One Canada-based staff has been designated to promote strategic opportunities between the Bilateral Program and PDIB.

In light of the budget situation, no new bilateral projects have been considered for approval in Bolivia since 2011, and there is limited room for additional programming until at least 2015.

Furthermore, Canada-based staff at the Mission in La Paz has been reduced from three to two in July 2014, and the local PSU contract will expire on January 31, 2015.

Notwithstanding these current programming limitations, the Bolivia Program has taken measures to respond to the recommendations within existing projects and any new ones.

Recommendations: 1. Building ownership and alignment of CIDA programming through engagement and consultation with partners has been a strength of the program during the review period. It is recommended that the program make a renewed commitment to this approach in its future program planning efforts.

Commitments/Measures: The Program agrees with this recommendation. Since 2011, the Program has consistently built ownership of Program activities by the Government of Bolivia, as well as ensured alignment of Canada's bilateral Program with Government of Bolivia priorities. For example, for all of its bilateral projects, the Program has invited a relevant Government of Bolivia representative to participate as member of a Project Steering Committee (or similar committee).

1.1 Consultations with the Government of Bolivia on any new strategic programming orientations or project proposals will be held within the framework of the updated country strategy.

1.2 The Program will maximize coordination (e.g. joint monitoring missions).

1.3 Provided a focus on Children & Youth is maintained, the Program will accept the recent offer of the donor community for Canada to lead the Donor’s Health Coordination Group.

1.4 The Program will promote further coordination and harmonization of Canada’s aid footprint in Bolivia, as well as with other donor’s activities, particularly in the Department of Chuquisaca where Canada has significant bilateral and partnership initiatives.

Responsible: PTLs & HOA

Completion Date:

1.1 Completed as of January 2015.

1.2 Completed as of December 2014.

1.3 Completed as of June 2014.

1.4 Completed as of June 2014.

Recommendations: 2. In light of limited institutional capacity in most sectors in Bolivia, effectiveness and sustainability will require concerted attention.  It is recommended that future engagement include a focus on capacity development, taking into consideration how Canadian value added and knowledge can best be brought to bear in the development of explicit plans for major institutional partners.

Commitments/Measures: The Program agrees with the recommendation. Since 2011, the Program has focused on sustainable and effective capacity development of partners. For example, in both the health and economic growth sectors, two key projects (SOCODEVI Economic Growth and PLAN CANADA Public Health Caminas) are forming alliances with sub-national governments to maximize sustainability.

2.1 The Program will ensure that capacity and sustainability issues and proposed measures to address such issues are identified in relevant documents for each new project and in annual country reporting (Project Management Summary Report) for all projects.

Responsible: PTLs & HOA

Completion Date: 2.1 Completed as of May 2014.

Recommendations: 3. The program should build on progress in strengthening integration of gender as a crosscutting theme by: ensuring that all projects include a gender analysis, strategy, indicators and targeted resources; developing a standard gender monitoring and reporting system to assess gender integration across all projects and sectors; and sharing knowledge with partners and stakeholders about gender challenges, opportunities and lessons.

Commitments/Measures: The Program agrees with the recommendation, which will necessitate appropriate resources being made available to follow through on the commitments. In 2011-12, training was provided to all CBS and local consultants in gender equality and program planning and monitoring. Furthermore, gender equality training was coordinated with the Canadian Civil Society Coordinating Group in Bolivia (COCAB) members in 2013.

3.1 The Program will assure that Program Staff that have not taken HQ Gender Equality Training in the last five years, participate in the refreshing training programs.

3.2 The Program will ensure that all project officers (local and Canadians) coordinate planning and monitoring activities with the local Gender Equality specialist. At least one monitoring mission per project per year will include gender equality reporting. The Program will also ensure that gender equality results are included in its annual country reporting.

3.3 The Program will plan a systematic exchange of best practices on gender equality and on women’s empowerment within the country and between country programs, with involvement from the local and HQ gender equality specialists as well as relevant project officers.

3.4 The Program will actively join the Regional Community of Practice on Gender Equality that is coordinated by the Peru Program.

Responsible: PTLs & HOA

Completion Date:

3.1 Completed as of January 2015

3.2 Completed as of May 2014

3.3 Third quarter 2014

3.4 Completed as of January 2015.

Recommendations: 4. The program should improve mainstreaming of environmental sustainability by developing an environment implementation plan, and working with like-minded donors to integrate environmental considerations into pooled fund objectives.  Depending on the nature of future investments, the program should make use of positive lessons gained from past SEG programming, including building partner capacity to implement environmental standards.

Commitments/Measures: The Program agrees with the recommendation. Historically, the Program has ensured that all SEA and CEAA requirements have been met at the Program and project level.

Some experienced Canadian NGOs active in Bolivia (e.g. SOCODEVI, CECI, PLAN, OXFAM and CUSO) have been instrumental in developing ways of living more sustainably in rural and semi-urban communities that have taken many forms from (a) reorganising living conditions, (b) reappraising economic sectors, or work practices, (c) using Canadian experience to develop new technologies, to (d) adjusting individual lifestyles that conserve natural resources. The dissemination of these practices and lessons learned has already benefitted bilateral programming and implementation.

4.1. The Program will ensure effective coordination with experienced donors (e.g. GTZ, Swiss, and the Belgians), who have proven to have considerable experience and technical expertise in the area of sustainable environment.

4.2. The Program will develop a plan with current project executing agencies with solid environmental expertise to ensure that knowledge and best practices are transferred to local partners (e.g. SOCODEVI, Plan International Canada, and Colleges and Institutes Canada - CICAN).

4.3 The Program will actively join the Regional Community of Practice on Environmental Sustainability that is coordinated by the Peru Program.

Responsible: PTLs, HOA & Specialist

Completion Date:

4.1 Completed as of August 2014

4.2 Completed as of June 2014

4.3 Completed as of March 2014

Recommendations: 5. While a program level performance management framework has been articulated, the program should ensure that it specifies clear expected results with verifiable indicators, and to the extent possible collect adequate baseline information to facilitate eventual evaluations.

Commitments/Measures: The Program agrees with the recommendation.

5.1 The Program will update its Program level Logic Model and Performance Management Framework, in the context of the updating of DFATD development country strategies. It will ensure that expected results are clear that verifiable indicators are identified and baseline data is collected.

Responsible: PTLs & HOA

Completion Date: 5.1 Completed as of September 2014

Recommendations: 6. There is scope for greater visibility of program accomplishments with partners and stakeholders.  The program should develop an appropriate communications strategy to raise its profile.

Commitments/Measures: The Program agrees with the recommendation:

6.1 The program will develop a communication plan, in coordination with the Embassy in Lima and in agreement with DFATD strategy of visibility and recognition.

6.2 The Program in coordination with COCAB will circulate a newly published brochure that clearly describes the ample coverage of development activities of Canadian civil society and its volunteers in Bolivia to its partners and the various stakeholders of the Program in Bolivia. This brochure will be translated in both English and French to be distributed in Canada.

6.3 The Program, in coordination with the Embassy in Lima, will summarize best practices and experiences in Bolivia to be incorporated in the Mission Webpage every four months.

Responsible: Director & HOA

Completion Date:

6.1 Completed as of July 2014

6.2 Completed as of January 2014

6.3 March 2015

Date modified: