Bolivia Country Program Evaluation - 2005-2010 - Synthesis Report

Foreign Affairs, Trade and Development Canada (DFATD)

January 2014

Acknowledgments

The Development Evaluation Division wishes to thank all who contributed to this evaluation for their valued input, their constant and generous support, and their patience.

Our thanks go to the team that carried out the evaluation.  It was led by Philippe Bâcle of Baastel and included three experts from the same firm (Dean Pallen – expert in sustainable economic growth; Catherine Gander – expert in governance and gender equality; and Alain Lafontaine – expert in the environment) and three Bolivia experts (Bertha Pooley – expert in health; Aida Ferreyra Villarroel – expert in gender equality; and Fernando Aramayo – expert in governance).

The Development Evaluation Division would also like to thank the management and staff of the Bolivia Country Program for their valuable support.

From the Development Evaluation Division, we wish to thank Frantz Pierre-Jérôme, Evaluation Manager and Sarah Gibson, Senior Editor, for their contribution to the synthesis report, and Michelle Guertin, Evaluation Team Leader, for guiding the evaluation to completion.

James Melanson
Head of Development Evaluation

List of Abbreviations

Executive Summary

1.1 Bolivia Country Program^{Footnote 1}

This evaluation assesses the Bolivia Country Program’s performance between 2005 and 2010 as part of the former Canadian International Development Agency (CIDA), amalgamated as of July of 2013, with Foreign Affairs, Trade and Development Canada.  The total funding disbursed by the Agency in Bolivia for the period was $119.66 million, spread across 233 projects, and involving: the Geographic Programs Branch which contributed $71 million (59% of the total); Partnerships with Canadians Branch having contributed 40%, (its most significant country in the Americas and globally over some of the review period); and Multilateral and Global Programs Branch contributing the remaining 1%.  Sectors of focus were democratic governance, health, and economic growth.

This evaluation responds to the Government of Canada’s Financial Administration Act and Treasury Board Evaluation Policy (2009) that require evaluation of all program expenditures every five years.  It aims to assess results achievement, inform decision-making, draw lessons learned, and improve program performance.  Evaluation criteria included relevance, effectiveness, efficiency, economy, coherence, and sustainability, with attention to cross-cutting themes of gender equality and environmental sustainability.  A sample of twenty-four projects that were in operation between 2005 and 2010 was drawn, encompassing 50% of total disbursements with a proportionate representation of funding from the three program branches.

1.2 Results

The evaluation finds good performance and contribution to reducing poverty in Bolivia.  The program built on previous investments and responded to emerging priorities to promote development in basic health care, democratic governance and sustainable economic growth (SEG).  The program’s on-going policy dialogue allowed it to respond effectively to changing policy environments in Bolivia and at CIDA.  Maintaining relevance to Bolivian needs increased chances for sustainability.

Notably, the program’s ability to respond to Bolivia’s changing priorities positioned it, and Canadian non-governmental organizations and their counterparts, to contribute to poverty reduction in Bolivia.  The hydrocarbons project was notable in this regard.

1.3 Conclusions

The evaluation provides evidence that the program helped improve living conditions and standards in the key programming sectors of health, governance, and SEG.  Factors contributing to success included a focus on building consistent and long-term relationships with Bolivian counterparts, alignment of development programming under Bolivian priorities, and coordination with other donors.

Policy dialogue proved effective in facilitating results across all sectors of priority.  In governance and in gender equality, a participatory approach allowed the program to secure government, donor and civil society organization participation.  Canadian interventions contributed directly to the strengthening of important national institutions, including the Auditor General, the National Electoral Court, and the Ombudsman Office.

Based on evidence from the sample of projects, key interviews, and relevant documentation, the following conclusions can be drawn on the evaluation criteria:

Relevance: While navigating changes to political leadership in Bolivia and to corporate orientation at CIDA, the program remained relevant to Canadian strategic priorities and Bolivian needs.  It delivered aid across priority sectors according to the 2003-2007 Country Development Programming Framework and the revised 2009 Country Program Strategy.  A major strength was the focus on building consistent, long-term relationships with key partners and beneficiaries through a highly participatory process.

Effectiveness: There were good development results achieved over the review period, particularly in SEG where the hydrocarbons project contributed to the country’s ability to benefit and redistribute revenues by establishing a more transparent and efficient tax and royalty collection system.  The program also contributed to better health services for the most vulnerable, and facilitated decentralization of services and strengthened governance and democratic institutions.

Sustainability: Encouragement of ownership by Bolivian stakeholders and other donors was a good strategy to help sustain some of the program’s investments in the face of limited local administrative capacity.  In the hydrocarbons sector there was an improvement in institutional capacity and sustainability. However, investments in health were rated as less sustainable due to the difficulties of developing strong local administration.

Gender Equality: During the early part of the review period, integration of gender equality across the portfolio was minimal given that the local gender specialist was focused on managing CIDA’s Gender Equality Fund.  In 2010, Holland became the lead on the Gender Equality Fund, which allowed the gender specialist to improve the integration of gender equality as a crosscutting issue in the program.  There were important contributions from the gender-focused programming, such as increased political participation by women, strengthened capacity of women’s organizations, and improved protection of women’s rights.  The program also played a leadership role in promoting policy dialogue on gender equality with other donors, Bolivian government partners and non-governmental organizations.

Environmental Sustainability: The evaluation demonstrated that in Bolivia, applying the Canadian Environmental Assessment Act was not enough to ensure full integration of environmental considerations across the program.  A case in point was water and sanitation where the focus was more on securing water supply than in promoting conservation and integrated management of water resources.  The program did nonetheless provide some good examples of promoting and integrating the environment, particularly in the SEG sector.  Overall there was room for improvement in the mainstreaming of environmental issues in the country program and projects, as recommended by the strategic environmental assessment conducted in 2009.

Coherence: The program demonstrated good examples of external coherence with other donor programs through involvement in multiple pooled funds and by participating effectively in a key multi-donor coordination group.  The evaluation also found relatively good internal coherence between the Geographic Program Branch and Partnership with Canadians Branch on some key projects, although a more systematic approach to coordination with civil society organizations would have been useful.

Efficiency: Based on document review including the 2011 Americas Program benchmarking exercise, the program performed favourably in terms of cost-efficiency, when compared to similar CIDA programs in other middle-income countries.  As well, there was an appropriate blend of larger pooled fund investments with relatively lower overhead cost ratios, and smaller but innovative remote area health interventions with relatively higher overhead cost ratios.  However, several limitations, also evident in other Country Programs, affected efficiency during the evaluation period, including high staff turnover, delays in project approval and limited delegation of spending authority.

Management Principles: The program demonstrated strong evidence of adherence to the principles of the Paris Declaration on Aid Effectiveness (ownership, harmonisation and alignment).  Policy dialogue was also a strength, facilitated through involvement in a number of pooled funds.  Even with shifting political circumstances in Bolivia, and new articulation of corporate priorities at CIDA, general alignment of program interventions with Bolivian priorities was maintained.

Performance Management: At the project level, results-based management principles, monitoring, evaluation and risk management practices were generally applied systematically.  There was a program level performance management framework in place, but it did not provide a sufficient overview of planned results with measurable indicators, and baseline data was not established, which limited the ability to articulate broader achievements and results.  Knowledge management, communication and dissemination of information about the program’s works and achievements could have been stronger.

1.4 Recommendations

A number of recommendations arise from the evaluation findings and conclusions:

• Building ownership and alignment of CIDA programming through engagement and consultation with partners was a strength of the program during most of the review period. It is recommended that the program make a renewed commitment to this approach in its future program planning efforts;
• In light of limited institutional capacity in most sectors in Bolivia, effectiveness and sustainability will require concerted attention. It is recommended that future engagement include a focus on capacity development, taking into consideration how Canadian value added and knowledge can best be brought to bear in the development of explicit plans for major institutional partners;
• The program should build on progress in strengthening integration of gender as a crosscutting theme by:
  • Ensuring that all projects include a gender analysis, strategy, indicators and targeted resources;
  • Developing a standard gender monitoring and reporting system to assess gender integration across all projects and sectors; and
  • Sharing knowledge with partners and stakeholders about gender challenges, opportunities and lessons;
• The program should improve mainstreaming of environmental sustainability by developing an environment implementation plan, and working with like-minded donors to integrate environmental considerations into pooled fund objectives. Depending on the nature of future investments, the program should make use of positive lessons gained from past SEG programming, including building partner capacity to implement environmental standards;
• While a program level performance management framework has been articulated, the program should ensure that it specifies clear expected results with verifiable indicators, and to the extent possible collect adequate baseline information to facilitate eventual evaluation; and
• There is scope for greater visibility of program accomplishments with partners and stakeholders. The program should develop an appropriate communications strategy to raise its profile.

1.0 Introduction

1.1 Objectives of the Evaluation

The findings and conclusions of this evaluation of the Canadian International Development Agency (CIDA)’s Bolivia Country Program (BCP)^{Footnote 2} are based on evidence gathered in 2012.  The evaluation supports accountability and learning, and has three objectives:

• to report the results achieved by the program over the five fiscal years between 2004-2005 and 2009-2010;
• to assess the program’s overall performance in achieving these results; and
• to document and disseminate findings and lessons, and formulate recommendations to improve performance.

1.2 Country Context

Despite modest economic progress and its wealth in oil, natural gas, and minerals, Bolivia remains one of South America’s poorest countries and one of the world’s most unequal societies. According to the World Bank, Bolivia qualifies as a lower middle-income country with a gross national income per capita of $4,510 in 2010^{Footnote 3}.  Continuing high poverty levels mean nearly 65% of its population lives under the national poverty line, and 84% of its rural population lives in extreme poverty.  In 2011, the country ranked 108 out of 187 on the Human Development Index developed by the United Nations Development Programme.^{Footnote 4}

Over the period under review here, Bolivians radically restructured the country’s political economy. In 2006, an overwhelming majority of Bolivians voted for Evo Morales, the country’s first indigenous Aymarian to be elected President.  The new government outlined an economic and political vision for Bolivia that rejected the “free market economic model” encouraged by the World Bank and the International Monetary Fund.

The government’s new vision called for a mixed economic model built upon state management of the basic sectors of the economy.  The 2006 laws to nationalize natural resources, hydrocarbons and the mining sector were aimed at increasing government revenues and redistributing wealth.  The 2006 National Development Plan (NDP) provided a five-year overarching framework for change built upon four pillars:

• Inclusive economic growth and sustainable productive development, with increased state intervention in key sectors of the economy such as hydrocarbons and public utilities.  This pillar supports rural development, food security, access to land, and small enterprise development;
• Human development and access to social services.  This pillar promotes expanded coverage in health and education through sub-national governments, and measures such as social safety nets and redistributive cash transfers to fight poverty and social exclusion;
• Country ownership of its development process by promoting equitable political, economic and cultural relations between Bolivia and other nations; and
• Empowerment of social and indigenous organizations, with the redefinition of traditional power relationships in the country.

To diminish the influence of donors on internal policy, the government passed a 2007 law regulating international cooperation and foreign funding.  The Ministry for Development Planning and its Vice-Ministry of Public Investment and Foreign Financing were charged with the responsibility of coordinating international technical and financial cooperation.  The Constitution of 2009 further defined the role of the state in the economy and society and devolved powers to sub-national governments and to indigenous communities to facilitate the implementation of the NDP.

Nevertheless, the spirit of national consensus dissipated over the period under review as political tensions climbed and sporadic violent protests erupted.  Progress was slow and the government did not meet the competing demands of all its voters.  A national debate ensued over the administration and redistribution of resources in the nationalized sectors of the economy.  Foreign interests complained about the nationalization law while political opposition groups contended that weaknesses in the law still left Bolivia’s natural resources vulnerable to exploitation by foreign corporations.

1.3 International Cooperation in Bolivia

The 2006-2011 NDP and the 2007 international cooperation laws in Bolivia changed the operating environment for CIDA’s Bolivia Country Program.  It responded by allying with other donors and the Bolivian government to form the Group of Partners for Development, a policy forum to promote Bolivia’s leadership in donor harmonization and alignment according to the principles of the 2005 Paris Declaration on Aid Effectiveness.

However, the socialist and nationalistic approach to development outlined in the NDP and the 2007 foreign-aid law complicated the relationships between the Bolivian government and Western donors. A chill in relations ensued.^{Footnote 5} The absolute amount of donor funding to Bolivia dropped significantly during the first years of the Morales government.  At the same time, it became particularly critical of the United States Agency for International Development (USAID) for its alleged funding of political opposition groups as part of its aid program.  In 2008, the US Drug Enforcement Agency was expelled from Bolivia, and in 2009, USAID was also asked to suspend its governance program.  Nonetheless, the United States remained the lead donor to Bolivia between 2005 and 2010 and overall funding from other donors increased again from 2008 to 2010, but generally did not reach pre-2006 levels.

Between 2005 and 2010, the top ten bilateral donors disbursed a total of US$2.7 billion to support development projects in Bolivia, averaged at US$454 million a year.  The principal donors included the United States (28%), Spain (17%) and Japan (11%), followed by Germany, Netherlands, Denmark, Sweden, Canada (4%), Switzerland and France.  In recent years, new donors have appeared, such as Venezuela, Brazil, China, and Cuba, all promoting a socialist agenda compatible with that of the Morales administration.  Major donors have coalesced around initiatives related to hydrocarbons, public health, governance, economic growth, education, capacity building and the environment.

1.4 CIDA Program

Canada has a long history of providing financial and technical assistance to Bolivia, beginning in 1967, one year before the official creation of the former CIDA.  Given the country’s chronic poverty and inequality, Bolivia was selected as one of CIDA’s nine countries of focus in 2003, and retained that status through reviews in 2005 and 2009.

CIDA’s programming in Bolivia for the period covered by this evaluation (2005 to 2010) was guided by the 2003-2007 Country Development Programming Framework (CDPF), the 2007 Canadian Government Americas Strategy, the 2009 Country Program Strategy, and other strategic programming documents.  It was also framed in line with Bolivia’s NDP.  Funding was provided through three channels: government to government; multilateral organisations and global programs; and civil society organisations.

The objective of CIDA’s assistance during this period was to reduce poverty and enhance living standards by helping to meet basic human needs and to consolidate Bolivia’s economic and political reforms.  This was pursued through efforts to improve access and capacity in the health and water and sanitation sectors, and to modernize the state’s regulatory capacity, public sector efficiency, and democratic practices.  In addition, CIDA programming addressed the integration of gender equality and environmental sustainability (as crosscutting issues), as required by the Agency’s policies.^{Footnote 6}

In 2009, with the articulation of CIDA’s Bolivia Country Program Strategy, the overall programming objective stayed the same but the sectors of focus shifted to:

• Children and youth – improved maternal health, reduced child mortality and increased access to potable water and sanitation;
• Economic growth – increased participation of vulnerable populations in economic development through market-driven skills for employment, effective corporate social responsibility, and increased access to markets, property, credit and technology;
• Improved effectiveness and credibility of key democratic and oversight institutions.

During the 2005-2010 period, CIDA’s disbursements to Bolivia totalled $119.66 million, covering 233 projects and making it one of the largest country programs funded by the Agency in Latin America.  Between fiscal years 2004-2005 and 2009-2010, disbursements for projects worth more than $250,000 totalled $111.58 million, across 64 projects.  More than half of the assistance was through the bilateral country program, one-third through partnerships with Canadian civil society organizations^{Footnote 7}, and a smaller fraction through multilateral and global programming,^{Footnote 8} as presented in Figure 1.  The sectoral breakdown of disbursements is illustrated in Figure 2, with democratic governance and health accounting for almost two thirds of the funds. Funding was disbursed mostly through civil society (61%) and the United Nations (20%), as illustrated in Figure 3.

Figure 1: Disbursements to Bolivia according to programming channels between fiscal years 2004-2005 to 2009-2010

Figure 2:Disbursements to Bolivia according to programming sectors from fiscal years 2004-2005 to 2009-2010

Figure 3: Disbursements to Bolivia by type of executing agency from fiscal years 2004-2005 to 2009-2010

Building on the priorities of the 2009 Country Program Strategy, the 2010-2015 CDPF focused on economic growth, governance, and children and youth.

2.0 Evaluation Methodology

2.1 Scope

The methodology for country program evaluation assesses overall program performance, as well as results in sectors and crosscutting themes based on a representative sample of investments.

The evaluation team chose projects from among all those in operation, whether just beginning, in progress, or sun setting, and for which sufficient data existed.  The sampled projects were intended to be representative of the whole program in terms of:

• sectoral and thematic coverage;
• delivery channels, including:(i) directive and responsive government-to-government projects; (ii) targeted multilateral activities; and (iii) targeted partnership with Canadians civil society organizations projects; and,
• delivery modalities

In the end, a sample of twenty-four projects was selected, representing 50% of total program disbursements for the five-year period of this evaluation (See Annex 1 for the list of projects in the sample).  Ten represented health sector programming; eight, governance; and five sustainable economic growth (SEG).  Thirteen projects were funded through bilateral programming, ten through partnerships with Canadian civil society organizations^{Footnote 9}, and one through multilateral institutions and global programs.

2.2 Evaluation Questions and Criteria

Development results are examined by asking what was achieved by the program during the evaluation period.  Program management effectiveness is assessed by asking how and why the intended results were achieved, or not.  The evaluation assesses these points based on eight criteria: core ones recommended by Canada’s Treasury Board; and additional ones from the Organisation for Economic Co-operation and Development’s Development Assistance Committee.

Development Results

What was achieved by the program during the evaluation period?

• Relevance: To what extent are the development interventions’ objectives consistent with beneficiaries’ requirements, country needs, global priorities, and partners and donors’ policies? To what extent are the development interventions appropriate in a given sector, region or country?
• Effectiveness: To what extent were the development interventions’ objectives achieved or likely to be achieved, taking into account their relative importance?
• Sustainability: To what extent will the development interventions’ benefits continue after major development assistance has been completed? What is the probability of continued long-term benefits?
• Crosscutting themes: To what extent are gender equality and environmental sustainability integrated throughout the program framework?

Management Factors

How and why were the intended results achieved or not?

• Coherence: What are the coordination activities and policy dialogue with national and other international actors? What is the program’s role in coordinating CIDA’s delivery channels and Canada’s whole-of-government efforts?
• Efficiency: To what extent are resources (e.g. funds, expertise, time) translated into results and based on optimum allocation of resources?
• Management principles: To what extent are the principles of the 2005 Paris Declaration on Aid Effectiveness (i.e., local ownership, alignment, and harmonization) adhered to?
• Performance management: To what extent is there a management strategy to assess the performance of development interventions against intended results, including results-based management, monitoring and evaluation functions; risk management; and timely corrective action to address problems when they arise?

2.3 Data Gathering Methods and Analysis

Data was obtained at CIDA headquarters, and from Canadian stakeholders, support agencies, non-governmental organizations, and monitors involved in implementing the program.  Supported by three Bolivian experts, the team also collected data in Bolivia during a three-week mission in September-October 2012.

Several lines of evidence were gathered, ensuring triangulation of findings:

• Literature and document review (see Annex 2 for the list of documents reviewed)
• Focused interviews (150) with individuals and groups of key stakeholders, including beneficiaries (see Annex 3 for the list of people interviewed), based on interview guides approved by CIDA;
• Focus groups with civil society, technical and financial partners, Canadian and Bolivian non-governmental organizations; and with CIDA staff at headquarters and in the field; and
• Field visits (10) to a number of projects.

The data analysis focused on (see Annex 4 for the evaluation matrix):

• Project objectives, expected and achieved results, performance (based on evaluation questions and criteria^{Footnote 10}), type of investment and delivery channel; and
• Description of the overall program, its objectives, expected and achieved results, and performance (based on evaluation questions and criteria).

2.4 Evaluation Challenges and Limitations

The evaluation team faced some constraints and limitations in collecting and analyzing data.  These were deemed manageable and do not detract from the overall validity of the findings, conclusions and recommendations.

2.5 Data Availability and Reliability

• Data available from CIDA’s management information system proved uneven, making it difficult to compile a complete and accurate view of programming and funding by year, sector and delivery mechanism.
• Performance information was difficult to obtain for some multilateral and partnership initiatives because reporting is not systematically completed on a country basis.
• The efficiency criterion was not assessed to its full extent due to the lack of available information about management costs.

To overcome these constraints, the evaluation team expanded its research at CIDA - triangulating and comparing lines of evidence, and validating them with the largest possible number of contacts within and outside CIDA to create a reliable picture of the program’s performance.

2.6 Canadian Contribution and Attribution

The results reported in the present evaluation may be attributed to Canada where the Agency has been the sole executive party of the development interventions.  In many instances, however, Canada acted as a partner in development interventions.  Only Canada’s contribution to results can be recognized; direct attribution to Canada of the results achieved is not feasible.

In the context of this evaluation, given the large number of development interventions in which Canada is a partner, the focus is placed on Canada’s contribution.

3.0 Findings for Development Results Criteria

The program’s three sectors achieved predominantly “highly satisfactory” results across all five evaluation criteria measuring development results as demonstrated in the following sections (see Annex 5 for project scores by sector).

3.1 Relevance—Highly Satisfactory

The program scored well on relevance because it committed to stakeholder participation, and focused on multiple needs as articulated by the most vulnerable.

For the period under review (2005-2010), CIDA’s Bolivia Program rated in the highly satisfactory range for its relevance to Bolivian, Canadian and Agency development priorities and to the needs of the Bolivian people (summarised by sector in the table below).

Figure 4: Relevance Scores by Programming Sector

3.1.1. Relevance to shifting Bolivian priorities.

The program’s long-standing commitment to involving stakeholders at the design and implementation stages allowed it to navigate the political changes in Bolivia while maintaining relevant programming.

The evaluation confirms that the 2003-2007 CDPF in place at the outset of the review period reflected both Agency and Bolivian aims.  The programming was in line with the 2005 Paris Declaration on Aid Effectiveness.  The program’s sectors of concentration reflected three corporate priorities: health; water and sanitation; and modernization of the state, including public sector reforms.  The plan also promoted the crosscutting issues of gender equality and environmental sustainability.  Further, it channelled most of its disbursements to Bolivia through institutional pooled funds (also called basket funds), which required a greater focus on Bolivians’ own capacity to manage and implement.

The CDPF responded to Bolivian priorities and reflected the 2001 Poverty Reduction Strategy that called for more efficient, transparent, and accountable institutions, more professionalism in the public service, judicial reform to reduce corruption, a strengthening of municipal governments and a general enhancement of civil society participation.  It also built on a history of CIDA investment in health benefitting vulnerable Bolivians.

The high relevance of this programming stemmed from a practice of participatory project design and of working with partners including other donors, Ministries, community leaders, non-governmental organizations and municipal authorities.  As a result, many parts of the program proved resilient during the 2006 political realignment in Bolivia, and the 2006 and 2009 elections of the Morales government on a “socialistic-indigenous” platform.

The new administration’s agenda had implications for the delivery of development assistance in the country.  The 2006 National Development Policy provided new guiding principles based on ideas of social inclusiveness, and economic decentralization and diversification.  Also, the 2007 law to govern international cooperation in the country introduced new standards and procedures.  For donors, there followed a period of adjustment between 2006 and 2008 as tensions rose over perceptions about the possible dilution of Bolivian development plans by western donors.

Throughout this period, CIDA maintained the confidence of the Bolivian Government.  Bilateral discussions led to the ratification of initiatives begun under the 2003-2007 CDPF.  The program moved forward with investments in health and sustainable economic development.  It also retooled some of its existing governance sector programming, (designed in collaboration with the outgoing political establishment) to better reflect the new political landscape.

Sustainable Economic Growth: Evaluation evidence suggests that support of non-governmental organizations to promote entrepreneurship and to support micro finance contributed to small-scale economic growth.  Likewise, support to reform of the hydrocarbon industry helped increase revenues and facilitated the government’s efforts at wealth redistribution.

Health: Most of the health projects in this evaluation sample were developed during the transition period (2006-2009)^{Footnote 11}, and reflected the benefits of a participatory design process.  The program’s major health investment, to support the Ministry of Health and Sport, arose out of consultations with the government and aligned explicitly with the NDP.  The Program to Support the Health System project and its components responded to Bolivian desires for universal access to quality health care, environmentally sensitive health initiatives, nutrition, and the full ownership of health services.

The programs’ water and sanitation projects also aligned well with Bolivian priorities set out in Bolivia’s National Water Resource Strategy, the National Watershed Plan and the National Desertification Plan.  The Zero Malnutrition Program reflected Bolivia’s Health Sector Development Plan, and the Micronutrient Initiative’s projects fit well with Bolivia’s Strategic Nutritional Plan.  The projects focused on delivering comprehensive health care by (1) designing indigenous-sensitive health care models; and, (2) offering training to deliver services, nutrition, and water-and-sanitation to remote regions.

Stakeholders agreed that CIDA was “very relevant and crucial to improve(ing) Bolivia’s health indicators and (for) align(ing) interventions with the international community.”

Governance: The ability of governance sector programming to adjust to the new regime also reflected its highly participatory design process.  Three governance sector projects included in the evaluation sample predated the Morales election.  All remained relevant afterward.  The Strategic Governance Mechanism supported four institutions: the Ombudsman’s Office, National Statistic Institute, National Electoral Court, and Auditor General’s Office—identified jointly with the Bolivia Vice Ministry of Public Investment and Foreign Financing.  When the 2009 Constitution broadened the responsibilities of these institutions, the program only increased its relevance.  The Andean Regional Gender Equality Fund (Phase three), designed to increase women’s role in decision-making, maintained relevance before and after the 2006 election.  The fund was designed to complement the gender policies of the previous government and support decentralization, also central to the new platform.  Promoting a more pro-active approach to gender equality, the Government created a new National Women’s Machinery under the Ministry of Justice.  A National Plan for Equality of Opportunities set the government’s priorities in promotion of women’s rights in land ownership, access to education, representation and participation in public office, and prevention of gender-based violence.

The program also retooled some of its programming to emphasize contributions to the Bolivian aim of increased representation of indigenous peoples.  It focused on promoting human rights and access to justice in collaboration with the Ombudsman’s Office and Public Defender’s Office; and elector participation and access to municipal and national government, with the National Electoral Court, the Association of Women Councillors of Bolivia and the Federation of Municipal Associations.

3.1.2. Relevance to realigned Agency priorities

The 2009 Country Strategy continued many of the general themes outlined in the 2003-2007 CDPF, readjusting emphasis to take account of emerging Canadian strategic priorities.

The Government of Canada’s 2007 Americas Strategy emphasised economic prosperity, democratic governance, and security and stability.  In 2009, CIDA refocused corporate priorities on food security, economic growth, and children and youth.  Consequently, the program reconfigured its investments so that i) governance became a crosscutting theme (along with gender equality and environmental sustainability); ii) longer-term health and water and sanitation programming contributed to children and youth; and, (3) the hydrocarbons project emphasized its contribution to sustainable economic growth while continuing to contribute to governance.

3.1.3. Relevance to a range of beneficiary needs

Innovative and culturally appropriate health care projects in remote and less populated areas served a population overlooked by the Bolivian government and other donors.  Governance and sustainable economic development programming addressed needs of national institutions, respecting the government’s socialist principles.

3.2 Effectiveness—Highly Satisfactory

There were good development results achieved over the review period, particularly in SEG.

The effectiveness of the Bolivia Country Program over the period under evaluation was rated as highly satisfactory and is summarised by sector in the figure below. The program demonstrated progress towards and achievement of stated project, program, and corporate objectives, particularly those related to the reduction of poverty.

Figure 5: Effectiveness Scores by Programming Sector

3.2.1. Sustainable Economic Growth programming contributed to poverty reduction

The hydrocarbons project evolved over the five years under review to achieve results in governance and sustainable economic growth, securing modest gains in reducing Bolivian poverty.

The project helped strengthen national institutions relevant to the hydrocarbon industry and provided guidance around natural gas pricing for export to Brazil and Argentina.  This “helped greatly,” according to interviewed stakeholders, in seeing the average annual revenue from the hydrocarbon sector increase to US$ 2.1 billion, between 2006 and 2011, from US$322 million, between 2000 and 2005.  The increased revenue contributed to the reduction of the country’s dependency on development aid funds to 17% in 2010, down from 53% in 2005.

The hydrocarbon project also contributed to the country’s ability to redistribute hydrocarbon revenues.  It helped establish a more transparent and efficient tax and royalty collection system, supported by training from the Canadian Revenue Agency in strategic planning, legal contracts and tax collection, auditing, and fraud investigation.  The redistribution of increased revenue helped put the country on track to meet the Millennium Development Goal of reducing by half the number of people living on less than a dollar a day.^{Footnote 12}

The program also contributed to small-scale economic growth by supporting Canadian and Bolivian non-governmental organizations. The Société de Coopération pour le Développement International project supported the emergence of a new and highly viable economic activity—oregano production in high altitude, semi-arid conditions.  Nearly 1000 small-scale farmers in eight municipalities currently participate.  Those interviewed for this evaluation spoke of using the extra income generated to pay for children’s education, health care, and for increasing consumption.  The project also invested in further developing local markets, including increased production and sale of organic eggs.

The Uniterra project promoted the concept of Fair Trade to help local producers gain access to world markets on favorable terms.  It trained 30 organizations in Fair Trade and the principles of economic solidarity, and 80 women-leaders in the management of microenterprises.  It also certified (with a private Bolivian institution) 160 productive organizations.  The project supported trade fairs, conferences, showrooms, promotional campaigns, and the opening of markets in Germany, Switzerland, Italy, Spain and Sweden.  It improved the ability of artisans to defend their interests and engage in political dialogue.

The participation in the work of these non-governmental organizations at a strategic level by the Canadian Executive Services Overseas’ Volunteer Cooperation program,^{Footnote 13} amplified the gains across the whole sector.  The program helped the textile business, supported by a non-governmental organization, to establish a highly profitable store, Coconut.  As well, the Canadian Executive Services Overseas is helping companies to develop their export capacity, in cooperation with the Trade Facilitation Office of Ottawa.

3.2.2. Health programming contributed to enhanced public policy and delivery of services to the most vulnerable

The program’s culturally relevant health projects contributed to policy development in Bolivia and the provision of increased healthcare in remote regions and to marginalized peoples in Beni and Pando.

Several projects seeded or provided templates for health-related policy developments, including Bolivia’s 2001 Basic Sanitation Development Plan, the National Food Security Plan and the Nutrition Policy.

The Support for the Ministry of Health and Sport project and its component projects performed well.  A formal demographic and health survey in 2013 will generate comprehensive data to support conclusions about its effectiveness.  In the interim, this evaluation found evidence of accessible healthcare services and improvements to the capacity of the Ministry of Health and Sports and to Departmental Health Service. Indicators for the Project to Improve the Coverage and Quality of Health Care show an increase in the number of children below the age of two vaccinated (from 50% to 70%); a 10% increase in services delivered by skilled workers in three departments; and the skills of over 50% of the health personnel updated or upgraded.  In Oruro Department, almost 80% of facilities were updated to be user friendly and culturally sensitive.

Other projects such as the National Fund for Productive and Social Investment, which responded to infrastructure and cultural needs amplified the Program to Support the Health System outcomes.  .  However, there was evidence to suggest that infrastructure design did not always take the opinions of end users or local health personnel into account.

The evaluation also confirms that the program’s health sector projects contributed to tangible outputs and outcomes for the intended beneficiaries:

• Incorporation of practices into Bolivia’s national policies : Basic Sanitation Development Plan, National Food Security Nutrition Policy, regional water and sanitation models;
• Chronic malnutrition diminished by 28% during the 5-year period of the Zero Malnutrition Program; anemia reduced 80% to 40% in children under 2; maternal and child coverage increased and processes were put in place to guarantee access to quality health services benefiting 228,000 mothers and 125,00 children in isolated areas of Beni and Pando;
• Communities implemented Potable Water and Sanitation Committees and best water, hygiene and sanitation practices;
• Municipalities allocated funds for health, water and sanitation activities, some increasing their contribution by 20% to 50%.
• Hospitals earned certifications for meeting supplies and training standards;
• Women gained more reliable access to food supplements for their children;
• Families and communities participated in information sessions regarding best practices in the different health areas;
• Municipal technicians received training to implement water and sanitation models; and,
• Water pumps and showers were installed in schools.

3.2.3. Governance programming supported decentralization and capacity building.

The Municipal Partnership Program built on achievements of a previous phase, and facilitated decentralization, the transfer of revenues to municipal governments, and mechanisms for dispute resolution between federal and local levels of government.  It provided advice incorporated into the 2005 Presidential Decree on Hydrocarbon Revenue, encouraged increased constitutional powers for municipalities, women’s participation in local governance, and support for local economic development.

The Strategic Governance Mechanism focused on the National Electoral Court, the Bolivian National Institute of Statistics, and the Auditor General’s Office to increase transparency and capacity for public policy-making.  The Defensoria del Pueblo project contributed to the consolidation of the Ombudsman of Bolivia as the “leading institution on human rights”.  Bolivian opinion polls revealed that Bolivians considered this institution “to be the (...) closest to citizens and the most committed to defending human rights.” The Office increased its presence throughout the country, empowered citizens, promoted its conflict resolution services, increased the number of investigations into complaints by 68%, and improved civic education about human rights.  However, much remains to be done to promote indigenous rights and to fully extend the institution’s presence into rural areas.  In addition, the political flux in Bolivia between 2001 and 2006 meant a high turnover in staff and continuing limited capacity of the judicial system.

The National Electoral Court, which sought to increase participation and integration in the electoral process, particularly for indigenous people, contributed to the following results:^{Footnote 14}

• Increased accuracy, transparency, security, preservation and efficiency of civil registry through a verification process and digitization (digitized over 14,400 civil registry books in 2009 to improve web accessibility);
• Increased transparency and credibility of the electoral process (e.g., according to public opinion polls, 87% of Bolivians considered the National Electoral Court to be a credible, trustworthy institution);
• Increased citizen awareness of the importance of registration and voting (e.g., training in democratic/electoral values to 14,600 secondary school student in 2009);
• Increased citizen participation in elections (e.g., higher rate of voter registration and 95% turnout for the 2009 elections);
• Increased access to government services (e.g., providing over 51,000 birth certificates to undocumented citizens in 2008-2009, and corrected another 46,000).

The Trilateral Criminal Defence project aimed to improve access to justice for impoverished and indigenous persons.  Implemented by the Quebec Bar and the Public Defender’s Offices in Chile and Bolivia, it helped improve the capacity of Bolivian public defenders to protect the rights of accused persons, increased public awareness of rights, and improved the capacity of legal aid offices.  In all, 91 lawyers were trained to protect the rights of the accused, including 43 women and several indigenous professionals.

3.3 Sustainability—Satisfactory

While local administrative capacity was often modest, strong buy-in from Bolivian stakeholders helped to ensure longer-term implementation.

CIDA’s Bolivia program, for the duration of the period under review (2005-2010) was rated in the satisfactory range for ensuring the sustainability of its programming. There were challenges to creating sustainable governance programming, while other sectors attained highly satisfactory ratings (summarised in the figure below).

Figure 6: Sustainability Scores by Programming Sector

3.3.1. The program cultivated support in Bolivia for the long-term extension of health and governance projects.

The evaluation confirms on one hand that government capacity was not sufficient to guarantee the long-term effects of the program’s investments.  It confirms on the other hand that some key results presented relatively strong sustainability factors and would remain after CIDA’s interventions ended.  The program achieved satisfactory scores by cultivating the partnership of the Bolivian government in health and governance programming and encouraging continued commitment to programs even after external funding ceased. Sustainability was encouraged at the local level by transferring responsibilities/ownership to, and securing the commitment of, some municipal and departmental governments.

Health: Health programming in general presented relatively strong sustainability factors.  Indicators of the high probability of lasting effects of the program’s investments are documented in project mid-term and final reports.  The sustainability of multi-sectoral interventions in the health sector was supported by the approval of the National Food and Nutrition Policy.  In projects such as the Program to Support the Health System and Malnutrition Zero, methodologies were scaled up, institutional personnel upgraded their skills, government institutions were empowered and gained leadership, and municipalities adopted budgets for health activities.  One area for improvement would be the development of sustainability plans from the beginning, during the design phase. Other examples of sustainability in health include:

• The National Fund for Productive and Social Investment within the Program to Support the Health System project established standards for local counterparts that were adopted by departmental and municipal governments responsible for maintaining the infrastructure and supplies provided by the National Fund for Productive and Social Investment.
• The Zero Malnutrition Program’s support to the technical programs of the Ministry of Health and Sports resulted in a reduction of 74% in chronic malnutrition within implementing municipalities, and an increase in maternal and child coverage so that the percentage of children under 2 years of age experiencing stunted growth was decreased by 28% over the same period.  Final evaluation reports of the Program to Support the Health System and Zero Malnutrition Program pointed to improvement in coverage of health services, and enhanced skills among health actors in the target areas.
• Training activities for Community Health Workers were devolved to municipal governments;
• An increase in municipal resource allocation for the promotion of maternal and child health; and,
• The adoption of a partnership strategy by some health projects to involve local non-governmental organizations and indigenous organizations; this increases the potential for social and financial sustainability upon termination of the projects.

Governance: In the governance sector, certain project design factors add to the long-term potential of the program’s investments.  Among these were building on long-standing relationships with key Bolivian institutions, (the Bolivian National Institute of Statistics, the Ombudsman’s Office and the National Electoral Court), aligning funding priorities with the institutions’ core mandates, and taking a long-term outlook.

As well the use of pooled funding arrangements encouraged local ownership and therefore sustainability.

3.3.2. Improvement needed in government capacity to guarantee sustainability of projects in all three sectors.

Sustainable Economic Development: New laws and standards for exploration, technical development and management are needed to solidify the outcomes of the hydrocarbon project.  Government funding and other financial incentives to support small-scale economic activity (rural entrepreneurs, cooperatives and other associations) are very limited.  For example, in relation to Fair Trade there have been many declarations but little policy and direction.  Overall, however, the program’s contributions to Bolivia’s reduced reliance on foreign assistance appear to promise more nationally driven sustainable development.

Governance: Most of the program’s governance partners changed their structures, mandates and even staff upon the installation of the Morales administration. This eroded institutional capacity building in which the program had invested over the previous decade.  Also, the politicization of foreign aid delivery in Bolivia following the 2007 foreign aid laws introduced government controls that threatened the autonomy and credibility of initiatives.  Many mid-size institutional partners remained quite dependent on international funding which became less available as several donors reduced funding to governance projects in Bolivia.  The relatively short-term, small-scale projects supported through CIDA’s responsive governance funds (Responsive Fund for Local Governance, Canadian Executive Services Overseas and Gender Equality Fund) also tended to be “one off” activities which partners lacked the technical or financial capacity to sustain.

Health: Similar challenges also affected health sector sustainability:

(1) Multiple changes and staff turnover reduced the sustainability of achievements in capacity development in the central government;

(2) The institutional memory of health interventions tended to reside in extra-national organizations such as UNICEF instead of government institutions; a formal and systematic approach to institutionalize that memory in relevant actors at the central and departmental levels could help overcome this challenge; and,

(3) Some projects did not develop the administrative and financial skills of local actors (e.g. the Nutrition Unit and Departmental Health Service personnel).  Short-term consultants carried out several activities without adequate knowledge or ability to transfer skills, which reduced technical sustainability.

3.4 Crosscutting Theme of Gender Equality – Positive Results

The program was slow to promote gender equality at the design phase, but achieved satisfactory and highly satisfactory results. The promotion of gender equality in sustainable economic development was particularly strong.

For the duration of the period under review (2005-2010), CIDA’s Bolivia program achieved mixed ratings for the effectiveness of its integration of gender as a crosscutting theme.  Ratings by sector are summarised in the figure below.^{Footnote 15}

Figure 7: Gender Equality (Crosscutting Theme) Scores by Programming Sector

3.4.1. The program was slow to integrate gender equality at the design phase.

Overall, the evaluation found that through its gender-focused programming, Canada contributed to important results such as increased political participation by women.  It made gender integration part of policy dialogue among stakeholders, strengthened capacity of women’s organizations, and improved protection of women’s rights.  However, evaluation evidence also noted that in the early phase of the period under review, the program was slow to integrate gender equality at the design phase of most of its projects.  As of 2008, only a few projects had an explicit gender strategy and a gender committee.  In 2010, a mid-term evaluation of the Strategic Governance Mechanism in Bolivia^{Footnote 16} recommended several steps for CIDA to better integrate gender equality at the institutional and program levels.  The program responded with a detailed plan to reinforce the gender perspective in its programming with notable examples of gender integration in bilateral programming and partnership projects.

In addition, through several pooled-funded projects, the program played a leadership role in promoting policy dialogue on gender with other donors and Bolivian government partners.  A 2011-2012 Peer Review of CIDA’s performance in gender integration confirmed progress achieved in this area.  The Peer-Review also recognized that the program could do more to integrate gender as a crosscutting issue.  

3.4.2. Most projects across all three sectors achieved concrete gains toward gender equality.

The present evaluation affirms that satisfactory and highly satisfactory scores reflect the achievement of concrete results.

Sustainable Economic Growth projects (the five projects factored into the sample) were by far the most effective at systematically promoting gender equality, receiving highly satisfactory scores.  The projects included activities that targeted women by promoting economic and political empowerment and professional development.  One project in particular, created 17 women-led and 22 mixed community groups that built capacity in sustainable agriculture and natural resource management.  Many women received training as entrepreneurs and community leaders.  Three projects (World University Service of Canada - Centre for International Studies and Cooperation, Canadian Executive Services Overseas, Société de coopération pour le développement international) in particular trained women entrepreneurs in producing textiles, handicrafts and oregano.  The projects also provided equal opportunity training for all workers (women included).

The success of crosscutting gender equality programming in the Hydrocarbon project is encapsulated by one manager: “Women hold less than 10% of technical or professional jobs in the oil and gas industry, so we’ve still got a long way to go.  But, we have made some progress in changing sexist attitudes and stereotypes, and in giving women more equal opportunity.  The CIDA project helped open the door for women to enter the hydrocarbon sector in Bolivia.”

Health sector programming was the weakest at promoting gender equality.  Only two of the ten projects in the evaluation sample received scores of “highly satisfactory” while the remaining eight earned scores of “satisfactory.”  Stakeholders interviewed for the evaluation suggested that the program “missed opportunities.”

Governance: During the review period, some governance sector programming integrated gender equality effectively.  Of the six governance projects reviewed, four projects received a score of “highly satisfactory” and two “satisfactory.”  Three of the initiatives that performed well were pooled funds with institutional partners.  Among pooled-fund donors, CIDA was the one that most consistently promoted gender mainstreaming in institutional policies and programs.  The program also provided technical assistance and policy advice to support partners in integrating gender equality.

The satisfactory projects helped enable indigenous women to participate in elections and access social services and promoted gender equality in water committees, while the highly satisfactory projects achieved broader results.  One of the four highest-scoring projects, (Municipal Partnership Program with the Federal of Canadian Municipalities) promoted the participation of women councillors in municipal government.  The other three contributed to pooled funds with institutional partners and set a standard for gender equality programming.  That was particularly the case when CIDA and the Netherlands designed and launched the Women’s Participation in the Economy Fund ($20 million) to continue gender-focused programming after the Gender Equality Fund ended.

The Andean Regional Gender Equality Fund, phase three, achieved results in 27 projects in seven Bolivian departments and over seventy municipalities.  It facilitated community support of women’s rights and promoted gender equality on the national level, training women to participate in the country’s decentralization process.  Concrete outputs and outcomes include:

• Political training for 8,283 women and 633 men (at the municipal, and parliamentary levels) to create policies promoting women’s rights;
• Support for 15,000 women (from 1,000 organizations) who proposed changes to Bolivia’s new Constitution that resulted in 34 new articles to promote gender equality; and,
• Support to the Municipal Comprehensive Legal Services to support solutions to gender related crime.

The Gender Equality Basket Fund was not however renewed due to limited capacity of the Bolivian Vice-Ministry of Equality of Opportunities.

Stakeholders interviewed reported “we did not talk much about gender equality until CIDA raised it.  CIDA helped put gender on our agenda.”

3.5 Crosscutting Theme of Environment—Satisfactory

The program promoted environmental sustainability at the design phase, but lacked the technical expertise to realise its aims.

The program made progress toward achieving environmental sustainability. All three programming sectors, achieved scores in the “satisfactory” range and the program complied with the Canadian Environment Assessment Act (CEAA). Sustainable economic development programming scored 3.9; health, 4.0; and, governance, 3.4 (see figure below).

Figure 8: Environmental Sustainability (Crosscutting Theme) Scores by Programming Sector

3.5.1. The program promoted environmental sustainability at the design phase.

The program improved (over the five years of the evaluation period) the integration of environmental issues at the planning stage, and went beyond compliance with the Canadian Environmental Assessment Act (CEAA) in implementing health projects.  The Country Program Framework that ended in 2007 planned for several environment-related results and identified several indicators for monitoring.  The Country Program Strategy of 2009 launched a strategic environmental assessment.  The exercise indicated that for the 2010-2015 period, the program could expect to achieve some positive environmental impacts and few negative ones.  However, more work needs to be done to fully integrate environmental considerations across sectors.  For example, in water and sanitation, the focus was more on securing water supply than on promoting conservation and integrated management of water resources.  In governance, effort was mostly related to compliance with the CEAA.  SEG was the sector in which environment integration was most proactively pursued.

3.5.2. The program lacked the expertise to follow through with implementation

The evaluation demonstrated that the program was in compliance with the Canadian Environmental Assessment Act.  All screenings and forms were cleared as per provisions of the Act and contributed to some environmental projects, resulting in the program’s modest “satisfactory” score.  However, much more remains to be done to improve mainstreaming of environmental sustainability as concluded in the 2009 Strategic Environmental Assessment.  The program lacked specialist support, at headquarters and in the field.  The program also did not play an active role in policy dialogue on environmental issues as other donors have and is not very involved in capacity building of partners.

3.5.3. The program achieved some results in environmental sustainability in all three sectors

Despite the foregoing limitations, attention to crosscutting programming to support environmental sustainability yielded some results.

Sustainable economic growth: Environmental sustainability programming in the sustainable economic growth sector achieved satisfactory ratings.  The Hydrocarbon Project did not contribute any unwanted environmental impacts and it supported energy development practices and funded the Socio-environmental Monitors Training Program.  The projects proactively integrated environmental concerns as outlined below:

• The Canadian Lutheran World Relief project supported the Defence Committee of the Pilcomayo River to advocate for environmental management of the river basin;
• The World University Service of Canada, Uniterra initiative and Fair Trade agenda under the Centre for International Studies and Cooperation promoted environmental sustainability and workplace health and safety standards;
• The second Canadian Executive Services Overseas project promoted Fair Trade, and its strong environmental practices, with at least one group of entrepreneurs; and,
• The Société de coopération pour le développement international project established high benchmarks in relation to organic farming and water resource management in an arid region.

Health: Two of ten health projects promoted environmental sustainability and acknowledged environmental concerns as a determinant to children’s health and nutrition.  The Water and Sanitation Committee and two projects supporting UNICEF (water and sanitation project (PACSAS) and Save the Children (5-year project)), promoted environmentally sustainable water usage and health practices.  The University of Calgary water resources management project identified several priority areas in order to promote environmental sustainability, and the Université du Québec à Montréal (UQAM) Eco-développement communautaire project promoted several ways of addressing environmental sustainability in Bolivia.

Governance sector projects complied with environmental standards, but only two elaborated specific environmental components.  Notably, however, the Canadian Executive Services Overseas program sponsored a Canadian adviser to help municipal governments strengthen solid waste management systems and the Municipal Partnership Program included environmental analysis and results.

4.0 Findings for Management Factors Criteria

The program achieved “highly satisfactory” and “satisfactory” results across all four evaluation criteria measuring management factors.

Figure 9: Overview of Management Factors Scores at the Program Level

4.1 Coherence – Highly Satisfactory

The program demonstrated good examples of external coherence with other donor programs through its involvement in multiple pooled funds and by participating in a key multi-donor coordination group, and internal coherence within CIDA, particularly between the Geographic Program Branch and the Partnership with Canadians Branch.

4.1.1. CIDA worked well with its partners to complement and optimize their relative strengths

The program facilitated external coherence as a founding and active member of the Group of Partners for Development, a group of 23 multilateral and bilateral donors formed in 2006.  Through its leadership, the program supported the group by mediating between donors, non-government organizations and governments, and by contributing to resolution of potential conflicts. The program chaired of Group’s committees, including the Gender Equality Committee convened to coordinate gender projects.  It also participated in six other committees: water and sanitation; health; public administration; decentralization; non-governmental organizations; and gender equality. The following examples of external coherence resulted:

• Save the Children’s work (CIDA funded) to support the Swedish International Development Agency’s (SIDA) priorities and principles regarding the rights of socially excluded children;
• The emergence, under the program’s leadership, of the multi-donor PACSAS fund, despite challenges in engaging the Minister of Health;
• Coherence between the program’s Zero Malnutrition project (UNICEF component) with six other United Nations organizations:  World Food Programme; Pan American Health Organization; United Nations Population Fund; the Food and Agriculture Organization of the United Nations; and, the United Nations Industrial Development Organization;
• The Peer Review of CIDA’s Effectiveness in Gender Mainstreaming conducted by the Inter-Agency Gender Committee (composed of the donor countries) praised the Fund for its leading role in creating coherence among international gender equality investments.

By the end of the review period, the program had earned high commendations from stakeholders for its leadership and contributions to the Group of Partners for Development.

4.1.2. Internal coherence

There were some notable examples of coherence between the Geographic Program Branch and the Partnership with Canadians Branch projects in Bolivia, including:

• The Partnership with Canadians Branch’s Canadian Executive Services Overseas project worked closely with the Local Public Sector Reform Fund and the Bolivia Hydrocarbons Regulatory Assistance Project providing complementary technical assistance to enhance results and sustainability.
• The Plan International Canada project was scaled-up to 26 municipalities and fed into the Malnutrition Zero Program and the “Camino” bilateral project.

While there was not initially a systematic mechanism to promote inter-branch coherence, the recent constitution of the Coordination of Canadian Civil Society Organizations in Bolivia (Coordinación de las Organizaciones de la Sociedad Civil Canadienses en Bolivia) does move in this direction. 

4.2 Efficiency - Positive Results

The program’s efficiency was rated as satisfactory, demonstrating some successes and areas for improvement.

4.2.1. The program maintained generally satisfactory levels of cost efficiency

The 2011 benchmarking exercise of CIDA’s Americas Branch programs on Agency programs in some middle-income countries such as, Peru, Vietnam, Indonesia and Colombia^{Footnote 17} revealed the Bolivia program’s relative success in achieving cost efficiency in comparison to similar programs.  Documentation reviews and interviews carried out during the evaluation process confirmed Bolivia’s favourable performance in terms of cost-efficiency and efficient use of financial resources.

The evaluation, however, identified some limitations affecting program efficiency including:

• Significant delays in project approval seriously affected the timely and effective implementation of several key projects.
• Key Bolivian partners experienced issues disbursing funds on schedule due to their limited institutional and absorptive capacity, and lack of technical expertise to implement programs. This problem was exacerbated by the Government of Bolivia’s lengthy administrative processes; and,
• Local funds (e.g., Responsive Fund for Governance, Local Fund for Health Sector Support) tended to have higher average transaction costs for small projects than larger bilateral projects. However, local funds enabled the program to explore and test new models and methodologies, and to spread risk between different approaches (e.g., budget support, grants, and contributions).

It is worth noting that the program agreed to deliver health services in some remote areas that were difficult to access, less populated, and had limited administrative capacity at the department and municipal levels.  Even though working in these conditions proved to be costlier per beneficiary, it contributed to an overall and rational division of labour among donors.  Other donors praised this approach.

4.2.2. The degree of decentralization affected efficiency

CIDA’s Aid Effectiveness Action Plan 2009-2013 aimed to increase the effectiveness of the development cooperation program.  However, during the review period restrictions on the level of decentralised approval authority, (limited to $100,000), affected the ability of the program to plan and deliver in efficient timeframes with partners and other donors in Bolivia.

4.2.3. Human resources management affected efficiency

Stakeholders praised the leadership, open-mindedness and flexibility demonstrated by the field team, assisted by experienced analysts at the Program Support Unit.

There was however relatively high turnover during the review period.  Three heads of aid managed the program in the field between 2005 and 2010.  The number of consultants and staff at the Program Support Unit fluctuated given changes in the program during the period, when there was also uncertainty due to lack of a long-term contract.

Management Principles—Highly Satisfactory

The program showed strong adherence to the 2005 Paris Declaration principles but its promotion of local ownership fluctuated during the evaluation period.

4.2.4. Successful promotion of ownership

The 2003-2007 CDPF in effect during the review period resulted from a two-year consultative process with Bolivians, Canadians and the international aid community where Bolivians played a leading role in identifying areas of intervention.  It was based on thorough analysis of Bolivian needs, priorities, capacities, Canadian value-added, and other donor programming.

During the development of the 2009 CPS and 2010-2015 CDPF, the program appeared less committed to the spirit of Paris Declaration principles.  The 2009 CPS reflected the Agency’s new thematic priorities, and while within the country’s overall development needs, did not emerge explicitly from consultation with the Bolivians.  CIDA chose two thematic priorities (health and sustainable economic development).  Likewise in 2011, CIDA selected health as the sole sector of intervention.

At the sectoral and project levels, the degree of local ownership varied.  The governance program tended to have a high degree of local appropriation.  Projects funded through responsive local funds generally had high local ownership because they were fully designed and implemented by Bolivian partners.  Pooled funds built the capacity of institutional partners to plan, implement, and monitor their strategic plans and priorities.

Despite repeated efforts by CIDA, health sector projects tended to have limited ownership by the Government of Bolivia, reflected in low absorptive capacity and administrative problems.

The program promoted ownership in several SEG projects, for example developing the capacity of the Ministry of Hydrocarbon and Energy and Yacimientos Petroliferos Fiscales Bolivianos to prepare and implement their sectoral strategies.  

4.2.5. Alignment was a challenge given the changing Bolivian and CIDA priorities

Alignment with Bolivian development priorities was a challenge, mainly because national plans were weak and Government of Bolivia’s priorities kept changing.  The 2006-2011 NDP was very broad, and was more a vision than an operational development plan with clear guidelines.  CIDA also re-articulated its own priorities during the evaluation period.  Despite these limitations, individual projects were generally aligned with Bolivian partner priorities.  When CIDA modified the programming sectors of priority, the program consulted with the Government of Bolivia in order to ensure alignment with their priorities for those sectors.

The level of alignment depended to some degree on the delivery channel.  For example, many Sustainable Economic Growth projects managed by Canadian and Bolivian partners tended to be designed and delivered locally.  Given that these actors did not have strong working relations with the Government of Bolivia, and funding came most of the time from independent sources (donors, international non-governmental organizations) they were less aligned with Government priorities.

4.2.6. The program was relatively successful in participating in policy dialogue

CIDA was one of the founders of the Group of Partners for Development in 2006 and played an active role in the group during the review period.  Stakeholder interviews revealed appreciation for the initiative in promoting harmonization and policy dialogue.  Interviewees recognized CIDA as a leader in gender equality and a strong contributor to policy dialogue, particularly in the health sector.

“Canada tried to remove bottlenecks in relations between the government and donors.  It helped the (Group of Partners for Development) mediate conflicts.  For example, when Canada was the leader of the Group’s (non-governmental organization) subgroup, it played a very constructive role in resolving the issue of tax treatment of non-governmental organizations.  Canada took that issue to the very high levels of government, and got the dialogue started.  Canada showed real leadership and political acumen.” -Donor representative

“Canada helped establish dialogue and improve relations with the Bolivian government at a very difficult time.  Canada helped close the distance that had existed between donors and the government for a few years.  The Head of Aid had innate diplomatic skills, and his leadership was very appreciated here.” - Donor representative.

However, the program’s credibility in the field suffered from changes in CIDA country priorities, the lack of long-term funding, budget cuts, and long project approval times. For example, for nearly two years, CIDA negotiated with other donors a large-scale water and sanitation project that did not receive final approval. In another instance, the program participated in multi-donor discussions to develop a program supporting decentralization, and then had to withdraw following a funding decrease in the governance sector.

4.3 Performance Management—Challenges Noted

Performance management was generally applied systematically at the project level but weaknesses were observed at the program level limiting the program’s ability to communicate results.

4.3.1. The program became an advocate for Results Based Management

After implementing CIDA’s Result-based Management policy in 2008, the program became a strong advocate for building results-based approaches into its projects.  It provided training and technical assistance to partners in Bolivian government and civil society in implementing and executing results-based management, and in monitoring and evaluation.

In the health sector in particular, results-based monitoring trickled down to the departmental and municipal levels.  Local health partners (Departmental Health Services) appreciated the training and technical assistance they received in results-based management.

The Performance Measurement Framework (PMF) for 2003-2007 was developed at a time when CIDA was experimenting with different models.  The format used did not provide a clear overview of planned results with measurable indicators.  In addition, baseline data was not established, thus it was difficult to monitor and evaluate results at the program level.  Results could not be assessed at the impact level, as is often the case.

Nevertheless, results-based management principles and practices were applied quite systematically at the project level.  Most logic models followed results-based management principles, although some outcome statements were too ambitious and lacked realistic indicators.  The program often monitored results of bilateral funding to pooled funds with a system harmonized with that of partner institutions.  These systems proved effective in promoting CIDA’s results-based management values.

CIDA reporting requirements were not consistent from one project to the other.  The quality of project reports was generally good, although some focused more on activities and outputs than on results achieved.

4.3.2. Demonstrated improvement of monitoring and evaluation

At the project level, monitoring and evaluation became more systematic after 2007, based on a recommendation from the 2007 Country Program Evaluation.  Some projects had long-term monitoring strategies and were being tracked by the program and the Program Support Unit.  Eleven of the 24 projects (or project components) in the sample underwent independent evaluations.  The evaluation reports that were available were generally of sufficient quality to derive significant and relevant results, and presented qualitative results in an adequate manner.  Demonstrating concrete results remained a challenge, however, especially for governance-related projects.

4.3.3. The program fostered effective risk management at the project level

The 2007 Country Program Evaluation recommended that the program should place “greater effort on risk analysis and contingency planning and on periodic review of the project design to ensure that the project continues to be supported by the government and continues to be sustainable”.  The program took some steps in response.  The 2010-2015 CDPF incorporated more regular risk analysis and mitigation into the Performance Management Framework.  Risk mitigation strategies were also updated annually in project performance reports. 

Risk management at the country-program level, however, had some shortcomings:

• The program did not use its risk register as an ongoing program management tool;
• Program staff in the field and headquarters did not regularly assess risks or necessary mitigation at the program or sectoral levels; and,
• CIDA headquarters did not have mechanisms to effectively integrate risk analysis conducted by individual field officers. 

At the project level, risks were identified and mitigated quite effectively.  For example, the program used a risk management tool for small pilot projects, which tested new partners before they got approval to receive larger bilateral support.

4.3.4. Information and knowledge management – an important challenge

Data collection proved to be a challenge during the evaluation.  The program generated and accumulated an impressive quantity of useful information, data and documents.  However, this knowledge was not well organized or easily accessible.

The program did not have an effective knowledge management system and did not effectively communicate or disseminate information about its works and achievements.  There was no corporate process to analyze or systemize lessons learned.  Many stakeholders reported that the Agency has some good stories to tell about its work in Bolivia but they had difficulty finding basic information about the Bolivia Country Program’s strategy and policies in Bolivia.

5.0 Conclusions

The evaluation provides evidence that the program helped improve living conditions and standards in the key programming sectors of health, governance, and SEG.  Factors contributing to success include a focus on building consistent and long-term relationships with Bolivian counterparts, and alignment of development programming under Bolivian priorities with other donors.

Policy dialogue, in particular, proved effective in facilitating results across all sectors of priority.  In governance and in gender equality, a participatory approach allowed the program to negotiate the scope of strategic technical assistance and helped to secure government, donor and civil society organization participation.  Canadian interventions contributed directly to the strengthening of important national institutions, including the Auditor General, the National Electoral Court, and the Ombudsman.

On the basis of evidence from the sample of projects, key interviews, and relevant documentation, the following conclusions can be drawn on the evaluation criteria:

Relevance: While navigating changes to political leadership in Bolivia and corporate orientation at CIDA, the program remained relevant to Canadian strategic priorities and Bolivian needs.  It delivered aid across priority sectors according to the 2003-2007 Country Development Programming Framework and the revised 2009 Country Program Strategy.  A major strength was the focus on building consistent, long-term relationships with key partners and beneficiaries through a highly participatory process.

Effectiveness: There were good development results achieved over the review period, particularly in SEG where the hydrocarbons project contributed to the country’s ability to benefit and redistribute revenues by establishing a more transparent and efficient tax and royalty collection system.  The program also contributed to better health services for the most vulnerable, and facilitated decentralization of services and strengthened governance and democratic institutions. 

Sustainability: Encouragement of ownership by Bolivian stakeholders and other donors built on Bolivia’s capacity to sustain some of the program’s investments.  This facilitated improvement in long-term institutional capacity, particularly in the hydrocarbons sector.  However, program investments in governance and health were rated as less sustainable due to the difficulties of developing strong local administration, particularly in the health sector.

Gender Equality: During the early part of the review period, integration of gender equality across the portfolio was minimal given that the local gender specialist was focused on managing CIDA’s Gender Equality Fund.  In 2010, Holland became the lead on the Gender Equality Fund, which allowed the gender specialist to improve the integration of gender equality as a crosscutting issue in the program.  The program did contribute to important results through its gender-focused programming, such as increased political participation by women, strengthened capacity of women’s organizations, and improved protection of women’s rights.  The program also played a leadership role in promoting policy dialogue on gender equality with other donors, Bolivian government partners and non-governmental organizations.

Environmental Sustainability: The evaluation demonstrated that in Bolivia, applying the Canadian Environmental Assessment Act was not enough to ensure full integration of environmental considerations across the program.  A case in point was water and sanitation where the focus was more on securing water supply than in promoting conservation and integrated management of water resources.  The program did nonetheless provide some good examples of promoting and integrating the environment, particularly in the SEG sector.  There was still room for improvement in the mainstreaming of environmental issues in the country program and projects, as recommended by the strategic environmental assessment conducted in 2009. 

Coherence: The program demonstrated good examples of external coherence with other donor programs through involvement in multiple pooled funds and by participating effectively in a key multi-donor coordination group.  The evaluation also found relatively good internal coherence between the Geographic Program Branch and Partnership with Canadians Branch on some key projects, although a more systematic approach to coordination with civil society organizations would have been useful.

Efficiency: Based on document review including the 2011 Americas Program benchmarking exercise, the program performed favourably in terms of cost-efficiency, when compared to similar CIDA programs in other middle-income countries.  As well, there was an appropriate blend of larger pooled fund investments with relatively lower overhead cost ratios, and smaller but innovative remote area health interventions with relatively higher overhead cost ratios.  However, several limitations, also evident in other Country Programs, affected efficiency during the evaluation period, including high staff turnover, delays in project approval and limited delegation of spending authority.

Management Principles: The program demonstrated strong evidence of adherence to the principles of the Paris Declaration on Aid Effectiveness (ownership, harmonisation and alignment).  Policy dialogue was also a strength, facilitated through involvement in a number of pooled funds.  Even with shifting political circumstances in Bolivia and new articulation of corporate priorities at CIDA, general alignment of program interventions with Bolivian priorities was maintained.

Performance Management: At the project level, results-based management principles, monitoring, evaluation and risk management practices were generally applied systematically.  There was also a program level performance management framework in place, but it did not provide a sufficient overview of planned results with measurable indicators, and baseline data was not established, which limited the ability to articulate broader achievements and results. Knowledge management, communication and dissemination of information about the program’s works and achievements could have been stronger.

6.0 Recommendations

A number of recommendations arise from the evaluation findings and conclusions:

• Building ownership and alignment of CIDA programming through engagement and consultation with partners has been a strength of the program during most of the review period.  It is recommended that the program make a renewed commitment to this approach in its future program planning efforts.
• In light of limited institutional capacity in most sectors in Bolivia, effectiveness and sustainability will require concerted attention.  It is recommended that future engagement include a focus on capacity development, taking into consideration how Canadian value added and knowledge can best be brought to bear in the development of explicit plans for major institutional partners,
• The program should build on progress in strengthening integration of gender as a crosscutting theme by:
  •  Ensuring that all projects include a gender analysis, strategy, indicators and targeted resources;
  •  Developing a standard gender monitoring and reporting system to assess gender integration across all projects and sectors; and
  • Sharing knowledge with partners and stakeholders about gender challenges, opportunities and lessons.
• The program should improve mainstreaming of environmental sustainability by developing an environment implementation plan, and working with like-minded donors to integrate environmental considerations into pooled fund objectives.  Depending on the nature of future investments, the program should make use of positive lessons gained from past SEG programming, including building partner capacity to implement environmental standards.
• While a program level performance management framework has been articulated, the program should ensure that it specifies clear expected results with verifiable indicators, and to the extent possible collect adequate baseline information to facilitate eventual evaluation.
• There is scope for greater visibility of program accomplishments with partners and stakeholders.  The program should develop an appropriate communications strategy to raise its profile.

Annex 1: Detailed Project Sample

Annex 2: List of Documents Reviewed

General Documents

CIDA

• ACDI-CIDA. (n.d.). Risk Register: Bolivia Program.
• ACDI-CIDA. (2002). Bolivia Country Development Programming Framework, 2003-2007.
• ACDI-CIDA. (2007). Bolivia Country Program Evaluation. Executive Report.
• ACDI-CIDA. (2009). Bolivia Program. Americas Directorate. Country Development Programming Framework (CDPF) 2010-2015.
• ACDI-CIDA. (2009). Bolivia CPS. SGDE-EDRMS-#4120991-v15.
• ACDI-CIDA. (2010). Revue des approaches-programmes à l’ACDI. Rapport Synthèse.
• ACDI-CIDA. (2011). Countries of focus. [http://www.acdi-cida.gc.ca/countriesoffocus].
• ACDI-CIDA. (2012). Bolivia Country Program Evaluation. Scoping Mission -June 10- June 15th, 2012

Scoping Mission Report.

• ACDI-CIDA. (2012). Development Effectiveness Review – United Nations Development Programme (UNDP).
• ACDI-CIDA. (2012). MGPB Country Profile: Bolivia.
• GOSS GILROY INC. (2007). Bolivia Country Program Evaluation. Final Report.
• LAINE, J. (ACDI-CIDA). (n.d.). Notes from CIDA’s La Paz Mini-Retreat December 3, 2009.
• LAINE, J. (ACDI-CIDA). (n.d.). Operationalizing Aid Effectiveness: Rationale and Overview for the Bolivia Program’s Team Retreat.
• LAINE, J. (ACDI-CIDA). (n.d.). Report on Bolivia Program Team Planning Retreat March 1-3, 2011.
• LAINE, J. (ACDI-CIDA). (2010). A Draft Implementation Framework for CIDA’s Bolivia Country Program 2009.
• MAECI-DFAIT. (2011). Evaluation of the Americas Strategy. Final Report.
• PIERRE-JEROME, F. (ACDI-CIDA). (2012). Evaluation Background Profile. Bolivia Program (FY 2005-2010).
• UASCC. (n.d.). Tourist Information Bolivia and the City of La Paz.
• UASCC. (2011). Manual Administrativo de la UASCC.

COCAB

• (n.d.). Plan de Actividades de la Coordinadora de la COCAB (Mai Le) : Septiembre 2012-Agosto 2013.
• (n.d.). Presupuesto Estimado : Septiembre 2012-Agosto 2013.
• (2012). Agenda del Dia. Primer encuentro de implementación de la plataforma de ONG’s canadienses en Bolivia.
• (2012). Agenda del Dia. Reunión de Coordinación de las Organizaciones de la Sociedad Civil Canadienses en Bolivia.
• (2012). Resumen del Dia. Reunión de Coordinación de las Organizaciones de la Sociedad Civil Canadienses en Bolivia.
• PLATAFORMA DE ONG'S CANADIENSES EN BOLIVIA. (2012). Primer encuentro de implementación de la plataforma de ONG’s canadienses en Bolivia. Presentación del trabajo del comité ad hoc de la plataforma. PPT.
• PEREYRA, R. (2011). Reunion Sociedad Civil ACDI 2011. PPT.

Government of Bolivia

• GACETA OFICIAL DE BOLIVIA. (2007). Plan Nacional de Desarrollo. “Bolivia Digna, Soberana, Productiva y Democrática para Vivir Bien”. Lineamientos Estratégicos 2006-2011. [http://www.sns.gob.bo/planificacion/documentos/5_PND%20VERSION%20OFICIAL.pdf].
• NARVAEZ GUZMAN, R.B. (dir.). (2010). Evaluación Nacional de la Declaración de París en Bolivia. Segunda Fase/ National Evaluation of the Paris Declaration in Bolivia. Second Phase.

Other Bilateral and Multilateral Donors

• AUSAID. (2011). Policy Dialogue Evaluation – Terms of Reference.
• CUSO INTERNACIONAL. (n.d.). Cuso Internacional Volunteers. Live stories.
• CUSO INTERNACIONAL. (n.d.). Programa Inclusión Economica y Social de la Juventud. Programa regional Bolivia, Perú, Salvador y Jamaica. CUSO-VSO. 2009-2014. Resumen Ejecutivo.
• CUSO INTERNACIONAL. (n.d.). Sembrando Vientos de Cooperación en Bolivia. Video.
• CUSO INTERNACIONAL. (2012). Programa y Acciones en Bolivia 2009-2014. PPT.
• DANIDA. (2005). Asociación entre Bolivia y Dinamarca. Estrategia de Cooperación para el Desarrollo 2005-2010. ISBN 87-7667-268-9.
• DANIDA. (2012). Danida 50 years 1962-2012. ISBN 978-87-7087-612-2.
• DFID. (2008). Monitoring Policy Dialogue: Lessons from a Pilot Study. Evaluation Report WP27.
• EMBAJADA DE BÉLGICA, OFICINA DE COOPERACIÓN AL DESARROLLO. (2012). Actividades de la Cooperación belga al Desarrollo en Bolivia.
• EUROPEAN COMMISSION. (2007). Bolivia Country Strategy Paper 2007-2013.
• FLINT, M., RANCE, S. and RICHARDSON, L. (PARC) (2005). Evaluation of DFID Country Programmes. Country Study: Bolivia 2000-2004.
• GOVERNMENT OF JAPAN. (2009). Country Assistance Program for the Plurinational State of Bolivia.
• INTER-AMERICAN DEVELOPMENT BANK (IDB). (2004). Bolivia. The Bank’s Country Strategy with Bolivia (2004-2007).
• INTER-AMERICAN DEVELOPMENT BANK (IDB). (n.d.). Bolivia. The Bank’s Country Strategy with Bolivia 2008-2010.
• INTER-AMERICAN DEVELOPMENT BANK (IDB). (2004). Country Program Evaluation (CPE) Bolivia 1990-2002.
• INTER-AMERICAN DEVELOPMENT BANK (IDB). (2010). Managing for Development Results. Progress and Challenges in Latin America and the Caribbean. ISBN 978-1-59782-137-7.
• INTER-AMERICAN DEVELOPMENT BANK (IDB). (2011). Country Program Evaluation: Bolivia 2008-2010.
• INTERNATIONAL LABOUR ORGANIZATION (ILO) – EVALUATION UNIT. (2012). I-eval Resource Kit. Guidance Note 8.
• INTERNATIONAL MONETARY FUND (IMF) AND THE INTERNATIONAL DEVELOPMENT ASSOCIATION (IDA). (2001). BOLIVIA – Poverty Reduction Strategy Paper Joint Staff Assessment.
• KLIEST, T. (coord.) (2010). Examen par les paris de la fonction d’évaluation du Service public fédéral belge Affaires étrangères, Commerce extérieur et Coopération au développement. Rapport final.
• SIDA. (n.d.). Estrategia de País Bolivia 1 de Enero de 2003- de 31 Diciembre de 2007.
• UNDP. (2011). International Human Development Indicators.       [http://hdrstats.undp.org/en/countries/profiles/BOL.html].
• WORLD BANK. (2004). Report and Recommendation of the President of the International Bank for Reconstruction and Development, International Development Association, International Finance Corporation and the Multilateral Investment Guarantee Agency to the Executive Directors on a Country Assistance Strategy for the Republic of Bolivia.
• WORLD BANK. (2006). Bolivia. Towards a New Social Contract Options for the Constituent Assembly.
• WORLD BANK. (2011). International Development Association, International Bank for Reconstruction and Development, International Finance Corporation and Multilateral Investment Guarantee Agency Country Partnership Strategy for the Plurinational State of Bolivia for the Period FY2012-2015. Report No. 65108-BO.
• WORLD BANK. (2012). Bolivia Data. [http://data.worldbank.org/country/bolivia#cp_wdi].

Sector/Programming Area Documents

Democratic Governance

• (n.d.) Final Terms of Reference – Evaluation of civil society engagement in policy dialogue.
• ACDI-CIDA. (2012). CIDA and Civil Society in Bolivia. Aid Effectiveness and Country Programs Unit Partnerships with Canadians Branch, CIDA. Draft. EDRMS 4456557.
• ALBARRACÍN SÁNCHEZ, W. (n.d.) Bolivia a Treinta Años de Democracia.
• ESPADA, J.L. and EROSTEGUI, S. (n.d.). Development effectiveness in Bolivia: A process in gestation. In TOMLINSON, Brian (ed.). (2011). Democratic Ownership and Development Effectiveness: Civil Society Perspectives on Progress since Paris. The Reality of Aid International Coordinating Committee. IBON Books. ISBN 978-971-0483-96-7.
• GANDER, C. (2010). Mid-Term Evaluation of Strategic Governance Mechanism in Bolivia (CIDA). Final Report.
• LOK, J. (ACDI-CIDA). (n.d.). Impact of Canadian Aid on Democratic Governance in Bolivia (2008-2011).

Health

• MESSINA, M. (coord.) (2012). Revision Analitica de los Programas de Agua, Saneamiento e Higiene de UNICEF en Países de América Latina y el Caribe. Informe Final.
• CIDA (SAINT-LAURENT, L)  (2011). CIDA Bolivia Programme Health Strategy 2011-2020. Draft.

Economic Growth

• JORDAN, J.A. (UASCC). (2012). El Sector Extractivo en Bolivia y la Responsabilidad Social Corporativa (RSC). Análisis de Situación. Andean Regional Initiative (ARI), Promoting Effective Corporate Social Responsibility, ACDI.
• (n.d.). Report on Treaty 8 Tribal Association Projects in Bolivia.

Crosscutting Issues Documents

Gender Equality

• ACDI-CIDA. (n.d.). Gender Equality Strategy – Bolivia Program.
• ACDI-CIDA. (2004). Gender Equality Action Plan – Bolivia Country Program.
• COMITÉ INTER-AGENCIAL DE GÉNERO EN BOLIVIA (CIAG). (2012). Revisión de Pares/Peer Review del Trabajo de Género de la Embajada del Canadá y la Agencia Canadiense de Desarrollo Internacional (ACDI) en Bolivia.
• FERREYRA, AIDA. (2012)  Género.  Análisis de Contexto. 
• LAFORCE, J. (C.A.C. International). (2007). Evaluation of the CIDA GE Policy Implementation in Bolivia. Field Mission Report.
• SALINAS MULDER, S., DOLORES CASTRO, M. and ZUAZO ARANA, S. (2010). Análisis de los cambios en la equidad de género en Bolivia en el período 2004-2009. Informe Final.

Environment

• BAASTEL. (2009). Strategic Environmental Assessment – Bolivia’s Country Strategy.
• CUSO INTERNACIONAL. (n.d.). Protecting Environments Managing Natural Resources.

Other Documents

• (n.d.). Evaluation Basket Funding Bolivia.
• BIRDSALL, N., KHARAS, H. and PERAKIS, R. (2012). The Quality of Official Development Assistance Assessment 2009: Is Aid Quality Improving? Centre for Global Development & Brookings.
• FORSS, K. and BANDSTEIN, S. (2008). Evidence-based Evaluation of Development Cooperation: Possible? Feasible? Desirable? NONIE Working Paper no. 8.
• GOSS GILROY INC. (2012). Formative Evaluation of CIDA’s Volunteer Cooperation Program (2009-2014). Draft Final Report.
• JONES, H. (2011). A Guide to Monitoring and Evaluating Policy Influence. Background Note. Overseas Development Institute (ODI).
• ROTAROU, E.S. and UETA, K. (2010). Foreign Aid Effectiveness in Bolivia. Kyoto University, Japan.

Project Documents by Sector

Democratic Governance

Project: A020910018 – Trilateral Criminal Defence/Barreau Qc

Planning Documents

• ACDI-CIDA. (2005). Memorandum.

Financial Reports

• ACDI-CIDA. (2010). Quarterly Financial Report and Budgetary Forecast.

Reports

• BARREAU DU QUÉBEC. (2010). Tripartite Project Technological Transfer in the Field of Criminal Defense Between the Quebec Bar and the Bolivian and Chilean Public Defender Offices. Bolivia Programme. Project # A-020910-018-PR1. Agreement # 7049154. Project Closing Report 16th January, 2008 – 31st March, 2010.

Project: A030421001 – Public Sector Reform (Bolivia)

Planning Documents

• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• ACDI-CIDA. (n.d.). Evaluación de medio termino del fondo local de apoyo a las reformas del sector público proyecto ACDI no.A030421. Resumen Ejecutivo.
• ACDI-CIDA. (n.d.). Fondo local de apoyo a las reformas del sector público. Fichas esquemáticas.

Project: A030740001 – CESO Bolivia

Planning Documents

• ACDI-CIDA. (2001). Project Approval Document (PAD).
• ACDI-CIDA. (2001). Preliminary Environmental Assessment.
• ACDI-CIDA. (2001). Annex to PAM – Women in Development (WID).
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Reports

• CESO. (2006). Bolivia Public Sector Reform Support Project. CIDA Project #7013432. February 11, 2001 – May 31, 2006. End-of-Project Report.
• EVAPLAN CONSULTATION INC. (2004). Bolivia Public Sector Reform Support Project. CIDA Project #7013432. Mid-term Evaluation Report. Final Report.

Project: A032846001 – Peacebuilding Initiative with CNE

Planning Documents

• ACDI-CIDA. (n.d.). Project Summary.
• ACDI-CIDA. (n.d.). Memo.
• ACDI-CIDA. (2005). Project Approval Document (PAD).
• ACDI-CIDA. (2005). Logical Framework Analysis.
• ACDI-CIDA. (2005). Strategic Environmental Assessment.
• ACDI_CIDA. (2005). Canadian Environmental Assessment Act (CEAA) Applicability.
• ACDI-CIDA.(2005). Gender Equality Assessment.
• ACDI-CIDA.(2006). Mémoire à la Ministre.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• ACDI-CIDA. (n.d.). Matriz de resultados.
• ACDI-CIDA. (2006). 2005/2006 Project/Program Performance Report.
• ACDI-CIDA. (2007). 2006/2007 Project/Program Performance Report.
• CNE.(n.d.). Corte Nacional Electoral. Informe de Actividades Gestión 2008.
• CNE.(n.d.). Corte Nacional Electoral. Informe de Actividades Gestión 2009.
• CNE.(n.d.). Informe de Actividades Gestion 2009. Canasta de Fondos. PPT.
• ÓRGANO ELECTORAL PLURINACIONAL. TRIBUNAL SUPERMO ELECTORAL. (2010.). Informe de Actividades Gestion 2010.

Project: A030422001 – Institutional Support Defensor del Pueblo

Planning Documents

• ACDI-CIDA. (2000). Preliminary Environmental Assessment.
• ACDI-CIDA. (2000). Annex to PAM – Women in Development (WID).
• ACDI-CIDA. (2002). Project Approval Document (PAD).
• ACDI-CIDA. (2002). Logical Framework Analysis.
• ACDI-CIDA. (2003). Contribution Arrangement between the Government of Canada and Defensor del Pueblo.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• ACDI-CIDA. (2005). 2004/2005 Project/Program Performance Report.
• ACDI-CIDA. (2006). 2005/2006 Project/Program Performance Report.
• ACDI-CIDA. (2007). 2006/2007 Project/Program Performance Report.
• Aguilar & Asociados S.R.L. (2005). Evaluación de Medio Término del Plan Estratégico Quinquenal. Resumen Ejecutivo.
• Grupo Integral S.R.L.(2006). Informe de la Evalución Externa Final del Plan Estratégico Quinquenal 2001-2005. 

Project: A032217001 – Strategic Governance Mechanism (SGM)

Planning Documents

• ACDI-CIDA. (n.d.). Decision Memo.
• ACDI-CIDA. (2005). Gender Equality Assessment.
• ACDI-CIDA. (2006). Logical Framework Analysis.
• ACDI-CIDA. (2006). Project Approval Document (PAD).
• ACDI-CIDA. (2012). Project Detail from CRAFT.
• INE. (2007). Plan Estratégico Institucional 2005-2009.
• SUMMERBELL, J. (ACDI-CIDA). (2005). Concept Paper. Strategic Governance Mechanism (SGM).

Financial Reports

• INE. (2009). Remisión ejecución presupuestaria gestión 2008 Basket Fund.
• INE. (2009). Ejecución del presupuesto por nivel de asignación y fecha de corte.

Reports

• ACDI-CIDA. (2008). 2007/2008 Project/Program Performance Report.
• ACDI-CIDA. (2009). 2008/2009 Project/Program Performance Report.
• ACDI-CIDA. (2010). Management Summary Report.
• ACDI-CIDA. (2012). Management Summary Report – Investment.
• ACDI-CIDA. (2012).  Clasificacion de Documentos del SGM.  PPT.
• ACDI-CIDA. (2012).  Resultados de 24 Proyectos del FRAG. 
• ACDI-CIDA. (2012).  Presentación de Resultados del Proyecto de Género del Ministerio de Autonomías.
• ACDI-CIDA. (2012).  Informe Final, Proyecto del FRAG con Ministerio de Autonomías.
• ACDI-CIDA. (2012).  Reporte de Cierre, Proyecto del FRAG con Ministerio de Autonomías.
• DANISH INSTITUTE FOR HUMAN RIGHTS. (n.d.). Plan Estratégico Institucional (2007-2011) de la Defensoría del Pueblo de Bolivia. Revisión de Medio Término.
• GALLARDO, ELIANA. (2012) Orientaciones para la Incorporación de la Perspectiva de Género en el Mecanismo Estratégico de Gobernanza-SGM.  
• INE. (n.d.). Informe de seguimiento y evaluación de actividades y proyectos enero – diciembre 2008.
• JUST GOVERNANCE GROUP. (2012). Evaluación final plan estratégico institucional de la Defensoría del Pueblo de Bolivia 2007-2011.
• RIVAS DÁVILA, F. (2009). Evaluación POA 2008 al Cuarto Trimestre.
• (n.d.). Defensor del Pueblo. Programa de Operaciones Anual 2008. Evaluación Anual.
• (n.d.). Defensoría del pueblo. Ejecución Física – Financiera del POA – Gestión 2009 por proyectos.
• (2009). Programa de operaciones anual, 2009. Resultados esperados. PPT.
• (2009). Informe de Evaluación Anual. Programa de operaciones anual, 2009.
• (2011). Evaluacion anual “programa de operaciones anual 2010”.
• (2011). Evaluacion anual “programa de operaciones anual 2010”. PPT.

Project: A033735PRG – Municipal Partnership Program 2007-2010

Planning Documents

• FEDERATION OF CANADIAN MUNICIPALITIES (FCM). (2010). Municipal Partners for Economic Development (MPED). Detailed proposal.

Reports

• FEDERATION OF CANADIAN MUNICIPALITIES (FCM). (2010). FCM Municipal Partnership Program 2007-2010. End of Program Report June 2007 – July 2010. Vol. 1.
• FEDERATION OF CANADIAN MUNICIPALITIES (FCM). (2010). FCM Municipal Partnership Program 2007-2010. End of Program Report June 2007 – July 2010. Vol. 2 – Annexes.
• GUAY, M. & KOTÉ, G. (RBMG). (2010). Évaluation du programme des partenariats municipaux (PPM) 2007-2010 de la fédération canadienne des municipalités (FCM) et la comparaison des modalités de programmation. Rapport final d’évaluation.
• KASSAM, Y. & TURCOT, P. (E.T. Jackson and Associates Ltd.). (2005). Mid-term Evaluation of the Municipal Partnership Program of the Federation of Canadian Municipalities: Towards Implementing a Strategic Approach. Final Report Executive Summary.
• KASSAM, Y. & TURCOT, P. (E.T. Jackson and Associates Ltd.). (2005). Mid-term Evaluation of the Municipal Partnership Program of the Federation of Canadian Municipalities: Towards Implementing a Strategic Approach. Final Report.

Health

Project: A032896001 - PACSAS

Planning Documents

• ACDI-CIDA. (2005). Program Approval Document for PACSAS.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Reports

• ALVETEG, T. et. al. (2009). Monitoreo del Programa Multidonante de Agua, Saneamiento e Higiene de UNICEF en Bolivia, Primera misión, septiembre octubre de 2008.
• ALVETEG, T. et. al. (2009). Monitoreo del Programa Multidonante de Agua, Saneamiento e Higiene de UNICEF en Bolivia, Segunda misión, septiembre de 2009.
• ALVETEG, T. et. al. (2011). Monitoreo del Programa Multidonante de Agua, Saneamiento e Higiene de UNICEF en Bolivia, Tercera misión, octubre 2010.
• UNICEF. (2012). Analisis de Logros e Impactos del Programa Multidonante de Agua, Saneamiento e Higiene en Bolivia 2006-2011 y Informe Anual Programático/Financiero 2011.

Evaluations

• ALVETEG, T. and ARRATIA, A. (2012). Análisis final del Programa Multidonante de Agua, Saneamiento e Higiene entre el Gobierno de Bolivia y UNICEF 2006-2011.

Project: A031442001 – MSD (PASS)

Planning Documents

• ACDI-CIDA. (n.d.). Logical Framework Analysis.
• ACDI-CIDA. (2007). Acuerdo de Contribución PASS.
• ACDI-CIDA. (2007). Plan de Implementación del Programa (PIP).
• POOLEY, B. (2008). Propuesta de Estrategia de Equidad de Género en el Programa Pass y el Componente PRICCAS.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• ACI-CIDA, and UNICEF. (n.d.). Informe Anual, Departamento del Beni – Gestión 2008.
• ACDI-CIDA, and UNICEF. (n.d.). Informe Anual, Departamento de Pando – Gestión 2008.
• ACDI-CIDA. (2011). Informe Evaluación Final Salud ACDI/BENI.
• COUPAL, F., JOHNSON, L. AND GUTIERREZ, W. (2009). Mid-Term Evaluation of the Bolivian Health Support Programme.

Project: A031442004 – FPS (PASS)

Planning Documents

• ACDI-CIDA. (2007). Acuerdo de Contribución PASS. Representado por el FPS.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• FPS. (n.d.). Informe Avance Físico Financiero 2008. [n.p.] [n.d.].
• FPS. (2008). Detalle Pagos Programa de Apoyo al Sector de Salud PASS.
• FPS. (2009). Informe Avance Físico Financiero.

Project: A033079001 – Grant to PAHO-Bolivia

Planning Documents

• ACDI-CIDA. (2005). Logical Framework Analysis (LFA).
• ACDI-CIDA. (2005). Project Approval Document (PAD).
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Project:  A033957001 – MSD Budgetary Support and Evaluation

Planning Documents

• ACDI-CIDA. (2012). Project Detail from CRAFT.

Reports

• COMITÉ TECNICO DEL CONSEJO NACIONAL DE ALIMENTACION Y NUTRICION (CT CONAN)- MSD. (n.d.). Programa Multisectorial Desnutricion Cero (PMD-C). Informe Gestion 2009.
• COMITÉ TECNICO DEL CONSEJO NACIONAL DE ALIMENTACION Y NUTRICION (CT CONAN)- MSD. (n.d.). Programa Multisectorial Desnutricion Cero (PMD-C). Informe Primer Semestre Gestion 2010.

Project: A033957002 – MI Micronutrient Component

Planning Documents

• ACDI-CIDA. (2012). Project Detail from CRAFT.

Reports

• ACDI-CIDA. (2010). Bolivia-Zero Hidden Hunger Annual Report 2009. CIDA Comments and Questions.

Project: A033957003 – UNICEF Joint Program to Support the Malnutrition Zero Program

PlanningDocuments

• (n.d.). Marco Logico Programa Sectorial Desnutricion Cero Gestion 2009.
• (n.d.). Marco Logico Programa Sectorial Desnutricion Cero Gestion 2010.
• (n.d.). Programacion Operativa Anual Gestion 2010 CT-CONAN.
• (n.d.). Year 2 WorkPlan.
• (2009). Reglamento Específico de Administracion del Fondo Canasta del Programa Desnutricion Cero.
• ACDI-CIDA. (n.d.). Annex A. Project Summary. Project Approval Document (PAD).
• ACDI-CIDA. (n.d.). Annex C. Backgrounder. Project Approval Document (PAD).
• ACDI-CIDA. (n.d.). Annex D. Risk and Mitigation Strategies. Project Approval Document (PAD).
• ACDI-CIDA. (n.d.). Annex E. Project Budget. Project Approval Document (PAD).
• ACDI-CIDA. (2012). Terms of Reference for the Evaluation of the Zero Malnutrition Program.
• MINISTERIO DE SALUD Y DEPORTES. (2009). Plan Estratégico 2007-2011 del Programa Sectorial de Desnutricion Cero.
• MINISTERIO DE SALUD Y DEPORTES. (2009). Plan Estratégico 2007-2011 del Programa Sectorial de Desnutricion Cero. Matriz.

Financial Reports

• ACDI-CIDA. (n.d.). CIDA – Zero Malnutrition Program in Bolivia. 2008 Financial Statement and 2009 Budget.
• ACDI-CIDA. (2010). CIDA – Zero Malnutrition Program in Bolivia. Financial Report January 2010.
• MINISTERIO DE SALUD Y DEPORTES. (2010). Ejecución del Presupuesto por Nivel de Asignación.

Reports

• COMITÉ TECNICO DEL CONSEJO NACIONAL DE ALIMENTACION Y NUTRICION (CT CONAN). (n.d.). Programa Multisectorial Desnutricion Cero. Informe Anual Gestión 2010.
• LAFORCE, J. (C.A.C. International). and SILVA, E. (2012). Evaluación del Programa Desnutrición Cero. Número de Proyecto ACDI: A033957. Informe Preliminar.
• MACLEAN, A. (Micronutrient Initiative). (2009). Zero Hidden Hunger Bolivia. Annual Narrative Report 2008.

Project: S063421001 – UQAM – Écodéveloppement communautaire

Planning Documents

• UQAM. (n.d.). Volet 1 du programme de PUCD. Concours 2005. Proposition détaillée.
• UQAM. (2007). Écodéveloppement communautaire et santé environnementale en Bolivie. Ecominga Amazónica. Bolivia 2007-2013. Plan de mise en oeuvre.

Reports

• BOISVERT, R. (2011). Évaluation mi-parcours du projet: Écodéveloppement communautaire et santé environnementale en Bolivie. Rapport final d’évaluation.
• UQAM. (2012). Rapport annuel 1er avril 2011-31 mars 2012.

Project: S061266PRG – AUCC-UPCD Program Tier 2

Planning Documents

• AUCC. (2012). Canadian University Projects in International Development Database (CUPID). Program details. [http://www.aucc.ca/programs-services/international-programs/cupid/?cue_action=details2&project_no_in=19/S61268-669/K].
• UNIVERSITY OF CALGARY. (n.d.). UPCD Tier 2 Full Proposal Submission: Water Management in Bolivia.

Financial Reports

• AUCC. (2012). AUCC Project Financial Report Calgary/Bolivia.
• AUCC. (2012). AUCC Project Financial Report Simon Fraser/Bolivia.

Reports

• BETHUNE, D., TORRES, J. and MULLIGAN, B. (2012). UPCD Tier 2 Project Number S61268-669 (2007-2013) Water Management in Bolivia. Annual Progress Report FY5: April 2011-March 2012.
• BROHMAN, J. et. al. (2012). Bolivian Specialization in CED. UPCD Tier 2: Annual Progress Report 2011-12.
• CHIBI, A. (n.d.). 669 2011-2012 Reports Analysis.
• CHIBI, A. (2012). UPCD Project 669 Calgary/Bolivia “Water Management in Bolivia”. AUCC Mission Report.
• UNIWORLD. (Winter 2012). Reaching out to the margins.
• UPCD. (2009). UPCD Tier 2 Survey.

Project: S062525PRG – Save the Children 2006-09

Planning Documents

Financial Reports

• ACDI-CIDA. (2011). Final Financial Report 2006-2011.

Reports

• BYRON, G. (2009). Realizing the Rights of the Socially Excluded Children Worldwide. Save the Children Canada – CIDA Canadian Partnership Branch Program 2006-2011. Mid-term Evaluation Report.
• COSSTICK, F. (ACDI-CIDA). (2008). Excerpt from Mission Report of November, 2008.
• SAVE THE CHILDREN CANADA. (n.d.). Realizing the Rights of the Socially Excluded Children Worldwide. Final Program Report 2006-2011. 

Project: S062900PRG – Plan International Canada 2005-10

Reports

• (n.d.). Project: Bolivia – Community Integrated Management of Childhood Illnesses Project. Results Tables from: August 18, 2005 to December 31st, 2008.
• (n.d.). Bolivia – Child Friendly Municipalities.
• (2007). Project Plan Canada. Bolivia Outputs and Outcomes for 10/12/2007.
• (2008). Midterm Evaluation Report of the Children’s Millenium Programme #S062900.
• (2008). Excerpt from Plan Canada year 3 report to CPB, September 2008.
• BERNARD, A. (2009). Children’s Millennium Programme Evaluation. Final Evaluation.
• DIAGNOSIS S.R.L. (2008). Midterm Evaluation of the Plan International Inc. Bolivia Program. “Child Friendly Communities”.
• FOSTER PARENTS PLAN.(2006). Year 1 Report (S-062900). Annual Report.
• PLAN CANADA. (2007). Year 2 Report (S-062900). Annual Report.
• PLAN CANADA. (2008). Year 3 Report (S-062900). Annual Report.

Economic Growth

Project: A030420001 – Bolivia Hydrocarbon Regulatory Assist.

Planning Documents

• ACDI-CIDA. (n.d.). Preliminary Environmental Assessment.
• ACDI-CIDA. (2002). Project Approval Document (PAD).
• ACDI-CIDA. (2002). Logical Framework Analysis.
• ACDI-CIDA. (2002). Annex to PAM – Women in Development (WID).
• ACDI-CIDA. (2012). Project Detail from CRAFT.
• AKHTAR, N. and DURRANI, G. (2010). Proyecto de Hidrocarburos Bolivia-Canadá. Plan de trabajo anual ministerio de hidrocarburos y energía. PPT.

Reports

• ACDI-CIDA. (n.d.). Bolivia Oil and Gas Project – Phase II. Results Achieved to-date.
• ACDI-CIDA. (2008). 2007/2008 Project/Program Performance Report.
• ACDI-CIDA. (2011). Bolivia Hydrocarbon Assistance Project. Summary of Results: September 2003-December 2010. PPT.
• IBM BUSINESS CONSULTING. Bolivia-Canada Hydrocarbon Regulatory Assistance Project. Quarterly Progress Report, October 1, 2006 – December 31, 2006. (2007).

Project: A021625001 – Bolivia Public Sector Reform Mining Comm

Planning Documents

• ACDI-CIDA. (2000). Project Approval Document (PAD).
• ACDI-CIDA. (2000). Preliminary Environmental Assessment.
• ACDI-CIDA. (2000). Annex to PAM – Women in Development (WID).
• ACDI-CIDA. (2000). Logical Framework Analysis.
• ACDI-CIDA. (2000). Concept Paper (CP).

Financial Reports

• MRNF. (2006). Projet REFORMIN. Réforme de l’industrie minérale et de l’environnement en Bolivie. Rapport annuel et financier pour l’année budgétaire 2005-2006 et plan de travail pour terminer le Projet en 2006.

Reports

• BÉLANGER, M. (MRNF). (2006). Projet REFORMIN. Réforme de l’industrie minérale et de l’environnement en Bolivie. Rapport final 2001-2006.

Project: S063681PRG – Programme Coop 2007-2012

Project: S061681PRG – Contribution du programme/Program Contribution

Project: S064520PRG – CESO 2009-2014 Volunteer Cooperation Prg

Project: S062447PRG – CESO Program Support 2004-2009

Project: S064494PRG – WUSC-CECI Program Support

Project: S062445PRG – Joint WUSC-CECI Program Support

Gender Equality

Project: A032298001 – Andean Region Gender Equality Fund

Planning Documents

• ACDI-CIDA. (n.d.). Propuesta estratégica para la tercera fase del FIG Bolivia.
• ACDI-CIDA. (2004). Project Approval Document (PAD).
• ACDI-CIDA. (2004). Logical Framework Analysis.
• ACDI-CIDA. (2004). Gender Equality Assessment.
• ACDI-CIDA. (2004). Canadian Environmental Assessment Act (CEAA) Applicability.
• ACDI-CIDA. (2012). Project Detail from CRAFT.

Financial Reports

• ACDI-CIDA. (n.d.). Fondo para equidad de género – ACDI FIG III. Presupuesto 2005-2009.

Reports

• ACDI-CIDA. (n.d.). Informe Anual. Fondo de Igualdad de Género Fase III, Bolivia. Información para el APPR FIG.
• ACDI-CIDA. (n.d.). Key Results in Gender.
• ACDI-CIDA. (n.d.). Información para el APPR FIG.
• ACDI-CIDA. (2005). 2004/2005 Project/Program Performance Report.
• ACDI-CIDA. (2006). 2005/2006 Investment Monitoring Report for Contribution Agreement.
• ACDI-CIDA. (2007). Informe Resumen Fondo de Equidad de Género FIG Bolivia.
• ACDI-CIDA. (2008). 2007/2008 Investment-level Monitoring Report: Contribution.
• ACDI-CIDA (n.d).   FIG III.  Key Results.  )
• ACDI-CIDA (n.d). Reporte de cierre del proyecto: “Estrategia de reposicionamiento de ACOBOL”.
• ACDI-CIDA (n.d). Reporte de cierre del proyecto: “Talleres de capacitación y formación en derechos de las mujeres” del Cumbre d Sajana,
• ACDI-CIDA (n.d). Reporte de cierre del proyecto: “Empoderamiento de mujeres con enfoque de género en tres distritos del municipio de Charazani”, del Centro Yanapasiñani.
• ARANIBAR, ANA MARIA(2008).  Mujeres de Oro.  ¿Quíenes Somos?  Cumbre del Sajana.
• BANDA BRAVA (2010).Sistematización Fondo de Igualdad de Género III.

Project: A032504001 – Gender Equality Basket Fund- Bolivia

Planning Documents

• ACDI-CIDA. (n.d.). Memorandum for the Minister.
• ACDI-CIDA. (n.d.). Informe proyecto de apoyo al plan estratégico institucional viceministerio de género y asuntos generacionales.
• ACDI-CIDA. (n.d.). Strategic Environmental Assessment.
• ACDI-CIDA. (n.d.). Canadian Environmental Assessment Act (CEAA) Applicability.
• ACDI-CIDA. (2004). Gender Equality Assessment.
• ACDI-CIDA. (2004). Institutional Support to the Office of the Vice-Minister of Gender, General Affairs and Family.
• ACDI-CIDA. (2005). Project Approval Document (PAD).
• ACDI-CIDA. (2012). Logical Framework Analysis.
• ACDI-CIDA. (2012). Project Detail from CRAFT.
• VICEMINISTERIO DE GENERO Y ASUNTOS GENERACIONALES DEL MINISTERIO DE JUSTICIA. (n.d.). Anexo 2, formulario 4, Articulación de la planificación de mediano plazo con el corto plazo. Matriz síntesis – programa de operaciones anual 2008.

Reports

• ACDI-CIDA. (2005). 2005/2006 Project/Program Performance Report.
• ACDI-CIDA. (2008). 2007/2008 Project/Program Performance Report.
• SALINAS, SILVIA.  (2010) Análisis de los cambios en la equidad de género en Bolivia 2004-09.

Environment

Emergency Assistance

Project: M012532001 – Bolivia Floods 2007/WFP EMOP 10616.0

Planning Documents

• ACDI-CIDA. (2007). Memorandum.

Reports

• WFP. (2008). Standard Project Report 2007.
• WFP. (2009). Standard Project Report 2009.

Project: M012803001 – Bolivia Floods WFP Flash Appeal 2008

Planning Documents

• ACDI-CIDA. (2008). Memorandum.

Reports

• WFP. (2008). Standard Project Report 2008.
• WFP. (2009). Standard Project Report 2009.

Annex 3: List of Persons Interviewed

CIDA - HQ and Bolivia

Name: Agnes Skornicz
Position: International Development Project Advisor (EGS)
Organization: CIDA

Name: Alberto Palacios-Hardy
Position: Former Head of Aid in La Paz
Organization: CIDA

Name: Amelie Pruneau
Position: A/Manager, Youth Participation (ETS)
Organization: CIDA

Name: Andrew Scyner
Position: Head of Aid, La Paz
Organization: CIDA – La Paz

Name: Anne-Marie Hodgson
Position: Program Policy Analyst
Organization: CIDA

Name: Chantal Boucher
Position: Former Project Officer – Bolivia, Peru
Organization: CIDA

Name: Diane Harper
Position: Director (SPCS)
Organization: CIDA

Name: Eileen Kilgour
Position:
Organization: CIDA

Name: Frances Cosstick
Position: International Development Project Advisor (HS)
Organization: CIDA

Name: Frantz Pierre-Jerome
Position: Evaluation Manager
Organization: CIDA

Name: Hélène Giroux
Position: Director General
Organization: CIDA

Name: Janet Ferreira
Position: International Development Project Advisor (ES)
Organization: CIDA

Name: Jennifer Thompson
Position: Former PWCB project officer for CLWR
Organization: CIDA

Name: Jim Sutherland
Position: Former BCP manager
Organization: CIDA

Name: John Lok
Position: First Secretary, Cooperation - Governance
Organization: CIDA – La Paz

Name: Jonathan Laine
Position: Program Manager, Bolivia Program
Organization: CIDA

Name: Lilly Nicholls
Position: Director
Organization: CIDA

Name: Lise Filiatrault
Position: Regional Director General (BMV)
Organization: CIDA

Name: Louise-Marie Thomassin
Position: International Development Projects Advisor
Organization: CIDA

Name: Luc St-Laurent
Position: First Secretary, Cooperation - Health
Organization: CIDA – La Paz

Name: Maria Javenia-Mackenzie
Position: International Development Project Advisor (PIES)
Organization: CIDA

Name: Marie-Eve Castonguay
Position: International Development Project Advisor (CGHRS)
Organization: CIDA

Name: Michelle Guertin
Position: Evaluation Team Leader
Organization: CIDA

Name: Minh Tien Nguyen
Position: International Development Project Advisor (ESS)
Organization: CIDA

Name: Moreno Padilla
Position: Senior Environment Specialist
Organization: CIDA

Name: Paul Ragusa
Position: Oil and Gas Specialist
Organization: CIDA

Name: Pierre J. Tremblay
Position: Evaluation manager
Organization: CIDA

Name: Rachel Bruneau
Position: Senior Development Officer
Organization: CIDA

Name: Rémi Turmel
Position: International Development Project Advisor (VIS)
Organization: CIDA

Name: Steve Jaltema
Position: Former BCP officer in La Paz
Organization: CIDA

Name: Darren Rogers
Position: former BCP officer
Organization: CIDA

Name: Susan Learoyd
Position: Program Policy Analyst
Organization: CIDA

Name: Vicky Laramee
Position: International Development Project Advisor (VIS)
Organization: CIDA

Unidad de Apoyo de Servicios de la Cooperacion Canadiense (UASCC)

Name: Carmen Estepa
Position: Health Specialist
Organization: UASCC

Name: Eliana Gallardo
Position: Gender Specialist
Organization: UASCC

Name: Gustavo Bracamonte
Position: Director
Organization: UASCC

Name: Mike Guerra
Position: Systems Administrator
Organization: UASCC

Name: Urzula Montes de Oca
Position: Governance Specialist
Organization: UASCC

Government of Bolivia representatives & Parastatal Organizations

Name: Ana María Aguilar
Position: Former responsible of the National Nutritional Council
Organization: Ministry of Health and Sports

Name: Eduardo Alarcón,
Position: Vice Minister, Energy & Production
Organization: Ministry of Hydrocarbons and Energy

Name: Enrique Torrico
Position: Chief of Technical Norms and Community Development
Organization: Ministerio de Agua y Saneamiento Básico

Name: Frank Molina
Position: Vice Minister, Energy Development
Organization: Ministry of Hydrocarbons and Energy

Name: Gustavo Zárate
Position: Director, Energy Development
Organization: Ministry of Hydrocarbons and Energy

Name: Guy Vargas
Position: Responsible of Planning Unit
Organization: Ministry of Health and Sports

Name: Jaime Villanueva Cardozo
Position: National Climate Change Program Coordinator
Organization: Ministry of Environment and Water

Name: Jorge Quiroga
Position: Gas Pricing Specialist
Organization: Ministry of Hydrocarbons and Energy

Name: Juan Pablo Cardozo.
Position: Vice Minister
Organization: Ministry of Environment and Water

Name: Leslie La Torre
Position: Coordinator of Fortalessa Project
Organization: Ministry of Health and Sports

Name: Hortensia Jimenez
Position: Vice Minister, Electricity & Alternate Energy
Organization: Minister of Hydrocarbons and Energy

Name: Maria Cristina Arellano
Position: Former Director General of Social and Environmental Management
Organization: Minister of Hydrocarbons and Energy

Name: Martin Maturano
Position: Vice Minister
Organization: Ministry of Health

Name: Mr. Jorge Rojas
Position: Technical Advisor APG Natural Resources
Organization: Minister of Natural Resources

Name: Mrs. Margot Ayala
Position: Director General Energy & Production),
Organization: Ministry of Hydrocarbons and Energy

Name: Mr. Nelson Bartolo
Position: Manager, APG Natural Resources
Organization: Ministry of Hydrocarbons and Energy

Name: Mrs. Norka Gonzales
Position: Manager Royalties, ex Intern)
Organization: Ministry of Hydrocarbons and Energy

Name: Isabel Chopitea
Position: Former Director General
Organization: Vice Ministry of Hydrocarbons and Energy

Name: Adriana Viera
Position: BHAP Consultant Industrial Engineer responsible for National Energy Balance
Organization: Working for Gov through BHAP project

Name: Jorge Leytón
Position: Economic Modeling Specialist Consultant
Organization: Working for Gov through BHAP project

Name: Xavier Eduardo Barriga
Position: Director General of Social and Environmental Management
Organization: Working for Gov through BHAP project

Name: Carlos Villegas
Position: President
Organization: Yacimientos Petrolíferos Fiscales Bolivianos (YPFB)

Name: Donald Osinga
Position: General Manager - Exploration
Organization: YPFB

Name: Erika Defillipis
Position: Former GSH Supervisor
Organization: YPFB

Name: Luis Carlos Sanchez
Organization: Director of Exploration & Promotion
Position: YPFB

Name: María Luisa Auza
Position: Director of Planning
Organization: YPFB

Name: Mr. Fernando Salazar
Position: VP Contracts Administration & “Fiscalization”
Organization: YPFB

Name: Mr. Javier Fernández
Position: General Manager Planning & Finance
Organization: YPFB

Name: Mrs. Leila Mokrani
Position: Advisor to YPFB Andina, Former Ministry Director
Organization: YPFB

Name: Zulema Espejo
Position: Director of International Affairs
Organization: YPFB

Name: Blanca Laguna
Position: International Relations Director
Organization: Ombudsman’s Office

Name: Rielma Mencias
Position: Former Ombudsperson
Organization: Ombudsman’s Office

Name: Waldo Albarracín
Position: Former Ombudsman
Organization: Ombudsman’s Office

Name: Armando Paredes
Position: Management Director
Organization: National Statistics Institute

Name: Willy Miranda
Position: Statistics Officer
Organization: National Statistics Institute

Name: Martha Oviedo
Position: Former Director
Organization: National Statistics Institute

Name: Silvia Terrazas
Position: Former Technical Officer
Organization: National Statistics Institute

Name: Antonio Costas
Position: Former President of CNE; now Director of SEGIP
Organization: National Electoral Court

Name: Salvador Romero
Position: Former President of CNE
Organization: National Electoral Court

Name: Pavel Perez
Position: Former CIDA Project Coordinator
Organization: Auditor General’s Office

Name: Jorge Leyton
Position: Judicial Director
Organization: Public Defender’s Office

Name: Alberto Condori
Position: Former CIDA Project Coordinator
Organization: Public Defender’s Office

Name: Irma Campos
Position: Former Director of gender unit
Organization: Vice-Ministry of Equal Opportunities

Donor Representatives

Name: Sergio Martin-Moreno
Position: Coordinador general
Organization: Spanish Agency for International Development Cooperation

Name: Adam Behrendt
Organization: Especialista desarrollo social
Organization: World Bank

Name: Mai Le
Position: Coordinator
Organization: COCAB

Name: Julio Pedroza
Position: Belgium Cooperation, Technical Advisor
Organization: Belgian Development Agency

Name: Alain Schmitz
Position: First Secretary
Organization: Embajada de Bélgica

Name: Lari-Henrik Worsof
Position: Head of Aid
Organization: Embajada Dinamarca

Name: Jerome Dubois-Nercent
Position: Head of Aid
Organization: Embajada Francia

Name: Alejandra Márquez
Position: Expert on Emancipation
Organization: Netherlands Embassy

Name: Anke van Dam
Position: Head of Cooperation
Organization: Netherlands Embassy

Name: Janet Trujillo
Position: Environment Expert
Organization: Netherlands Embassy

Name: Ann Stodberg
Position: Business Development
Organization: Embajada Suecia

Name: Isabel Ascarrunz
Position:
Position: Water & Sanitation and Environment Expert
Organization: Embajada Suecia

Name: Michel Thieren
Position: Representative
Organization: PAHO/WHO

Name: Christian Jette
Position: Democratic Governance Coordinator
Organization: UNDP

Name: Victor Hugo Bacarreza
Position: -
Organization: UNDP

Name: María de Angeles Loayza
Position: former CIDA Project Director
Organization: UNDP

Name: Rocio Chain
Position: Environment and Climate Change Programme Officer
Organization: UNDP

Name: Irma Peredo
Position: PACSAS Coordinator
Organization: UNICEF

Name: Marcolui Corsi
Position: Representative
Organization: UNICEF

Name: Rosario Quiroga
Position: Chief – Health Sector
Organization: UNICEF

Name: Isamaria Wallendcht
Position: Programme Officer
Organization: European Union

Name: Lawrence Odle
Position: -
Organization: USAID

Name: Wayne Nilsestem
Position: Director
Organization: USAID

Name: Stephanie Bellot
Position: -
Position: Organization de Planificacion y Desarrollo

Name: Navil Agrament
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Harley Rodríguez
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Jaime Garron
Position: -
Organization: Ministerio de Planificacion y Desarrollo

Name: Juan Carlos Soria
Position: La Paz Monitor
Organization: WFP

Name: Paolo Mattei
Position: Representative
Organization: WFP

Name: Sergio Torres
Position: Head of Programme
Organization: WFP

Bolivian Non-Government Organizations and Civil Society Organizations, Universities, Municipalities

Name: Maria Eugenia Rojas
Position: former Executive Director
Organization: Asociación de Concejalas de Bolivia

Name: Bertha Acarapi
Position: former Municipal Councillor in El Alto
Organization: Asociación de Concejalas de Bolivia

Name: Ana María Aranibar
Position: Director
Organization: Cumbre del Sajana

Name: Zonia Fabiani
Position: Director
Organization: Centro Yanapasiñani

Name: Dino Palacios
Position: former Director
Organization: Federación de Asociaciones Municipales de Bolivia

Name: Alicia Canaviri
Position: Executive Director from 2005 to 20010
Organization: CDIMA

Name: Teresa Condori
Position: Current Executive Director
Organization: CEDIMA

Name: Orlando Velásquez
Position: President – Río Pilcomayo Defence Council
Organization: CODERIP/CLWR

Name: Aquilino Villca
Position: Municipal Mayor
Organization: Curahuara de Carangas

Name: Alexia Escobar
Position: Coordinator
Organization: FCI Bolivia

Name: Fanny Zambrana
Position: Consultant for “Fundación para el Desarrollo”
Organization: FUNDES/CLWR

Name: Franklin Gonzales
Position: Consultant for “Fundación para el Desarrollo”
Organization: FUNDES/CLWR

Name: Andrés Condori
Position: Pichacani community promoter, Commercial Secretariat of the Producers Association of APROARTE PACHAMAMA
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ely Borda
Position: Chief financial Adminstrator (Jefatura Administrativa financiera)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Francisco Montalvo
Position: Community Promoter of Ura Rodeo
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Martha Puma
Position: Community Promoter of Chimpa rodeo
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Patricia Rivero
Position: Project officials (Jefatura de Planificación e Investigación)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ramiro Morales
Position: Project Technical Expert “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism).
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Ricardo Guzman
Position: Project Representative “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism).
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: Teodora Mamani
Position: Project Technical Expert “gestión de la Biodiversidad e Interculturalidad” (Management of Biodiversity and Interculturalism)
Organization: Instituto Politécnico Tomas Katari (IPTK)/CLWR

Name: José Cepeda
Position: Project Representative
Organization: PRODEC/CLWR

Name: Nicolás Barrios
Position: Facilitator “Proyecto de Desarrollo Campesino”
Organization: PRODEC/CLWR

Name: Olga Carlos
Position: Facilitator “Proyecto de Desarrollo Campesino”
Organization: PRODEC/CLWR

Name: Paulina Sánchez
Position: Facilitator
Organization: PRODEC/CLWR

Name: Aura Teresa Barba
Position: National Director of the Project ECOMINGA
Organization: Universidad Gabriel René Moreno

Name: Gabriela Flores
Position: Student of the Master Degree Module
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Julio Torres
Position: Coordinator of the AUCC Project from the Universidad Mayo Real y Pontificia de San Francisco Xavier
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Ronald Zapata
Position: Student of the Master Degree Module
Organization: Universidad Mayo Real y Pontificia de San Francisco Xavier

Name: Lidia Zenzano
Position: Special Project Coordinator
Organization: Universidad Nur

Name: Mr. Antonio Cabrera
Position: Former Associate Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Mr. Freddy Blackutt
Position: Associate Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Mr. Jorge Nunez
Position: Former Dean, Faculty Integral de Chaco
Organization: UAGRM

Name: Offman Blanco
Position: Dean Faculty Integral De Chaco
Organization: UAGRM

Name: -
Position: Presidenta
Organization: Movimiento de Economía Social y Comercio Justo de Bolivia

Canadian Non-Government Organizations and Implementing Agencies

Name: Lucie Laplante
Position: Former CIDA Project Coordinator
Organization: Quebec Bar

Name: Antoinette Chibi
Position: Agente de programa

Organization: AUCC

Name: Jen Avaz
Position: Manager UPCD

Organization: AUCC

Name: Angela Herbas
Position: Office Manager Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Bernarda Sarué.
Position: Community specilaist for environment and social monitors training programme of the Faculty Integral de Chaco
Organization: BHAP

Name: Jim Roy
Position: Canadian Advisor on Exploration Economic Model Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Nasser Akhtar
Position: Resident Project Coordinator - Bolivia Canada Hydrocarbon Project BCHP
Organization: BHAP

Name: Dr. Percy Garcia
Position: Canadian Advisor on Socio-Economic issues Bolivia Canada Hydrocarbon Project
Organization: BHAP

Name: Verónica Guzmán
Position: Gender Consultant
Organization: BHRAP

Name: Mary Pullen
Position: Project Manager
Organization: Canadian Crossroads

Name: Cecilia Requeña
Position: Former Bolivian Project Director
Organization: CESO

Name: Erik Meier
Position: Former Bolivian Project Director
Organization: CESO

Name: Fernando Gamon
Position: Project Manager
Organization: CLWR

Name: Limbert Paredes
Position: Business Manager Environmental Services Association of Engineers Potosí
Organization: CLWR

Name: Luisa Fernanda Velasco
Position: Program Manager
Organization: CUSO

Name: Mai Le
Position: Cooperante – COCAB
Organization: CUSO

Name: Francesco Gatta
Position: Nuevas iniciativas
Organization: CARE

Name: Aleja Apaza
Position: President of Traditional medicine
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Dr. Franco Rocha
Position: Hospital doctor
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Gragoria Calle
Position: Traditional midwife
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Jose Luis Castro
Position: Tamani
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Justina Condori
Position: (Mamathalla: autoridad originaria)
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Ruth Bolaños,
Position: Coordinador of the Project Un aguayo para un Parto sin Riesgo
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Sebastian Colque
Position: Tamani: Local authority
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Sofía Porco
Position: Health councillor
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Teodocia Chuquichambi
Position: (Sulka: Autoridad Originaria)
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Carlos Ramallo
Position: Bolivian representative
Organization: CESO

Name: Saley Eid
Position: Program manager
Organization: Child Fund

Name: Wendy McFarren
Position: Director
Organization: Child Fund

Name: Jhonny López
Position: Exectutive Director
Organization: CIES

Name: Roxana Rios
Position: Responsable educative
Organization: CIES

Name: Wilma Quinteros
Position: President
Organization: Movimiento de Economía Social y Comercio Justo de Bolivia

Name: Christian Tremblay
Position: Representative
Organization: Oxfam Quebec

Name: Sangita Patel
Position: Programme Manager
Organization: Plan Canada

Name: Daniel Rojas
Position: Asesor de programas salud y desarrollo infantil
Organization: Plan International

Name: Eda Quispe
Position: Health Coordinator of “Camina” Project
Organization: Plan International

Name: Gustavo Tapia
Position: Research and M/E Coordinator
Organization: Plan International

Name: Ignacio Canaviri
Position: Coordinator of “Camina” Project in Sica Sica Municipality
Organization: Plan International

Name: Ximena Ostria
Position: Gerente nacional de programas
Organization: Plan International

Name: Ana Amador
Position: Coordinadora salud
Organization: Samaritan´s Purse

Name: Alejandra Villafuerte
Position: Program Manager
Organization: Save the Children

Name: Carlos Villaroel
Position: Cochabamba Manager
Organization: Save the Children

Name: Luis Ramirez M
Position: Executive Director
Organization: Save the children

Name: Marlen Mondaca
Position:
Organization: Save the Children

Name: Natalie Folz
Position:
Organization: Save the Children

Name: Philippe Demers
Position: Project Coordinator
Organization: Socodevi

Name: Roberto Muñoz
Position: Technical Specialist and site manager for oregano project
Organization: Socodevi

Name: Sebastian Valdivieso
Position: Project Director
Organization: Socodevi

Name: Eduardo Alfaro
Position: National Bolivian Project Manager
Organization: Uniterra en Bolivia

Name: Rita Cano
Position: Program Officer
Organization: Uniterra en Bolivia

Name: Aura Teresa Barba
Position: Teacher
Organization: Universidad Autónoma « Gabriel René Moreno »

Name: Frida Villarreal
Position: Coordinator
Organization: Université du Québec à Montréal (UQAM)

Name: Lucie Sauvé
Position: Project Director
Organization: Université du Québec à Montréal (UQAM)

Name: Mr. David Bethune
Position: Canadian project Director
Organization: University of Calgary

Name: Katherine Murillo
Position: Program Manager, MPED Americas
Organization: Federation of Canadian Municipalities (FCM)

Beneficiaries

Name: Vicente Ojeda Velasquez
Position: President of the CHW
Organization: Asociación de Redes de Responsables Comunitarios de Salud

Name: Carla Reque
Position: Female Intern in 2011-2012 (now ministry employee)
Organization: BHAP

Name: Laura Quinteros
Position: Female Intern in 2012-2013
Organization: BHAP

Name: Carla Derpic
Position: Female Intern in 2011-2012 now ministry employee
Organization: BHAP

Name: Virginia Rendon:
Position: Intern in 2007-2008 became the Director at the Ministry, currently working in YPFB Planning Department.
Organization: BHAP

Name: 50 Students and teachers
Position: Participants and teachers enrolled in the environment-social monitoring programme Camrini
Organization: BHAP

Name: 15 Secondary school girls
Position: Students in fifth and sixth year and four current college students and former POPL, Participants in the Gender Equality Program for Schoolgirls to encourage career selection in Science and Technology
Organization: BHAP

Name: Bertha Gutiérrez
Position: Professor at Isaac Maldonado College participant in POPL
Organization: BHAP

Name: Erika Barrientos
Position: Second semester Petroleum engineer student and graduate of the POPL program
Organization: BHAP

Name: Giovanna Atila
Position: 8th semester university student in petroleum engineering and graduate of the POPL program
Organization: BHAP

Name: Jaime Leaño
Position: Professor at Augusto Villazón College and participant in the POPL program
Organization: BHAP

Name: Marlene Fernández
Position: Professor at Isaac Maldonado college and participant in the POPL program
Organization: BHAP

Name: Mary Elisa Justiniano
Position: Professor at Monseñor Salvatierra college and participant in the POPL
Organization: BHAP

Name: Selenne Cabrera
Position: Second semester university student studying petroleum engineer and graduate of the POPL program
Organization: BHAP

Name: Josefina Nina
Position: Mother with a recently born child
Organization: Centro de Salud intercultural Curahuara de Carangas

Name: Representative of Inca Pampa
Position: Women’s association of Ecological producers Inca Pampa “ADEMPROEINPA”
Organization: CLWR

Name: Representative of La Mendoza
Position: Producers from La Mendoza Women’s Association “AMUPROLAMEN”
Organization: CLWR

Name: Representative of Poco Poco
Position: Producers from the Valle de Poco Women’s Association “AMPROVPP”
Organization: CLWR

Name: Representative of Soroma
Position: Soromo Producers Women’s Association “AMUPRODEV”
Organization: CLWR

Name: Representative of Sotomayor
Position: Women’s Association of Vallas Sotomayor “ADEMVAS”
Organization: CLWR

Name: Representative of Tasa Pampa
Position: Association of Women Producers Tasa Pampa “AMPTP”
Organization: CLWR

Name: Representative of Uyuni
Position: Vallunas Uyuneñas Association “AVAU”
Organization: CLWR

Name: Representative of Viña Pampa
Position: Association of Women Producers and Handcraft producers Viña Pampa “APAVIP”
Organization: CLWR

Name: Representative of Viña Quemada
Position: Association of women entrepreneurs of Viña Quemada “AMEVIQ”
Organization: CLWR

Name: Andres Torres
Position: President of the Agrarian Union of Peasants for Aroma Province
Organization: Municipality of Sica Sica

Name: Arminda Barrios Quispe
Position: Vice-President of the CHW Association of Community Representatives of Health Networks
Organization: Municipality of Sica Sica

Name: Erika Tintaya
Position: Doctor of the Conani Health Centre
Organization: Municipality of Sica Sica

Name: Felix Calle
Position: Municipal Health Representative
Organization: Municipality of Sica Sica

Name: Franklin Flores
Position: President of the Municipal Council
Organization: Municipality of Sica Sica

Name: Hilaria Herrera
Position: Consejala – Health Commission
Organization: Municipality of Sica Sica

Name: Maria Elena Callejas
Position: Worker at the Conani Health Centre
Organization: Municipality of Sica Sica

Name: Vilma Colque Villanueva
Position: Vice-president of the Municipal Council- Tourism and Education Commission
Organization: Municipality of Sica Sica

Name: Walter Mayzo Alandia
Position: Mayor
Organization: Municipality of Sica Sica

Name: Zulma Velasquez
Position: Health Centre Conani
Organization: Municipality of Sica Sica

Name: Director
Position: CA de la Coop de Tomina
Organization: Socodevi

Name: Farmer 1 Raul
Position: Participant in the oregano project
Organization: Socodevi

Name: Farmer 2 Javier
Position: Participant in the oregano project
Organization: Socodevi

Name: Farmer 3 Juan Luis
Position: Participant in the oregano project
Organization: Socodevi

Annex 4: Evaluation Matrix

1. Relevance

1.1 Consideration of Bolivia’s development priorities (National Development Plan)

Question: To what extent are CIDA investment choices and related results relevant to Bolivia’s NDP with respect to priorities and development policies?

Indicator: Degree of compatibility of the BCP’s priorities and Bolivia’s priorities and development policies mentioned in the NDP and PRSP.

Question: To what extent are CIDA investment choices in Bolivia and related results relevant to the achievement of Millennium Development Goals (MDGs)?

Indicator:

• Degree of compatibility of CIDA’s investment choices with MDG planned achievements
• Type of adjustments brought to Canadian programming based on Bolivia’s degree of achievement of the MDGs

Question: To what extent is CIDA involved in the adequate sectors given Bolivia’s priorities?

Indicator: Degree of compatibility of the BCP’s priorities and Bolivia’s strategic orientations and priority sectors mentioned in the NDP

Question: To what extent were CIDA’s choices of investments strategic within the selected sectors?

Indicator: Degree of alignment between the NDP and CIDA’s investment choices

Question: To what extent were changes in the Bolivian context taken in account by adequate changes in the BCP?

Indicator: Type of adjustments brought to Canadian programming in light of the evolution of the context in Bolivia (socio-economic, political, environmental and technological)

Question: With the collaboration of other donors, to what extent has CIDA ensured the development of a framework to monitor results of the NDP, in order to ensure that its programmatic choices be strategic in view of the objectives and priorities, including poverty reduction?

Indicator:

• Existence of a framework to monitor results of the NDP
• Degree of alignment of CIDA’s programmatic choices with Bolivia’s objectives and priorities (including poverty reduction) derived from results from the NDP

Question: To what extent does CIDA participate in the monitoring of global development results on the basis of the NDP and does CIDA adopt programmatic decisions based on this analysis?

Indicator: Type of adjustments brought to Canadian programming in light of the strengths and limits with respect to the achievement of objectives and results of the NDP

Question: To what extent is CIDA perceived as adding value in the Bolivian development context? How?

Indicator:

• Level of satisfaction of relevant GoB representatives concerning CIDA programming from 2005 to 2010
• Perception of other donors on CIDA programming from 2005 to 2010

Data sources:

• CIDA’s BCP and CPDF with respect to policies and priorities
• Evaluation and monitoring reports on MDG achievement in Bolivia
• Monitoring reports on Bolivia’s NDP achievement with respect to results
• BCP investment profile in Bolivia
• Sectoral studies and analyses
• BCP’s staff
• Other donors’ staff
• Bolivian Government (VIPFE) staff
• GruS

Data collection methods:

• Document review
• Interviews
• Field visits

1.2 Strategic priorities, Canadian and CIDA policies

Question: Is the BCP aligned with Canadian priorities and policies for South America and Bolivia?

Indicator:

• Degree of compatibility of the BCP with CIDA’s policies and priorities in South America and Bolivia and with Canada’s international policy statements (countries, sectors, crosscutting priorities, etc.)
• Degree of compatibility of the program’s objectives

Question:

• To what extent does the BCP reflect CIDA’s strategic priorities (MDGs, aid effectiveness principles, etc.)?
• Has the BCP been the object of adjustments that reflect changes within CIDA?

Indicator:

• Degree of compatibility of the BCP with CIDA’s strategic priorities
• Type of adjustments brought to the BCP to adapt it to changes in the Agency

Question: Are the BCP approaches, initiatives and expected results in line with the comparative advantages (expertise and lessons learned from past experiences) of CIDA in Bolivia?

Indicator: Degree of compatibility of the BCP’s approaches, initiatives, and expected results with expertise and lessons learned from past experiences of CIDA in Bolivia

Data sources:

• CIDA’s BCP, CPDF and Bolivia Country Strategy
• Canada’s international policy statements
• CIDA’s policies and priorities in LAC, South America and Bolivia
• CIDA’s strategic priorities regarding MDGs, aid effectiveness principles
• Other strategic documents such as the Paris Declaration, or coming from the GruS
• Evaluations of the Bolivia Program commissioned by CIDA

Question: How does gender equality figure in the BCP? To what extent do the approaches, initiatives and results support the objectives of CIDA’s gender equality policy (1999)?

Indicator:

• Degree of identification and inclusion of gender equality objectives into the BCP
• Number and percentage of projects that specifically target women’s participation and empowerment
• Level of participation of men and women in projects
• Existence and level of use of indicators that measure impact on both men and women
• Coherence of gender objectives with nationally identified gender priorities (GoB and civil society)

Question: To what extent were CIDA investment choices and results relevant to environmental sustainability?

Indicator:

• Number, percentage (of total) and quality of projects targeting specifically environmental sustainability
• Capacity of participants to promote environmental protection and sustainable development
• Existence and level of use of indicators to measure environmental impact

Data sources:

• CIDA’s gender equality policy (1999) and CIDA’s gender equality and environment strategies
• Gender studies
• Project documentation
• BCP’s staff, gender specialist
• CIDA gender specialists
• Executing agencies’ staff
• Beneficiaries
• GoB Ministries

Data collection methods:

• Document review
• Interviews
• Field visits

2. Effectiveness (development results in terms of poverty reduction)

2.1 Planned versus actual results of the program

Question: Planned versus actual results of the program.  What were the results achieved versus those planned within the sample of selected projects?

Indicator:

• Variations between planned and achieved results
• Level of satisfaction with results of GoB and beneficiaries

Question: What were the main success factors or limiting factors that affected results achievement?

Indicator:

• Number and type of success factors that affected results achievement
• Number and type of limiting factors that affected results achievement

Question: What were the main key results achieved regarding the three strategic priorities of the BCP?

Indicator:

• Nature and extent of the results achieved with respect to improvement in health and water and sanitation
• Nature and extent of the results achieved with respect to improvement of good governance by supporting democratization, public sector reform, and the inclusive management of strategic sectors
• Level of access by the project beneficiaries to benefits and resources of development initiatives

Question: To what extent did CIDA develop a gender equality strategy and effectively promote gender equality results through its investments?

Indicator:

• Level of implementation of gender equality strategies for each project of the sample and at the sectoral level
• Type and number of adjustments brought to the gender equality indicators (social, political and economic participation)
• Quality of gender equality strategies and results
• Level of access by the project beneficiaries to benefits and resources of development initiatives
• Perceived changes in the lives of women

Question: To what extent did CIDA develop an environment strategy and effectively promote environmental sustainability through its investments?

Indicator:

• Level of implementation of environment strategies for each project of the sample and at the sectoral level
• Quality of strategies and results promoting environmental sustainability at the project and sectoral levels
• Quality of environment sustainability strategies and results

Question: Has the selection of delivery mechanisms contributed to achieve planned results in the targeted program areas:

• Democratic Governance: bilateral, partnership and multilateral
• Health: bilateral, partnership and multilateral
• Private sector development: bilateral, partnership and multilateral
• Quality of environment sustainability strategies and results

Indicator:

• Level of satisfaction of respondents on CIDA’s performance at the project and sector level
• Project performance by channel and adequacy of delivery mechanism

Data sources:

• Project documentation
• Monitoring and evaluation reports for sampled projects
• CIDA’s gender equality and environment strategies
• Strategic environmental assessment
• Program PMF
• Other donors’ evaluations
• Bolivia’ sectoral policies
• BCP’s staff, gender specialist
• GoB Ministries

Data collection methods:

• Document review
• Interviews
• Field observations
• Focus groups (Bolivian non-governmental organizations, civil society organizations, beneficiaries)
• Semi-structured interviews
• Focus groups
• Field visits

2.2 BCP contribution to assist in reducing poverty in Bolivia through institutional development

Question: Was the program equipped with sectoral strategies within strategic objectives with a view to ensuring sound articulation of its investments towards achievement of poverty reduction results?

Indicator:

• Existence of integrated sectoral strategies to orient CIDA’s interventions
• Level of compatibility of sectoral strategies with strategic objectives

Question: To what extent was the program strategy designed to address the needs of the poorest populations?

Indicator:

• Number of poverty reduction issues addressed in program strategy
• Quality of achieved results of the program strategy in poverty reduction
• Percentage of poorest populations benefiting from project results

Question: Are the program results in line with the objectives, results and indicators identified in the Bolivian PRSP and the NDP?

Indicator: Degree of compatibility of program strategy objectives with the objectives, results and indicators identified in the Bolivian PRSP and NDP

Question: What are the results achieved in terms of improvement of quality, equity and efficiency of the Bolivian health and water and sanitation sector, and what are the methods in place to measure these results?

Indicator:

• Evolution/improvement of the health sector indicators in terms of quality, equity and access by rural population
• Level of stakeholder participation in health and water and sanitation
• Perceived changes in the capacity of national, regional, local authorities to plan, manage, monitor and evaluate health and water and sanitation services
• Adequacy of monitoring tools to measure results achieved in health and water and sanitation

Question: What were the results achieved in terms of the democratization and public sector reform and what are the methods in place to measure these results?

Indicator:

• Evolution of governance indicators in Bolivia (degree of respect for democratic principles and the rule of law; evidence of accountability and transparency in the GoB; perceived changes in the capacity to plan, deliver, monitor and evaluate public policies and services; public participation in government policy development and program delivery)
• Adequacy of monitoring tools to measure results achieved in democratization and public sector reform

Question: What were the results achieved with respect to the inclusive management of strategic sectors such as mining and hydrocarbons, and what are the methods in place to measure these results?

Indicator:

• Evolution of indicators pertaining to the mining and the hydrocarbons sectors
• Existence of better policies, guidelines to sustain inclusiveness in the management of the strategic natural resources sector
• Degree of participation of Bolivian stakeholders in decision making

Data sources:

• Program and project annual reports, M&E reports, project approval documents (PADs)
• General Bolivia development studies
• GoB ministerial documents: policies, guidelines
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• Non-governmental organizations and civil society organizations representatives

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (Bolivian non-governmental organizations, civil society organizations, beneficiaries)

3. Sustainability (sustainability of results – degree of dependency on development assistance)

3.1 Time required to generate sustainable results

Question: Is the time allocated for project delivery and achievement of results adequate and sufficient to ensure sustainability of results?

Indicator:

• Capacity of executing agencies to deliver outputs and achieve results in time
• Level of adjustment to country and/or regions’ political and economic context
• Perceptions of executing agencies with regards to timeframe for product delivery

Question: To what extent have risks associated with sustainability of results been adequately identified, analysed and managed?

Indicator:

• Quality of sustainability strategies of the sampled projects
• Number of sustainability risks identified in program design
• Number of sustainability risk reduction strategies identified and implemented during program implementation

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews

3.2 Absorptive capacity and appropriation by Bolivian partners

Question: What is the degree of dependency of the GoB vis-à-vis donor contributions?

Indicator:

• Percentage of GDP covered by donor’s contribution
• Difference in economic indicators between targeted areas of interventions and national average

Question: Which partners (public sector—state and decentralized—, civil society and private sector) have performed in such a way as to contribute optimally to sustainability of results?

Indicator:

• Number of local partners having demonstrated that their projects have developed sustainable results
• Percentage of budget allocated to administrative tasks versus operative tasks
• Level of involvement and contribution of government institutions, local universities, schools, etc. in sampled projects

Question: To what extent is there local ownership to sustain results achieved?

Indicator:

• Total number of trained persons in project management during program implementation
• Number of local consultations that have been carried out during the implementation of the program
• Perceived changes in local ownership (examples)

Question: Generally speaking, have the partners planned enough financial resources to cover recurring costs?

Indicator:

• Correlation between planned and actual partner expenditures in sampled projects
• Type and quality of partner financial management system and tools
• Quality and timeliness of partner financial information
• Relevance and timeliness of partner financial allocation between projects considering needs and demands

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Bolivian partners
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (Bolivian non-governmental organizations, civil society organizations, beneficiaries)

4. Crosscutting issues (integration of priority themes: gender equality and environment)

4.1 Integration of crosscutting themes in the BCP

Question: To what extent has the BCP and the CDPF strategically and operationally integrated crosscutting themes?

Indicator:

• Existence of a policy framework for CCI
• Incorporation of relevant analyses of CCI into planning process
• Existence of program-level requirements for project approval regarding CCI
• Percentage of sampled projects that respect program-level requirements and that integrate CCI in operational activities
• Existence of a Fund or dedicated resources

Question: Has the program planned for sufficient resources to ensure operationalization and monitor integration of crosscutting themes?

Indicator:

• Existence of crosscutting strategy monitoring processes
• Percentage of total program budget allocated to CCI
• Existence of specific resources to support and monitor CCI

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives
• Gender specialist

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (Bolivian non-governmental organizations, civil society organizations, beneficiaries)

4.2 Planning, monitoring and evaluation processes of crosscutting themes within sample of selected projects

Question: Were concrete strategies to integrate CCI developed at the project level and did the narrative project reports systematically account for results?

Indicator:

• Existence of project-level strategies to integrate CCI during implementation
• Presence of CCI and analysis in project level narrative reports

Question: What results did the strategies to integrate CCI in project implementation strategies generate, by sector and delivery mechanism?

Indicator:

• Number of specific crosscutting evaluations realised at the project level
• Existence and quality of strategies to integrate CCI in project implementation
• Magnitude of results generated by such strategies

Question: To what extent has the capacity of the GoB and civil society to take into account CCI been enhanced?

Indicator: Level of awareness of CCI among representatives from the GoB and civil society

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (Bolivian non-governmental organizations, and civil society organizations, beneficiaries)

5. Coherence

5.1 Internal coherence of the program (content and duration of intervention)

Question: To what extent does the BCP constitute the result of an adequate coordination with other international initiatives in Bolivia?

Indicator:

• Level of compatibility of the BCP with other donors’ programming
• Level of compatibility of donor interventions and CIDA with Bolivian policy on technical cooperation
• Number of projects from the sample that overlap with other international initiatives
• Degree of integration with joint assistance strategies
• Type of mechanisms existing to facilitate coordination among donors

Question: What are the concrete means and measures adopted by CIDA to ensure coherence and optimization of synergies between the sectors/strategic orientations of the BCP?

Indicator:

• Number of projects from the sample that overlap with other sampled projects considering sectors/strategic orientations
• Level of compatibility of the BCP sampled projects considering sectors/strategic orientations
• Number and frequency of meetings between representatives of different branches of CIDA to coordinate programming among sectors

Question: What are the concrete means and measures adopted by CIDA to ensure coherence and optimization of synergies between the projects within each program area?

Indicator:

• Number of projects from the sample overlapping with other sampled projects considering programming areas
• Level of compatibility of the BCP sampled projects considering programming areas
• Number of meetings between representatives of different branches of CIDA to coordinate programming among programming areas

Question: What are the concrete means and measures adopted by CIDA to ensure complementarities and synergies between the different funding mechanisms—bilateral (directive, responsive, PBAs), multilateral and partnerships?

Indicator:

• Number of projects from the sample that overlap with other sampled projects considering delivery model
• Level of compatibility of the BCP sampled projects considering the delivery model
• Number of meetings between representatives of different branches of CIDA to coordinate programming among delivery model
• Type of adjustments brought to BCP sampled projects that derive from lessons learned from problematic projects in the past

Question: Does the distribution of interventions and investments (i.e. distribution of investments by strategic objective) adequately reflect, in a balanced manner, planned results for both strategic objectives of the BCP?

Indicator:

• Congruence between resource allocations considering the three major strategic objectives of the BCP
• Congruence between interventions and investments of the BCP and the planned results for each strategic investments

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• Sectoral studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives
• Other donor representatives/ GruS

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (with donors and other stakeholders)

5.2 Program coherence with Canada objectives in Bolivia

Question: Does the program implement actions (e.g. policy dialogue, integrated approach framework) to support synergy between different Canadian actors in Bolivia?

Indicator:

• Number of interventions and initiatives to achieve concertation and coordination in the Canadian actors’ programming in Bolivia
• Number, type and quality of communications with other Canadian actors in Bolivia
• Type of consultation mechanisms developed to ensure synergy among Canadian actors

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits
• Focus groups (Bolivian non-governmental organizations, civil society organizations, beneficiaries)

5.3 Policy dialogue to support programming

Question: Is the policy dialogue at CIDA sufficient to efficiently promote Canadian priorities (human rights, democracy, transparency, gender equality, environment, civil society participation, etc.)?

Indicator:

• Evidence that Canadian priorities have been integrated in policies and BCP in Bolivia
• Percentage of Bolivian population participating in civil society
• Number and type of formal mechanisms implemented in the context of the BCP to promote civil society participation in political decision making

Question: Does the CIDA policy dialogue with and influence on the GoB create an enabling political context for the development of local networks and partnerships and the strengthening of national and/or local institutions in the BCP targeted sectors?

Indicator: Number, type and quality of CIDA communications with the GoB to create an enabling political context for the development of local networks and partnerships and the strengthening of national and/or local institutions

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits

6. Efficiency (management efficiency and capacity to monitor, evaluate and draw lessons learned from programming)

6.1 Management and optimization of the cost-effectiveness ratio

Question: Has there been any delay or gap between the period when a project was approved and the period of its full implementation?

Indicator:

• Number and types of delays encountered
• Perceptions of executing agencies with regards to timeframe for project implementation

Question: To what extent does the communication framework between CIDA headquarters and the field allow more punctual decision making to ensure the production of results and respect program calendar?

Indicator:

• Number, type and quality of field reporting to headquarters
• Number of delays in regards to program calendars
• Level of compatibility of decision making with local political and economic contexts

Question: What effect has the shift to program approach and budgetary support had on efficiency of the BCP delivery?

Indicator:

• Change in BCP delivery costs
• Level of satisfaction of respondents with shift to program approach and budgetary support

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• CIDA headquarters representatives

Data collection methods:

• Document review
• Interviews
• Field visits

6.2 Efficiency of resource allocation and utilization

Question: To what extent did the BCP have access to appropriate human and technical resources to support management in the field and at headquarters?

Indicator:

• Number, quality and adequacy of BCP human/technical resources by sector, strategic objectives
• Level of satisfaction of CIDA field and headquarters representatives with human/technical resources
• CIDA and sampled projects staff turnover rate

Question: To what extent did the projects have access to appropriate financial resources to achieve results?

Indicator: Correlation between financial resources disbursed and achievement of results

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits

6.3. Cost-effectiveness ratio of project results (management expenses versus operational expenses)

Question: To what extent is the cost-effectiveness ratio between management expenses and operational expenses reasonable and comparable to other donor programs in Bolivia?

Indicator: Adequacy of management expenses versus operational expenses

Question: To what extent has the shift to program approach and budgetary support improved the efficiency of project implementation in terms of resources and time allocated to management?

Indicator:

• Number of projects submitted to CIDA
• Number of projects approved
• Timeliness of project approval considering local political and economic contexts
• Percentage of CIDA staff time allocated to management of projects, by delivery model and sector

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits

7. Management principles (aid effectiveness)

7.1 Ownership/buy-in

Question: To what extent is the BCP the result of local demand?

Indicator:

• Number of responsive sampled projects
• Number of directive sampled projects
• Level of satisfaction of the local stakeholders concerning adequacy of results with local political and economic contexts
• Existence, type and range of preliminary research on national needs and demand from beneficiaries
• Type and range of beneficiary needs and demand
• Level of participation of local stakeholders in identifying needs and demands
• Existence and quality of documentation justifying interventions
• Level of satisfaction of beneficiary country officials with the program design

Question: To what extent have the projects been conceived to transfer execution leadership to the local partners and to maximize local ownership?

Indicator:

• Degree of involvement of local stakeholders in the design, management and monitoring of the sampled projects during the program
• Results achieved in terms of capacity building and overall improvements and performance within beneficiary institutions

Data sources:

• Program and project annual reports, M&E reports, PADs
• General Bolivia development studies
• GoB ministerial documents
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Bolivian partners
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits

7.2 Alignment/harmonization

Question: To what extent do the local institutions control the strategies, management and execution of the BCP?

Indicator:

• Number of local stakeholders and informants involved in conceptualization phases of the program and building of the contents
• Number of Bolivian policies and procedures integrated in the sampled projects
• Level of participation of local stakeholders in implementation of the BCP

Question: To what extent have the Bolivian financial institutions been strengthened at the national and local levels?

Indicator:

• Number of representatives trained (by involved financial institutions and political level)
• Capacity of financial institutions to manage financial flows from international aid

Question: To what extent have the capacity building initiatives for the GoB been coordinated and harmonized with those of other donors in Bolivia?

Indicator: Number of CIDA BCP capacity building initiatives coordinated and harmonized with those of other donors in Bolivia

Data sources:

• Program and project annual reports, M&E reports, PADs
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Executing agencies’ staff
• Project managers
• Bolivian partners

Data collection methods:

• Document review
• Interviews
• Field visits

Question: To what extent has the CIDA BCP been implemented through concerted mechanisms with other donors in Bolivia?

Indicator:

• Percentage of total CIDA BCP budget allocated to multilateral projects
• Number and quality of concerted mechanisms used to implement these types of initiatives
• Percentage of total budget of concerted initiatives financed by CIDA
• Adequacy of CIDA efforts to disseminate and share among other donors acquired knowledge, lessons learned and best practices from Canadian experiences

Data sources:

• Program and project annual reports, M&E reports, PADs
• BCP’s staff
• GoB staff (Ministry of Foreign Affairs, MPD)
• Donor representatives
• Beneficiaries
• Executing agencies’ staff
• Project managers
• non-governmental organizations and civil society organization representatives

Data collection methods:

• Document review
• Interviews
• Field visits

8. Performance management (management results including monitoring and evaluation)

8.1 RBM as per CIDA guidelines

Question: To what extent were Canadian cooperation interventions in Bolivia based on past acquired knowledge and lessons learned and to what extent did CIDA try and acquire new knowledge and lessons learned through the BCP?

Indicator:

• Degree of integration of past lessons learned in designing and implementing the BCP
• Evidence that past evaluations have been used during the CIDA BCP design and implementation phases
• Level of adequacy of knowledge transfer and learning mechanisms
• Number and quality of M&E reports produced during BCP funding period

Question: To what extent have risks associated with Canadian cooperation interventions in Bolivia been adequate?

Indicator:

• Existence of a risk management framework
• Number and quality of risk reduction initiatives designed and used during BCP funding period
• Type of risks identified and corresponding mitigation strategies adopted to reduced potential risks during BCP implementation
• Quality of decisions taken during implementation of the BCP that were based on ongoing risk analysis and management

Data sources:

• Program and project annual reports, M&E reports, PADs
• BCP’s staff
• Executing agencies’ staff

Data collection methods:

• Document review
• Interviews
• Field visits

8.2 M&E mechanisms in place to ensure that a RBM approach was used during CIDA BCP implementation

Question: To what extent was the CIDA BCP PMF used to monitor BCP results?

Indicator:

• Existence and quality of the PMF
• Level of compatibility of the PMF and indicators with Bolivian implementation context and CIDA capabilities in the field
• Regularity of reporting
• Relevance of reporting

Question: Were the CIDA BCP M&E mechanisms adequate at the project and program levels and were the results of these mechanisms used in decision making?

Indicator:

• Quality of reporting and evaluation produced through the M&E mechanisms
• Evidence that the M&E mechanisms have influenced decision making and strategic orientation of the BCP

Question: To what extent has CIDA participated in joint monitoring/evaluation initiatives and respected the harmonization principle in its performance management in Bolivia?

Indicator:

• Number of joint M&E initiatives to which CIDA has participated
• Number of joint M&E missions to which CIDA has participated
• Adequacy of CIDA’s participation in joint M&E initiatives
• Level of satisfaction of partners in joint M&E initiatives regarding CIDA’s participation

Question: To what extent has CIDA participated in initiatives aimed at building local M&E capacity in Bolivia?

Indicator: Number of M&E initiatives aimed at building local M&E capacity in Bolivia sponsored by CIDA.

Data sources:

• Program and project annual reports, M&E reports, PADs
• BCP’s staff
• Executing agencies’ staff
• Other donor representatives

Data collection methods:

• Document review
• Interviews
• Field visits

Annex 5: Program Assessment/Project Assessments

Summary of Scores for Sectors/Crosscutting Issues, by Project

Annex 6: Management Response

Introduction

Overall, the Bolivia Country Program Evaluation (2005-2010) is a useful assessment of Canada’s development Program in Bolivia. It relies on information from project and progress reports as well as from extensive interviews with key informants, who were involved in decision-making and implementation of the Program. Importantly, the report assesses Program effectiveness during a period of significant change – i.e. the emergence of a new critical political and social environment in Bolivia and the redefinition of development priorities in Canada.

The report positively assesses (a) the understanding, leadership and participation of key stakeholders (national and international) involved in the processes that governed Program planning, implementation and monitoring, and (b) how well the objectives of the Program represented the interest, needs and aspirations of the communities and institutions involved. The report concludes that most of the observed projects and interventions were relevant to Canadian and Bolivian strategic priorities, and produced positive outcomes, but there is still a need to improve long-term institutional capacity to achieve sustainable investments (e.g. health and governance interventions).

With respect to the Program’s strategic capacity building approach, the evaluators’ understanding of the limited capacity of the Bolivian public service in retaining expertise and knowledge during a period (2006-10), is appreciated. The evaluators valued the efforts made by the Program and its partners, to increase the institutional and technical capacity of local authorities and institutions (e.g. municipalities) in support of the decentralization process led by the national government. The evaluators also recognized that the Program played a leadership role in promoting policy dialogue on gender equality and contributed to important gender equality results through its programming, such as increasing political participation by women, strengthening capacity of women’s organizations and improving protection of women’s rights.

Due to program budget reductions, the Bolivia Program annual budget is declining from $18M in 2011-2012 to $9M by 2014-2015. This reduced bilateral annual budget is roughly equivalent now to the annual Canadian civil society programming in Bolivia supported by the Partnerships for Development Innovation Branch (PDIB).

Given the importance of PDIB in the country, and since the creation in 2011 of the Coordination Group for Canadian Organizations in Bolivia (COCAB), the Program has intensified harmonization and coordination between its Bilateral and Partnership streams. This effort has enriched the Program’s assessment of development priorities in the country and its understanding of the vital role played by local governments (municipalities) in rural areas. It has also resulted in new partnerships between Bilateral and Partnership initiatives that serve to strengthen Canada’s aid effectiveness in Bolivia. One Canada-based staff has been designated to promote strategic opportunities between the Bilateral Program and PDIB.

In light of the budget situation, no new bilateral projects have been considered for approval in Bolivia since 2011, and there is limited room for additional programming until at least 2015.

Furthermore, Canada-based staff at the Mission in La Paz has been reduced from three to two in July 2014, and the local PSU contract will expire on January 31, 2015.

Notwithstanding these current programming limitations, the Bolivia Program has taken measures to respond to the recommendations within existing projects and any new ones.

Recommendations: 1. Building ownership and alignment of CIDA programming through engagement and consultation with partners has been a strength of the program during the review period. It is recommended that the program make a renewed commitment to this approach in its future program planning efforts.

Commitments/Measures: The Program agrees with this recommendation. Since 2011, the Program has consistently built ownership of Program activities by the Government of Bolivia, as well as ensured alignment of Canada's bilateral Program with Government of Bolivia priorities. For example, for all of its bilateral projects, the Program has invited a relevant Government of Bolivia representative to participate as member of a Project Steering Committee (or similar committee).

1.1 Consultations with the Government of Bolivia on any new strategic programming orientations or project proposals will be held within the framework of the updated country strategy.

1.2 The Program will maximize coordination (e.g. joint monitoring missions).

1.3 Provided a focus on Children & Youth is maintained, the Program will accept the recent offer of the donor community for Canada to lead the Donor’s Health Coordination Group.

1.4 The Program will promote further coordination and harmonization of Canada’s aid footprint in Bolivia, as well as with other donor’s activities, particularly in the Department of Chuquisaca where Canada has significant bilateral and partnership initiatives.

Responsible: PTLs & HOA

Completion Date:

1.1 Completed as of January 2015.

1.2 Completed as of December 2014.

1.3 Completed as of June 2014.

1.4 Completed as of June 2014.

Recommendations: 2. In light of limited institutional capacity in most sectors in Bolivia, effectiveness and sustainability will require concerted attention.  It is recommended that future engagement include a focus on capacity development, taking into consideration how Canadian value added and knowledge can best be brought to bear in the development of explicit plans for major institutional partners.

Commitments/Measures: The Program agrees with the recommendation. Since 2011, the Program has focused on sustainable and effective capacity development of partners. For example, in both the health and economic growth sectors, two key projects (SOCODEVI Economic Growth and PLAN CANADA Public Health Caminas) are forming alliances with sub-national governments to maximize sustainability.

2.1 The Program will ensure that capacity and sustainability issues and proposed measures to address such issues are identified in relevant documents for each new project and in annual country reporting (Project Management Summary Report) for all projects.

Responsible: PTLs & HOA

Completion Date: 2.1 Completed as of May 2014.

Recommendations: 3. The program should build on progress in strengthening integration of gender as a crosscutting theme by: ensuring that all projects include a gender analysis, strategy, indicators and targeted resources; developing a standard gender monitoring and reporting system to assess gender integration across all projects and sectors; and sharing knowledge with partners and stakeholders about gender challenges, opportunities and lessons.

Commitments/Measures: The Program agrees with the recommendation, which will necessitate appropriate resources being made available to follow through on the commitments. In 2011-12, training was provided to all CBS and local consultants in gender equality and program planning and monitoring. Furthermore, gender equality training was coordinated with the Canadian Civil Society Coordinating Group in Bolivia (COCAB) members in 2013.

3.1 The Program will assure that Program Staff that have not taken HQ Gender Equality Training in the last five years, participate in the refreshing training programs.

3.2 The Program will ensure that all project officers (local and Canadians) coordinate planning and monitoring activities with the local Gender Equality specialist. At least one monitoring mission per project per year will include gender equality reporting. The Program will also ensure that gender equality results are included in its annual country reporting.

3.3 The Program will plan a systematic exchange of best practices on gender equality and on women’s empowerment within the country and between country programs, with involvement from the local and HQ gender equality specialists as well as relevant project officers.

3.4 The Program will actively join the Regional Community of Practice on Gender Equality that is coordinated by the Peru Program.

Responsible: PTLs & HOA

Completion Date:

3.1 Completed as of January 2015

3.2 Completed as of May 2014

3.3 Third quarter 2014

3.4 Completed as of January 2015.

Recommendations: 4. The program should improve mainstreaming of environmental sustainability by developing an environment implementation plan, and working with like-minded donors to integrate environmental considerations into pooled fund objectives.  Depending on the nature of future investments, the program should make use of positive lessons gained from past SEG programming, including building partner capacity to implement environmental standards.

Commitments/Measures: The Program agrees with the recommendation. Historically, the Program has ensured that all SEA and CEAA requirements have been met at the Program and project level.

Some experienced Canadian NGOs active in Bolivia (e.g. SOCODEVI, CECI, PLAN, OXFAM and CUSO) have been instrumental in developing ways of living more sustainably in rural and semi-urban communities that have taken many forms from (a) reorganising living conditions, (b) reappraising economic sectors, or work practices, (c) using Canadian experience to develop new technologies, to (d) adjusting individual lifestyles that conserve natural resources. The dissemination of these practices and lessons learned has already benefitted bilateral programming and implementation.

4.1. The Program will ensure effective coordination with experienced donors (e.g. GTZ, Swiss, and the Belgians), who have proven to have considerable experience and technical expertise in the area of sustainable environment.

4.2. The Program will develop a plan with current project executing agencies with solid environmental expertise to ensure that knowledge and best practices are transferred to local partners (e.g. SOCODEVI, Plan International Canada, and Colleges and Institutes Canada - CICAN).

4.3 The Program will actively join the Regional Community of Practice on Environmental Sustainability that is coordinated by the Peru Program.

Responsible: PTLs, HOA & Specialist

Completion Date:

4.1 Completed as of August 2014

4.2 Completed as of June 2014

4.3 Completed as of March 2014

Recommendations: 5. While a program level performance management framework has been articulated, the program should ensure that it specifies clear expected results with verifiable indicators, and to the extent possible collect adequate baseline information to facilitate eventual evaluations.

Commitments/Measures: The Program agrees with the recommendation.

5.1 The Program will update its Program level Logic Model and Performance Management Framework, in the context of the updating of DFATD development country strategies. It will ensure that expected results are clear that verifiable indicators are identified and baseline data is collected.

Responsible: PTLs & HOA

Completion Date: 5.1 Completed as of September 2014

Recommendations: 6. There is scope for greater visibility of program accomplishments with partners and stakeholders.  The program should develop an appropriate communications strategy to raise its profile.

Commitments/Measures: The Program agrees with the recommendation:

6.1 The program will develop a communication plan, in coordination with the Embassy in Lima and in agreement with DFATD strategy of visibility and recognition.

6.2 The Program in coordination with COCAB will circulate a newly published brochure that clearly describes the ample coverage of development activities of Canadian civil society and its volunteers in Bolivia to its partners and the various stakeholders of the Program in Bolivia. This brochure will be translated in both English and French to be distributed in Canada.

6.3 The Program, in coordination with the Embassy in Lima, will summarize best practices and experiences in Bolivia to be incorporated in the Mission Webpage every four months.

Responsible: Director & HOA

Completion Date:

6.1 Completed as of July 2014

6.2 Completed as of January 2014

6.3 March 2015