Foreign Affairs, Trade and Development Canada
Symbol of the Government of Canada

Foreign Affairs, Trade and Development Canada

international.gc.ca

Formative Evaluation of GPP Funding to Support the Construction of the CWDF at Kizner, Russia

September 2009

(PDF Version, 477 KB) *

Table of Contents

Abbreviations, Acronyms and Symbols

ARAF
Accountability, Risk and Audit Framework
CWC
Chemical Weapons Convention
CWD
Chemical Weapons Destruction
CWDF
Chemical Weapons Destruction Facility
DFAIT
Department of Foreign Affairs and International Trade Canada
DPL
Destruction Process Line
FSU
Former Soviet Union
GP
Global Partnership
GPP
Global Partnership Program
IA
Implementation Agreement
IGX
Global Partnership Bureau at DFAIT (Ottawa and Moscow)
MPF
Metal Parts Furnace
Minpromtorg
Ministry of Industry and Trade of the Russian Federation
NTI
Nuclear Threat Initiative
OPCW
Organization for the Prohibition of Chemical Weapons
PAA
Program Activity Architecture
PMI
Project Management Institute
PMBOK
Project Management Body-of-Knowledge
Pribor
Federal State Unitary Enterprise--Federal Scientific and Production Centre "Pribor"
SCI
Surface Combustion Inc.
Stanko Agregat
Joint Stock Company Stanko Agregat
TBE
Teledyne Brown Engineering
TBS
Treasury Board Secretariat
UK MoD
Ministry of Defence of the United Kingdom of Great Britain
US
United States of America

Top of page

Executive Summary

This report presents the Formative Evaluation of the Global Partnership Program (GPP) Funding to Support Construction of the Kizner Chemical Weapons Destruction Facility (CWDF).

Project Background and Context

The Global Partnership Program (GPP) of the Department of Foreign Affairs and International Trade Canada (DFAIT) is providing support for the construction of a CWDF at Kizner, Udmurt Republic of the Russian Federation. Canada's contribution of up to $100 million dollars is designated for the provision of equipment required for the chemical weapons destruction process in the two main nerve agent destruction buildings of the facility.

Canada's support for construction of the Kizner CWDF is part of the commitment made under the Global Partnership against the Spread of Weapons and Materials of Mass Destruction (the Global Partnership) to assist Russia in meeting its obligations under the Chemical Weapons Convention (CWC).

Canada has already contributed to Russia's Shchuch'ye CWDF. The Shchuch'ye site is one of seven locations that hold Russia's declared stockpile of chemical weapons. As of June 2009, five of the seven Russian CWDFs have completed, or are in the process of completing the destruction of their CWD stock. The two remaining facilities - Pochep and Kizner - are at the stage of building the destruction facilities.

Project Delivery Structure

The original plan for the delivery of Canada's support to Kizner was to use the Ministry of Defense of the United Kingdom (UK MoD) as overall project coordinator for donor activities, similarly to the Shchuch'ye Project. In 2008, however, the UK decided to withdraw from the Kizner project. A revised project delivery mechanism was required and IGX took on the lead role as project coordinator.

The revised delivery structure is based on the 2004 Canada-Russia Bilateral Agreement and the established framework for cooperation on GPP projects. It also, includes: an Implementation Agreement (IA) between the Ministry of Industry and Trade of the Russian Federation (Minpromtorg) and DFAIT; two trilateral contribution agreements between Canada, Minpromtorg and the respective Russian equipment supplier; a contribution agreement between DFAIT and an American equipment supplier; a Memorandum of Understanding (MOU) between DFAIT and the UK MoD for the provision of project support and engineering advice; and a contract between DFAIT and an American engineering company for monitoring and technical support services.

Evaluation Objectives, Issues and Approach

This formative evaluation was a special requirement of the Treasury Board Secretariat for the release of a $4 million frozen allotment towards the Kizner project. The evaluation followed the main evaluation criteria of relevance and performance and was conducted in the period April 1- July 30, 2009. The data collection approach included: file, document and database review; interviews with DFAIT and IGX officers in Ottawa; and a mission to Moscow and London to interview project partners and stakeholders.

Key findings

The key findings of the study are as follows:

Relevance

  • Canada's contribution to the Kizner facility and the Russian CWD Program continues to directly support the goals and priorities of DFAIT and the Government of Canada, as detailed in the Department's Program Activity Architecture (PAA) and the 2009-2010 Report on Plans and Priorities (RPP).
  • Canada's support for the Kizner CWDF addresses an important and time-sensitive priority for Russia. The Kizner CWDF is a critical component of Russia's CWD program and Russia's ability to meet the 2012 CWC deadline for complete destruction of all chemical weapons.
  • Eventual cancellation of Canada's contribution would have significant and negative impacts not only for Russia's commitments to CWC, but also for the non-proliferation efforts of the international community.
  • Canada is receiving a good level of visibility with both the Russian partner and within the G8 community for the support of Russia's CWD program in general and the Kizner facility in particular.
  • Canada is benefiting directly and indirectly from the contribution to the Kizner CWDF; however there are further opportunities for leveraging the indirect benefits that are worth exploring.

Performance: Program Delivery Mechanisms

  • The design of Canada's delivery mechanism for Kizner is based upon a thorough analysis of international, recipient and Canadian capacities. Modifications to the originally planned mechanism were required due to the unexpected withdrawal of the UK partner; however the resultant design is equally robust.
  • IGX has formally documented lessons learned from Shchuch'ye and has effectively incorporated them in the Kizner design and delivery structure.

Performance: Performance and Project Management Framework

  • The performance framework set out in the Kizner Accountability, Risk and Audit Framework (ARAF) provides a basis for measuring and monitoring Kizner-related progress and outcomes. Given Canada's revised role as overall project co-ordinator, the current focus of IGX's performance management efforts is appropriately focused on project management aspects and milestone achievement indicators.
  • The project management framework for Kizner is consistent with the approved IGX framework. Roles, responsibilities and liabilities of all players and partners are well defined and all parties appear to clearly understand them.
  • The project management framework includes robust and appropriate communication and reporting protocols between key partners and stakeholders that support strategic decision-making and timely action on behalf of Canada. The only weakness identified in communication is the lack of formal reporting to IGX from Minpromtorg on the progress and status of construction activities at Kizner (including storage facilities), and the anticipated start date of operations at the two main destruction buildings.
  • A mature approach to risk management has been established and implemented to support Canada's contribution to the Kizner CWDF.

Performance: Efficiency and Economy

  • IGX has established, for all aspects within its control, numerous control and support mechanisms to ensure the efficient use of resources.
  • Canada's in-kind support to the Kizner project was delivered primarily through contribution agreements under Vote 10 authority. The only procurement arrangement between DFAIT and Teledyne Brown Engineering (TBE) was made in accordance with Government of Canada's procurement standards.
  • IGX has established robust processes to monitor the quality and reliability of the manufacturing processes, the safe and timely transport to Kizner and the appropriateness of storage facilities and arrangements. IGX has ensured that responsibility for the installation of the equipment at Kizner and its long-term maintenance rests clearly with, and is well understood by, the Russian partner.
  • No suggestions have been made for an alternative and more efficient program design and delivery mechanism. All stakeholders are pleased with the efficiency and effectiveness of the current model.

Performance: Achievement of Expected Outcomes

Although there have been some challenges and delays, as of mid-June 2009, considerable progress has been made by IGX against the timelines set out in the agreements with equipment suppliers. The Russian partner is pleased with the overall project progress to date. Although some uncertainties remain around the timely completion of the first and second main destruction buildings of the Kizner CWDF, there is strong assurance that the original objectives and scope of Canada's contribution to the Kizner CWDF will be achieved.

Lessons Learned

The evaluation resulted in the following lessons learned:

Need for external expertise and supports: The early recognition by IGX that external technical expertise would be required to manage and safeguard the Canadian contribution to Kizner is proving to be well founded. For future IGX programming, based on the Kizner experience, IGX should ensure that the planning phase includes careful assessment of the potential need to supplement IGX expertise with external resources.

Risk-Based Monitoring: IGX's formal, well planned and risk-based approach to monitoring has, to date, been very effective in managing and safeguarding Canada's contribution to Kizner. Future IGX projects should consider utilizing the Kizner-related approach to risk based monitoring as a model.

Project management: The sound project management approach embraced by IGX increases the likelihood that GPP projects are successfully delivered.

Reporting and Communication: The extent and frequency of reporting and the development of communication protocols between IGX and its partners have been instrumental in ensuring the continued progress and co-ordination of the Canadian contribution for Kizner. Future IGX projects should consider utilizing the Kizner-related approach of complementing formal reporting and communication protocols with informal and or supplemental approaches to address outstanding issues and gaps as they arise.

Risk Management Regime: A mature approach to risk management has been an integral element to the progress and success to date for Canada's contribution to Kizner. The integrated approach to risk management utilized within the Kizner project should be considered as a model for other IGX and DFAIT programming.

Leveraging the IGX experience: If there is no formal plan for leveraging the goodwill, expertise and networks gained or established through each project, the opportunities for increased visibility and synergies within IGX, DFAIT and or GoC programming will be lost.

Recommendations

The following recommendations stem from the analysis of the findings and the main lessons learned from the implementation of this project to date:

Recommendation #1:

The Kizner project management approach should be well documented and applied, where appropriate, to future IGX and DFAIT projects.

Recommendation #2:

IGX should ensure that, throughout the remainder of the Kizner project, sufficient focus remains on ensuring that formal and informal reporting and communication approaches are utilized, especially with regard to the construction status of the CWD facility.

Recommendation #3:

IGX should consider the need for a more formal strategy and approach to leverage the goodwill, expertise and networks gained or established through the Kizner project within IGX, DFAIT and or across government.

Top of page

1.0 Introduction

This report presents the findings, conclusions and recommendations of the Formative Evaluation of the Global Partnership Program (GPP) Funding to Support Construction of the Kizner Chemical Weapons Destruction Facility (CWDF).

1.1 Program Description and Background

The Global Partnership Program (GPP) of the Department of Foreign Affairs and International Trade (DFAIT) is providing funding to support the construction of a Chemical Weapons Destruction Facility (CWDF) at Kizner, Udmurt Republic of Russia. The Canadian support is being provided in the form of in-kind contributions for the provision of major equipment required for the chemical weapons destruction process in the two main nerve agent destruction buildings. Three major types of equipment are being provided: catalytic reactors; destruction process line (DPL) equipment and metal parts furnaces (MPF). As at June 2009, the estimated cost of the Canadian contribution is in the range of $98 million CDN, excluding salary and operational costs related to the support provided by IGX Ottawa and Moscow.1

Background and Context

In 1997, Russia ratified the Convention on the Prohibition of the Development, Production, Stockpiling, and Use of Chemical Weapons (the Chemical Weapons Convention or the CWC). One requirement of the CWC is complete destruction of declared stockpiles by all signatories by April 29, 2012. The Russian stockpile of 40,087 metric tones was the largest declared stockpile. At the time of signing the CWC in 1997, Moscow acknowledged insufficient funding to meet the CWC deadlines, and endorsed ratification only after assurances of financial assistance from Europe and the United States of America (US).

At the 2002 G8 Summit in Kananaskis, The Global Partnership against the Spread of Weapons and Materials of Mass Destruction (the Global Partnership) was launched. The Global Partnership is a cooperative threat reduction initiative intended to mitigate weapons and materials of mass destruction (WMD) proliferation and terrorism threats. The G8 partners agreed to raise up to $20 billion over 10 years up to 2012. Canada committed $1 billion over ten years to the Global Partnership and established the Global Partnership Bureau (IGX) within DFAIT to deliver on this commitment.

The G8 leaders agreed in 2002, that the initial focus of support by the Global Partnership should be on Former Soviet Union (FSU) countries in four priority areas, including chemical weapons destruction programming. The launch of the Global Partnership was thus a significant boost to the Russian CWD program.

Canada, as a major participant in the Global Partnership, has already contributed to several projects at the Shchuch'ye CWDF. The Shchuch'ye site is one of seven locations that hold Russia's declared stockpile. Canada's main contribution of $55 million dollars for the Shchuch'ye CWDF was used for the provision of major equipment, including catalytic reactors, destruction process line (DPL) and other miscellaneous equipment.

As of June 2009, five of the seven Russian CWDFs have completed, or are in the process of completing destruction operations. Shchuch'ye has already become operational. Two of the seven facilities - Pochep and Kizner are still under construction.

The Kizner CWDF

The Kizner CWDF is considered one of Russia's key facilities, since its CW stockpile represents 14.2% of Russia's stockpile by volume, and 45% of the total number of chemical weapons munitions. The specific type of chemical weapons stored at Kizner make this site a major proliferation threat with its arsenal of 5,680 metric tones of nerve agents stored in more than 2 million artillery and rocket launched munitions.

IGX has been considering and planning a potential contribution in support of the Kizner CWDF since 2005. In July 2006, the Government of Canada announced a contribution of $150 million to Russia under GPP. The 2007 submission to Treasury Board Secretariat sought approval for Kizner-related support for up to $120 million.

Project Delivery Structure

During the planning phase for the Kizner project, IGX considered three delivery mechanisms. Senior DFAIT and IGX management determined that the best option would be to build on the Shchuch'ye experience and provide Canada's support through the UK Ministry of Defence (UK MoD). As in the Shchuch'ye project, the intent was that the UK MoD would serve as the overall project coordinator for donor activities, provide technical expertise and advice to IGX, and oversee the contract that DFAIT would sign with a Principal Contractor for Canadian-funded activities. However, in 2008, the UK decided to withdraw from the Kizner project, and a revised project delivery mechanism was required.

In the revised delivery structure, Canada took over the lead role for implementing GPP's support for the Kizner CWDF. The IGX team in Ottawa became responsible for the overall program and project management, and IGX Moscow for monitoring and liaising with Minpromtorg and the Russian equipment suppliers.

The revised delivery structure includes the following players and arrangements:

Agreements between Canada and Russia

  • The 2004 Canada-Russia Bilateral Agreement for cooperation on GPP projects provides the overarching framework for Canada's support. The Bilateral Agreement deals with project implementation issues, including monitoring access, transparency, liability, Russian taxation exemptions, intellectual property rights, Russian permitting and licensing, adherence to Canadian safety and environmental standards, and privileges and immunities.
  • An Implementation Agreement (IA) between Minpromtorg and DFAIT was signed in November 2008. The IA operationalizes the conditions of the Bilateral Agreement and clarifies the specifics of the Kizner-related support including project description, scope, funding level, nature of arrangements with suppliers, communication protocols and other matters.

Agreements related to the in-kind contribution of equipment

  • A trilateral contribution agreement between Canada, Minpromtorg and the Federal State Unitary Enterprise--Federal Scientific and Production Centre "Pribor" (Pribor), was signed in December 2008 for the supply of catalytic reactors. The Pribor-related agreement authorizes total maximum payments of $16.45 million CDN.
  • A trilateral contribution agreement between Canada, Minpromtorg and the Joint Stock Company StankoAgregat (Stankoagregat) was finalized in July 2009 for the supply of Destruction Process Line (DPL) equipment, and was awaiting Minister's signature (as of July 31, 2009). Total expenditures related to this agreement, requiring payment in Russian roubles, are expected to be in the range of $40 million CDN.
  • A contribution agreement between Canada and Surface Combustion Inc. (SCI) was signed in January 2009, for metal parts furnaces (MPF) and auxiliary equipment. The agreement authorized total maximum funding of up to $32.6 million USD. Depending on currency fluctuations, it is anticipated that total cost will be in the range of $38 million CDN.

Agreements for support services to IGX

  • A Memorandum of Understanding (MOU), in the form of a contribution agreement between DFAIT and the Ministry of Defense of the United Kingdom (UK MoD), was signed in October 2008 for the provision of project support and engineering advice, on an 'as required' basis. The MOU authorized a total maximum funding of up to $2 million CDN. As of June 2009, IGX Ottawa was anticipating that the total amount of funding required under the MOU would be in the range of $400,000 CDN.
  • A contract between DFAIT and Teledyne Brown Engineering (TBE) was signed in February 2009, for the provision of the following monitoring and support services to IGX on a task authorization basis: Negotiation Support Services, Project Management Services, Monitoring Services and Other Related Services including research, analysis and translation. The contract authorized a total maximum funding of up to $4.5 million USD, including two years of contract renewal options. As of June 2009, IGX Ottawa anticipated that the total amount of funding required under the contract would be in the range of $3.5 million CDN.

Exhibit 1.1 provides an overview of the project delivery structure.

Overview of Project Delivery Structure

1.2 Evaluation Objectives, Issues and Questions

The main purpose of the evaluation was to provide evidence for project progress and advancement towards established goals and milestones. The evaluation followed the main evaluation criteria of relevance and performance, with emphasis placed on results and achievements to date.

Relevance was addressed through the following issues: alignment with DFAIT, Government of Canada and Russian priorities; relevance in the context of other donor activities, Canadian visibility, and benefits to Canada.

The issue of performance included four elements: Delivery Mechanism; Performance and Project Management Framework; Efficiency and Economy; and Achievement of Expected Outcomes.

1.3 Evaluation Approach

The evaluation was conducted in the period April to July 2009. The data collection approach included: file, document and database review; interviews with DFAIT and IGX officers in Ottawa; and a field visit to Moscow and London for in-person interviews with representatives of IGX Moscow, Minpromtorg, Pribor, Green Cross International2 and the UK MoD. Seventeen interviewees were included in the study.

The file, document and database review covered IGX-prepared documentation related to program and project background and planning; the Kizner-related contracts and agreements; the Kizner-related performance management frameworks; project plans; risk registers; correspondence and e-mails related to the project; project status and progress reports prepared by IGX and by project partners; monitoring plans and reports; reports and publications related to other donors' activities in support of the Russian CWD program and the G8 Global Partnership; reports and publications by the Organization for the Prohibition of Chemical Weapons (OPCW); miscellaneous news clippings provided by IGX and/or obtained through an Internet search; Green Cross International publications related to Russia's CWD program; and status reports related to the Kizner CWDF prepared by the Green Cross Public Outreach Office located near Kizner.

The next section presents the key findings and the analysis of the information and evidence collected.

Top of page

2.0 Relevance

The following aspects of relevance were addressed in this evaluation: Alignment with Government of Canada, DFAIT and Russian priorities and impact of cancellation; Canadian visibility and benefits to Canada.

2.1 Alignment with Government of Canada, DFAIT and Russian Priorities, and Potential Impact of Cancellation

2.1.1 Alignment with goals and priorities of the Government of Canada, DFAIT and IGX

Finding # 1:

Canada's contribution to the Kizner facility in particular, and the Russian CWD Program in general, continues to directly support the goals and priorities of DFAIT and the Government of Canada, as detailed in the department's PAA and the 09/10 RPP.

All IGX and DFAIT representatives view Canada's contribution to the Kizner facility in particular, and the Russian CWD program in general, as being fully consistent with and supportive of the goals and priorities of both DFAIT and the Government of Canada. As detailed in the Department's Program Activity Architecture (PAA) IGX programming is an integral component of DFAIT's "Diplomacy and Advocacy," activity which directly supports DFAIT's strategic outcome, "Canada's International Agenda.3 "In turn, this DFAIT strategic outcome directly supports the Government of Canada's outcome related to "A Safe and Secure World through International Cooperation."

IGX programming, including the contribution to Kizner, directly supports fulfillment of Canada's international commitment to the Global Partnership. Reports related to the 2008 G8 Summit in Toyako confirm that the G8 leaders continue to remain firmly committed to the goals and objectives of the Global Partnership. In Toyako, although the G8 leaders committed to the geographic expansion of the Global Partnership beyond the countries of the Former Soviet Union (FSU) to ensure new and emerging risks are addressed, they also identified the importance and continued need for support by the Global Partnership for ongoing chemical weapons projects in Russia.

Although the rationale for providing Canada's support for Kizner is well documented in initial concept papers, there was no overall IGX strategic or business plan document available for review, explaining the priority and position of the Kizner project within the context of the overall IGX programming. Moving forward, however, the evaluation team found that IGX was in the midst of preparing an overall strategy for IGX programming beyond the 2012 window.

2.1.2 Alignment with Needs and Priorities of the Russian Partner

Finding # 2:

Canada's support for the Kizner CWDF addresses an important and time-sensitive priority for Russia. The Kizner CWDF is a critical component of Russia's CWD program and Russia's ability to meet the CWC's April 2012 deadline for complete destruction.

As a signatory to the CWC, Russia is obligated to meet both the interim and final destruction deadlines. As of June 2009, Russia has met the extended CWC deadlines for Phase I and Phase II (1% and 20% destruction, respectively). The Phase III CWC deadline is December 31, 2009 which requires 45% destruction of declared chemical weapons, and the deadline for complete destruction is April 29, 2012.

Russia's ability to meet the final CWC deadline is dependent on several factors including:

  • the continued successful operation of the Shchuch'ye CWDF; and
  • the timely completion of the construction and commissioning of operations at both the Pochep and Kizner CWDFs.

One of the two destruction lines at Shchuch'ye began destruction operations in the first quarter of 2009, and the second line is expected to be operational in late 2009. Both the Pochep and Kizner facilities are under construction and planned to begin operations in 2009 and 2010, respectively.

Articles published in the Nuclear Threat Initiative's Global Security Newswire, and OPCW documents and reports illustrate Russia's continued progress and intent to meet all obligations under the CWC, despite the enormity of the challenge in terms of logistics, cost, technology and funding pressures. However, some of the documents also indicate that despite continued progress and publicly on commitments, some representatives of the CWD community remain doubtful of Russia's ability to achieve 100% destruction by April 29, 2012. Representatives of IGX noted that Russia is making good progress, and they remain optimistic that Russia will continue to make every effort to meet the April 29, 2012 deadline.

As of June 2009, Russia continued to express its firm commitment to the goals, objectives and deadlines of the CWC, including the construction and operationalization of the Kizner CWD. Minpromtorg representatives reiterated Russia's continued commitment to meeting the CWC deadline and noted that despite some complications, recent challenges in the global economy, and cuts to the Russian CWD budget, appropriate actions are being taken, and the committed deadline for destruction remains unchanged.

In terms of financial need, a review of IGX and OPCW- related documentation indicates that Russia has, since signing the CWC in 1997, repeatedly stressed the need for international donor assistance in order to meet the CWC imposed deadlines. Representatives of Minpromtorg expressed their sincere gratitude and appreciation for the considerable support provided by Canada for both the Shchuch'ye and Kizner CWDFs. Minpromtorg representatives noted that their CWD budget has suffered from some cuts in the first quarter of 2009, and this fact, combined with the decision of the UK to withdraw due to MoD budget planning issues and deadlines, has made the Canadian support even more critical to Russia's ability to complete the Kizner CWDF and commence destruction operations.

2.1.3 Impact of Cancellation

Finding # 3:

Eventual cancellation of Canada's contribution could have significant and negative impacts not only for Russia's commitments to CWC but also for the non-proliferation efforts of the international community.

All interviewees were of the view that cancellation of Canada's contribution to the Kizner CWDF would have significant and negative impacts, such as:

  • Severe damage to the positive relationship and trust developed between Canada and Russia;
  • Massive delays (potentially even cancellation) of the construction and commencement of operations at Kizner;
  • Severe impact and threat on the ability of Russia to meet the 2012 deadline; and
  • Tarnish the visibility and positive reputation Canada has achieved within the G8 and CWD community as a leader in the international non-proliferation activities.

2.2 Other Donor Activities, Canadian Visibility and Benefits to Canada

2.2.1 Other Donor Activities

Finding # 4:

Under the Global Partnership, Canada, other G8 partners and other donor countries have supported five of the seven Russian CWDFs. Canada is currently the only donor supporting the Kizner CWDF. With the exception of the two facilities still under construction (Kizner and Pochep), international donor activity in support of the Russian CWD program is winding down.

Interviewees explained that since the launch of the Global Partnership in Kananaskis in 2002, the G8 and other donor countries have co-ordinated and provided significant financial contributions to five of the seven Russian CWDFs. A review of OPCW related information, news reports and the UK's 2008 Annual Report on the Threat Reduction Programme indicate that Canada, the UK, the US, Germany, Switzerland, New Zealand, the Netherlands, Poland, Czech Republic, Italy and Norway have all provided funding towards Russia's CWD program.

As of June 2009, five of the seven Russian CWDFs are in the midst of, or have completed destruction operations. Although Switzerland and Germany continue to provide considerable support for construction of the Pochep CWDF, and the US continues to have some involvement in supporting the Shchuch'ye CWDF, all interviewees acknowledged that donor activity for the support of construction of Russia's CWDFs is in the wind down phase.

In terms of other donor support for Kizner, Canada and the UK made some initial efforts to secure additional funding from G8 and other donor countries, however no other donors came forward and in July 2008, the UK withdrew its support for the Kizner CWD. A representative of the UK MoD noted that many international donors were of the view that while support for current projects in Russia should continue, the international community should be cognizant of the fact that Russia:

  • has already made significant progress;
  • has developed an increased capacity for CWD programming; and
  • has access to sufficient domestic resources.

The UK MoD representative indicated that the UK, along with other donor states, are now turning their focus to a wider set of GP non-proliferation priorities and to states lacking sufficient resources.

2.2.2 Canadian Visibility

Finding # 5:

Canada is receiving a good level of visibility with both the Russian partner and within the G8 community for the support of Russia's CWD program in general and the Kizner facility in particular.

Representatives of Minpromtorg view Canada as a co-operative, respectful and very effective partner. IGX representatives confirmed this view, noting that IGX has an excellent rapport with and direct access to Minpromtorg officials at all levels. IGX representatives noted, as further evidence of Canada's visibility, which Russia specifically requested that Canada speak at the official opening ceremony for the Shchuch'ye CWDF in March 2009.

In terms of visibility within the international CWD community, Canada's leadership role in the Global Partnership and its support to the CWD program is highly regarded. Interviewees and documentation indicated that IGX managers and officers are active participants in CWD-related donor and G8 working groups. For example, there have been several instances where IGX representatives have made presentations and served as keynote speakers at international CWD programming conferences.

2.2.3 Direct and Indirect Benefits to Canada

Finding # 6:

Canada is benefiting directly and indirectly from the contribution to the Kizner CWDF; however there are opportunities for better leveraging of the indirect benefits.

Interviewees and documents consistently identify the direct and indirect benefits to Canada arising from the GP funding provided in support of the Kizner CWDF. Direct benefits include strengthening of the Canada-Russia bilateral relationship and of the political relations with G8 and other donor countries. Another important direct benefit is the earliest possible destruction of Russia's nerve agent stockpile and thus the reduction of global proliferation and terrorism threats which endanger Canadian security.

Indirect benefits to Canada include development of IGX contacts, networks and implementation structures that could be utilized by other DFAIT and OGD programming. For example, IGX Moscow representatives noted that they took part in a recent trade-related session to share their experience and knowledge with Canadian business representatives seeking science and technology business opportunities in Russia.

Although these examples are indicative of the positive achievements of GPP, the evaluation team noted that there was no documented strategy or plan ensuring that contacts, networks, and implementation structures developed by IGX are or will be fully leveraged.

Top of page

3.0 Performance

The issue of performance included four elements: Program Delivery Mechanism; Performance and Project Management Framework; Efficiency and Economy; and Achievement of Expected Outcomes.

3.1 Program Delivery Mechanism

Finding # 7:

The design of Canada's delivery mechanism for Kizner was based upon a thorough analysis of international, recipient and Canadian capacities. Modifications to the originally planned mechanism were required due to the unexpected withdrawal of the UK partner; however the resultant design is equally robust.

The originally planned project delivery mechanism included heavy reliance on the UK MoD as the overall project implementer, similar to the Schuch'ye project. However, when the UK unexpectedly withdrew its funding and participation in support of the Kizner CWDF, IGX was required to revise the delivery mechanism.

The revised delivery mechanism is described in Section 1.2 of this report. All respondents were of the view, that this delivery mechanism was well designed to serve as a robust delivery mechanism for the Kizner CWDF project.

A review of project planning documentation and discussions with IGX representatives indicated that international, recipient and Canadian capacities were considered and assessed in developing the revised delivery mechanism. Examples of the capacities assessed and the implications on the delivery mechanism are detailed below in Exhibit 3.1.

Table 1: Assessment of Capacities and Impact on Project Delivery Mechanism
Exhibit 3.1: Assessment of Capacities and Impact on Project Delivery Mechanism
Capacity AssessedImpact on Project Delivery Mechanism
The Russian partner has the best technical and other knowledge of design, manufacturing and other requirements related to the Kizner CWDF equipment
  • Assignment of sole responsibility for design of the equipment to Minpromtorg
  • Assignment of responsibility to Minpromtorg to sign off and formally accept all deliverables by the Russian suppliers as a precondition of milestone payments.
Pribor, StankoAgregat and SCI obtained technical, manufacturing, logistical, permitting and Russian importing experience in the supply of similar equipment to the Shchuch'ye CWDFNegotiation of contribution agreements with these three equipment suppliers, as directed by Minpromtorg
  • Russia expressed its capacity and willingness to comply with the accountability, due diligence and other requirements set out in the 2004 Canada -Russia Bilateral Agreement.
  • IGX has previously and successfully utilized the Bilateral Agreement to deliver other GPP projects.
Utilization of the 2004 Canada -Russia Bilateral Agreement.
The IGX teams in Ottawa and Russia, based on their experience with other GPP projects, and in particular with the Shchuch'ye CWDF, have developed understanding for working within the Russian bureaucracy and culture, and have gained sufficient trust and respect.Ability of the IGX team to fulfil the overall project implementer role
The IGX teams in Ottawa and Moscow acknowledged gaps in their technical knowledge related to CWD equipment and the considerable expertise of the UK MoDNegotiation of an MOU with the UK MoD for the provision of technical and other supports, on an as required basis
The IGX teams in Ottawa and Moscow acknowledged gaps in the availability and capacity of IGX resources to fulfil the requisite engineering/technical, project monitoring and other requirementsRetained, through an RFP process, a monitoring services company to provide such supports, as required.

3.1.1 Lessons Learned

Finding # 8:

IGX has formally documented lessons learned from Shchuch'ye and has effectively incorporated them in the Kizner design and delivery structure.

Interviews with IGX representatives and the UK MoD representative, and a review of project documentation indicate that considerable effort was made to identify, document and incorporate the lessons learned from Shchuch'ye in the Kizner- project. Examples of the Shchuch'ye lessons learned incorporated in the Kizner project design and delivery structure include:

  • limiting the scope of the project to equipment and purposely excluding more costly and complex infrastructure projects;
  • reliance on IGX Moscow for support in monitoring and liaising with the Russian partner and equipment providers;
  • reliance on a western project management company with experience in managing civil engineering projects in Russia; and
  • conducting site visits to inspect progress as a core practice and seeking independent technical advice.

3.2 Performance and Project Management Framework

3.2.1 Performance Management Framework

Finding # 9:

As the project proceeds, the performance framework set out in the Kizner ARAF will provide a basis for measuring and monitoring Kizner-related progress and outcomes. Given Canada's revised role as overall project co-ordinator, the current focus of IGX's performance management efforts is appropriately focused on project management aspects and milestone achievement indicators.

An Accountability, Risk and Audit Framework (ARAF) was developed for the overall GPP and for the Kizner-related project. When asked whether the performance frameworks set out in the ARAFs are being utilized, IGX representatives in Ottawa noted that the current focus is on project management aspects and milestone achievement.

A review of the Kizner and IGX ARAFs was conducted and the following is noted:

  • The indicators identified in the Kizner ARAF related to activity and output level indicators and measures are similar to those being actioned and monitored by IGX from a project management perspective; and
  • At the outcome level the ARAF for the Kizner project identifies measures and indicators only for the immediate and final outcome levels, and these indicators are either currently being monitored or will be monitored and tracked, as appropriate, as the project proceeds.

Examples of indicators included in the Kizner ARAF that are being considered, tracked and monitored by IGX include:

Activity and Output level

  • IGX design planning and consultation processes are appropriate and used by IGX management as articulated in the PMF;
  • IA signed with Russia;
  • clarity of governance agreements; and
  • adherence to project timelines and budgets.

Immediate Outcome level

  • level of contribution of GPP funded projects to construction of CWD capacity at Kizner; and
  • level and quantity of delivery of Canada's funding to the project compared to other contributors.

The Kizner project-related performance information being collected by IGX is aligned with the PAA, as indicated by a review of the expected results and indicators identified in the PAA and the performance information that will be available from IGX.

3.2.2 Project Management Framework

Project Framework, Roles, Responsibilities and Liabilities
Finding # 10:

The project management framework for Kizner is consistent with the approved IGX framework. Roles, responsibilities and liabilities of all players and partners are well defined and all parties appear to clearly understand them.

The IGX Project Management Framework is based on the Project Management Institute's (PMI) Project Management Body-of-Knowledge (PMBOK). A review of Kizner-related project documentation and the IGX Project Management Framework, as well as interviews with IGX Ottawa representatives, indicate that all major elements of the framework have been implemented including, the development of a concept paper, environment, capacity and other reports, risk analysis, feasibility analysis, project plans and risk registers.

With respect to roles, responsibilities and liabilities, a review of the seven legal arrangements and discussions with all respondents, indicate that these aspects are clearly defined, documented and understood by all the players and partners. Key roles, responsibilities and liabilities are summarized below in Exhibit 3.2.2.

Table 2: Key Roles, Responsibilities and Liabilities

Exhibit 3.2.2: Key Roles, Responsibilities and Liabilities

Player

Role, Responsibility or Liability

Development and approval of the design for all equipment being provided by Canada.

Minpromtorg

Identifying, and justifying the selection of, all equipment suppliers and for ensuring each proposal is in accordance with the Minpromtorg-approved design.

Minpromtorg

Certifying acceptance of the Russian manufactured equipment throughout the manufacturing process, as a pre-condition of milestone payment approval by IGX.

Minpromtorg

Safe transport of the equipment to Kizner

Each respective equipment manufacturer

Ensuring all required licenses and permits for transport and import (including tax exemptions) of the equipment are issued;

Minpromtorg

Assumes title to the equipment upon delivery at Kizner, at no time does title pass to Canada.

Minpromtorg

  • Asembly and installation of the equipment on site at Kizner;

  • Operation and ongoing maintenance of the equipment

Minpromtorg

Fulfilling all warranty obligations in accordance with the specific terms of their agreement.

Each respective equipment manufacturer

Ensuring that all spare parts, operating instructions and manuals, as detailed in each agreement are provided to the Russian partner.

Each respective equipment manufacturer

Exclusive responsibility and liability for the commissioning, and operation of the Kizner CWDF.

Russia

Communication and Reporting

Finding # 11:

The project management framework includes robust and appropriate communication and reporting protocols between key partners and stakeholders that support strategic decision making and timely action on behalf of Canada. The only weakness identified in communication is the lack of formal reporting to IGX from Minpromtorg on the progress and status of construction activities at Kizner (including storage facilities) and the anticipated start date of operations at the two main destruction buildings.

Representatives of IGX, Minpromtorg, the UK MoD, and Pribor explained that formal and informal communication and reporting mechanisms have been well established.

With respect to the communications between IGX Ottawa and Moscow, frequent e-mail exchanges, regularly scheduled meetings, and site visits ensure sufficient two-way communication. Within IGX Ottawa, sufficient reporting and communication is ensured through frequent team meetings, day to day interaction and formal bi-monthly status reports to senior IGX management. The formal requirement for communication and reporting between and among IGX, Minpromtorg and the other partners are well established in the legal arrangements and include:

  • Monthly teleconference calls between TBE, IGX, Minpromtorg and Pribor. Both Pribor and Minpromtorg representatives noted the usefulness of these calls as a way to raise and address issues in a very timely manner;
  • Formal minutes prepared by TBE for each monthly conference call;
  • Official correspondence between IGX and Minpromtorg;
  • Monthly written status reports from equipment suppliers, including updated project plans and risk registries (Note that one of the additional responsibilities of TBE is to develop comprehensive project plans and risk registries for the Russian equipment suppliers); and
  • Comprehensive quarterly reports to IGX from TBE on project progress and status, including an updated overall project plan.

Communication and cooperation between IGX and Minpromtorg is described by representatives of both organizations as very good. Informal communication between IGX and Minpromtorg for the Kizner project was noted by both parties as being direct, open, co-operative and frequent, unlike the Shchuch'ye project, where Canada did not have a direct relationship with the Russian partner.

However, even though there are formal requirements included in both the Russia-Canada Bilateral Agreement and the Implementation Agreement, there is no evidence that Minpromtorg provides IGX with formal reporting related to the status and progress of construction of the Kizner CWDF (including storage facilities) and or the anticipated start date of operations at the first and second destruction buildings. IGX must remain informed of these timelines in order to co-ordinate the timing of the Canadian investment and to avoid early delivery of the equipment and the resultant need for a lengthy storage period that could put the equipment at risk and or require additional Canadian financial support to ensure sufficient storage and safeguarding.

In the absence of formal reporting on the status and progress of construction of the Kizner CWDF, IGX representatives explained that the issue is raised in discussions with Minpromtorg and during monitoring visits. However IGX representatives also explained that they rely upon alternate and supplemental means for the information including: review of progress reports from the Green Cross Public Outreach Office located near the Kizner site; media reports in the Russian press and CWD community publications; and progress reports submitted by Russia to the OPCW.

Although the information obtained from Minpromtorg and other sources may not be complete and current, IGX officers remain confident that they have sufficient information to provide IGX senior management with credible evidence that Canadian investments are safeguarded.

3.2.3 Risk Management

Finding # 12:

A mature approach to risk management has been established and implemented to support Canada's contribution for the Kizner CWDF.

A review of risk registry and other documentation shows that a robust and well documented risk management regime has been implemented. IGX respondents explained the following:

  • IGX maintains a detailed project level risk register of both operational and strategic risks. Several versions of this risk registry were reviewed and it is noted that the registry appears to be comprehensive4 and updated on a regular basis;
  • TBE, in consultation with Pribor and IGX, maintains a detailed risk registry for the catalytic reactors project, and will also maintain one for the DPL equipment project once the agreement is in place;
  • Surface Combustion Inc. (SCI) maintains a risk registry related to the MPF project.

The practice of risk management appears to be well integrated within the project management approach, day to day operations and decision making. For example:

  • A discussion of key risks and mitigating strategies is included in each bi-monthly IGX status report;
  • Updated versions of risk registries are included in routine reporting to IGX from TBE;
  • IGX representatives explained that the scoping and planning of monitoring visits are developed based largely on consideration of information in the risk registries; and
  • All IGX representatives readily offered his/her view on the nature and status of key risks.

3.3 Efficiency and Economy

3.3.1 Efficiency and Economy5

Finding # 13:

IGX has established, for all aspects within its control, numerous control mechanisms and supports to ensure the efficient use of the funding level provided in support of the Kizner CWDF.

The TB Submission for the Kizner project sought approval for total project costs of up to $120 million. As of June 2009, it was anticipated, that the total Kizner-related contribution would be in the range of $98 million,6 excluding costs related to support provided by the IGX team in Ottawa and Moscow.

Based on a review of planning documents, IGX senior management appears to have determined the appropriate scope and funding level for the Kizner-related support based on several years of discussion and negotiation within IGX and between IGX, Minpromtorg, the UK MoD and the G8 CWD-related donor working group. Discussions with IGX personnel indicate that the project scope and level of funding to be provided was based on consideration of the following factors:

  • IGX's view of the extent of financial need of the Russian partner;
  • the extent of progress made by Russia towards fulfillment of their CWC obligations;
  • the availability of other donor funding to support this final facility in the Russian CWD program; and
  • consideration of factors related to the Canada-Russia bilateral relationship.

In terms of the efficient use of the level of Canadian funding provided, Canada can control or influence some aspects of planning, manufacturing and delivery to help ensure the efficient use of Canadian funding. However, the efficient use of Canada's contribution is also dependent on the actions and decisions of Minpromtorg, over which Canada has only limited influence.

Representatives of IGX Ottawa and Moscow, Minpromtorg, Pribor and the UK MoD cited numerous examples of the mechanisms, controls and supports that have been established to help ensure efficient use of the funding level provided by Canada. Planning and other project documentation confirm establishment and implementation of these mechanisms, controls and supports. Examples include the following:

  • The scope of Canada's Kizner-related contribution is limited to equipment. Construction and infrastructure projects were specifically excluded to avoid increased complexity, overhead and project management costs. As well, the scope of Canada's contribution is limited to the same type of equipment provided by Canada and the UK at Shchuch'ye in order to build on the knowledge and experience gained by both parties.
  • All three major equipment types are being supplied by the same manufacturer involved in the Shchuch'ye in order to capitalize on their experience with Russian requirements for technology, design, shipping, permitting and, in the case of the American supplier, with their prior experience with Russian importing requirements.
  • Senior IGX management took a proactive role in negotiations with the three equipment suppliers to ensure the appropriateness of scope and best value. The knowledge gained by IGX and the UK MoD with respect to the technical aspects, design and cost of the similar equipment supplied at Shchuch'ye served as benchmarks for these negotiations.
  • IGX entered into agreements with the UK MoD and TBE for the provision of technical support throughout the project, on an 'as required' basis, in areas where the IGX team did not have in-house expertise.
  • A strong project management framework has been established to formally and routinely monitor the progress and the risks of the project ensure issues are quickly identified and resolved, thus maximizing the efficient use of the Canadian funding.

3.3.2 Procurement and Contributions Agreements

Finding # 14:

Canada's in-kind support to the Kizner project was effected primarily through contribution agreements under Vote 10 authority. The only procurement arrangement is between DFAIT and Teledyne Brown Engineering, and the procurement was carried out in accordance with Government of Canada's procurement standards.

The only procurement arrangement involved in the provision of GPP funding to support the Kizner CWDF is the arrangement related to the provision of project monitoring and other support services. A Merx-posted RFP process for these services was conducted through PWGSC and a contract for up to $4.5 million USD was awarded to TBE. Discussions with Ottawa IGX representatives and a review of documentation related to the RFP process confirm that Government of Canada procurement standards were followed.

With respect to the equipment being provided in support of Kizner, the arrangements with the equipment manufacturers were effected through contribution agreements. The total amount to be funded through the contribution agreements is expected to be in the range of $94 million CDN,7 depending on currency fluctuations and any future scope amendments.

Interviews with IGX personnel, including IGX's Justice Canada legal representative, explained that because the Crown does not receive title to the goods or equipment, the arrangements with equipment manufacturers are not procurements. As well, since GPP is an assistance program, the in-kind and other arrangements involved in GPP programming are not subject to legal obligations imposed under various trade agreements such as the Agreement on Internal Trade (AIT) and the North American Free Trade Agreement (NAFTA).

In the submission to TBS for GPP Phase II, authority for utilizing contribution agreements and Vote 10 funding in such instances was requested and granted. In the submission IGX noted the following:

  • Because of the very special nature of some of the projects under GPP, the use of in-kind contribution arrangements is likely to be limited and infrequent; and
  • Despite the fact that trade and competitive contracting obligations do not apply, in order to ensure transparency and value for money, IGX will ensure that any supplier selected to deliver the "in kind" form, will be selected in an open and justifiable manner and will ensure that any such arrangements stand the test of public scrutiny in maters of prudence, probity and transparency.

Interviews with IGX representatives and documentation related to the three contribution agreements with equipment manufacturers indicated that IGX took several steps to fulfill the Department's due diligence obligations, including:

  • Requirement for Minpromtorg, as the party responsible for development and approval of equipment design and for supplier selection, to provide justification statements for each supplier selected;
  • Requirement for each supplier to submit detailed technical and cost proposals, project plans, quality control regimes and other supporting documentation to IGX;
  • Detailed technical and financial review of the proposals and other documentation submitted by each supplier by IGX, and as required, by UK MoD representatives; and
  • Involvement of senior IGX management in the negotiation of the scope, price and other key terms and conditions.

3.3.3 Monitoring aimed at ensuring the quality and safeguarding of the Canadian contribution

Finding # 15:

IGX has established robust processes for monitoring the quality and reliability of the manufacturing processes, the safe and timely transport to Kizner and the appropriateness of storage facilities and arrangements. IGX has ensured that responsibility for the installation of the equipment at Kizner and for the long term maintenance of the equipment at Kizner rests clearly with, and is well understood by, the Russian partner.

Based on interviews with IGX, UK MoD, and Pribor representatives and a review of the legal agreements and other project documentation, it became evident that IGX has established a very formal, well planned and risk-based approach to planning and conducting monitoring visits related to manufacturing, transport and storage.

IGX personnel have developed a tentative monitoring schedule for the duration of Canada's involvement in the project. The monitoring schedule was developed based on the assessment of risk, project progress and project milestones, however the scope and timing of each visit is subject to change depending upon actual progress and issues identified. IGX representatives explained that the planning for each monitoring visit, in terms of scope (# of stakeholders and partners to meet with, # of sites to be visited, # of items to be examined, etc.), timing, assessment criteria, and the expertise/personnel required, relies heavily upon consideration of the information included in the risk registries.

In terms of documentation and follow-up, a review of monitoring reports shows that each monitoring visit is well documented, issues and concerns are clearly identified and responsibility is assigned for prompt follow up on each identified issue.

Based on discussions with IGX representatives and a review of the June 2009 trip reports, it became evident that several challenges had occurred in ensuring IGX had access to sufficient and appropriate technical support for monitoring visits. Without such support for monitoring progress, identifying and resolving outstanding technical issues, the success of Canada's contribution could have been at risk. To date, IGX has been proactive and successful in working with both the UK MoD and TBE to quickly address any weaknesses in the quality.

Manufacturing: Responsibility for the manufacture of equipment in accordance with the Russian approved design rests with each supplier; however documentation and interviews indicate that IGX is actively involved in monitoring activities to ensure the quality of manufacturing. Monitoring activities related to the quality of the manufacturing process includes on-site inspection of premises, review of the adequacy of the implementation of quality assurance plans and of the oversight provided by each supplier over sub-contractors, physical inspection of completed deliverables, and monitoring to ensure the formal sign-off /certification by the Russian partner.

Transport to Kizner: Responsibility for the safe transport to the Kizner CWDF rests with each equipment supplier and Minpromtorg is responsible for facilitating all Russian related permitting and licensing requirements. As explained by UK MoD, IGX and Pribor representatives, transport of the equipment to Kizner is complex due to size, distance to the site, and the multiple layers of Russian government and military permitting and licensing requirements. The transport of the MPFs is further complicated not only because the equipment must be shipped from the US and imported into Russia, but also because the equipment has specific temperature sensitivities.

IGX officers are fully aware of all the transport-related issues and are actively working with all parties, monitoring progress and proactively taking action to ensure issues are identified and resolved. A good example of IGX's involvement is the shipment of the MPFs. Even though the planned shipment date was twelve months away, IGX officers, cognizant of the need to ensure that the MPFs are shipped by late summer/early fall due to temperature sensitivities, had started supporting SCI and Minpromtorg ahead of time with the planning and coordinating of the transportation, import, permit and license requirements.

Storage at Kizner: Responsibility for ensuring the safe storage of the Canadian provided equipment at Kizner rests with Minpromtorg. Discussions with IGX representatives and a review of project documentation indicate that IGX routinely includes and considers storage related issues within their monitoring regime.

No major concerns regarding the deterioration of the equipment were identified by interviewees, should the need for long term storage be required due to unanticipated delays in construction and commissioning of the Kizner CWDF, however, as explained by the UK MoD, IGX and Pribor representatives, there are at least two storage concerns that must be given careful consideration:

  • the solvents supplied by Pribor for the catalytic reactors must be stored in dry conditions; and
  • the MPFs have very strict temperature sensitivities.

As of the end of the evaluation team's field work in mid-June 2009, IGX representatives expressed some uncertainty around the adequacy of the Kizner storage facilities to support arrival of the first shipment of equipment from Pribor. A review of subsequent documentation related to the late June 2009 Kizner site visit indicated that the equipment had arrived and was stored. Although the monitoring team identified several storage-related deficiencies that required attention by the Russian partner, IGX representatives were satisfied with the overall adequacy of the storage facilities.

3.3.4 Alternative Program Design and Delivery

Finding # 16:

No suggestions have been made for an alternative and more efficient program design and delivery mechanism. All stakeholders are pleased with the efficiency and effectiveness of the current model.

All respondents reported that they are very satisfied with the program design and delivery mechanism established for provision of Canadian support for the Kizner CWDF. IGX representatives are pleased with the opportunity to utilize the Canada-Russia bilateral agreement as a basis for the overall project, as are representatives of Minpromtorg. A review of the relevant legal arrangements indicate that the comprehensive scope of the terms and conditions established, the scope of technical and other expertise and supports available on an as required basis to IGX, and the very positive and co-operative relationship between IGX and Minpromtorg have resulted in a very effective and efficient program delivery mechanism.

Representatives of Minpromtorg noted the following: "We are very pleased with the approach, issues are being resolved in a timely manner; the bilateral co-operation is fully satisfactory, equipment is being delivered without delay, the project has proven its high efficiency and we have no suggestions for improvement."

3.4 Achievement of Expected Outcomes

Finding # 17:

Although there have been some challenges and delays, as of mid-June 2009, excellent progress is being made by IGX against the timelines set out in the agreements with equipment suppliers and the Russian partner is very pleased with both the overall project and progress to date. Although there remain some uncertainties around the timing of completion of construction of the Kizner CWDF (including both the first and second main destruction buildings), there is strong assurance that the original objectives and scope of Canada's contribution to the Kizner CWDF will be achieved.

The original timeline anticipated by IGX, was that design documentation would be received by IGX from Minpromtorg in Spring/Summer 2007 and the Canadian provided equipment would be manufactured throughout 2008 and 2009 to accommodate the planned completion of construction of the Kizner CWDF (including both main destruction buildings) in 2009.

Both documentation and respondents indicate that these originally anticipated timelines have shifted outwards. As a result of delays in the provision of final designs to IGX by Minpromtorg, as well as an understanding by IGX that the Kizner construction schedule was likely to be pushed towards 2010, IGX negotiated delivery schedules with the equipment suppliers for the mid-2009 to 2010 timeframe.

The status of the three major equipment sub-projects, as at June 2009, is as follows:

  • Catalytic Reactors: Manufacture and production was on schedule as per the terms and conditions of the trilateral agreement, and in fact, was slightly ahead of schedule. Batch #1 was in the process of being delivered to the Kizner site and Batch #2 was in the midst of manufacture. The last delivery related to the catalytic reactors is planned for the first quarter of 2010.
  • Metal Parts Furnaces (MPFs): The manufacturing process was on schedule, as per the terms and conditions of the agreement with DFAIT and in fact, the monthly report noted that SCI was slightly ahead of schedule. Delivery of the MPF is currently scheduled for summer/early fall 2010.
  • Destruction Process Line (DPL) equipment: As of mid-June, Minpromtorg had not yet finalized the DPL design. Since the DPL equipment is the most important aspect of the Canadian support, IGX expressed some concerns with these design delays on the Russian side. For many months, IGX personnel have been proactively involved in discussion and negotiation with both Minpromtorg and StankoAgregat regarding both completion of the design documentation and finalization of the contract. We understand from subsequent discussion with IGX, that as of mid-July 2009, the trilateral agreement has been signed off by Minpromtorg and StankoAgregat and is awaiting signature by the DFAIT Minister, and IGX does not have any major concerns that the planned fall of 2010 delivery date is at risk.

With respect to the construction status and plans for the Kizner CWDF, Minpromtorg representatives informed the evaluation team in early June 2009 that the first of the two destruction buildings is planned for completion by the end of 2010. We understand from subsequent documentation provided by IGX that the second destruction building is eight months behind schedule. Given the current Kizner CWDF construction schedule and the current status of the manufacture and delivery of the Canadian supplied equipment, there are no concerns that any aspect of the Canadian support will interfere with Russian plans for commissioning the Kizner CWDF.

Despite the delays experienced in obtaining designs from the Russian partner and the extension of the Kizner CWDF construction schedule, Ottawa and Moscow IGX and UK MoD representatives remain optimistic that, because of the importance of the Kizner CWDF to the ability of Russia to meet the CWC final destruction deadline, the Russian partner will take the requisite actions to ensure that the Kizner CWDF is constructed and commissioned as quickly as possible.

Top of page

4.0 Conclusion

As of June 2009, excellent progress is being made by IGX against the timelines set out in the agreements with equipment suppliers; a very strong project delivery mechanism has been established and implemented, as has a robust project management and risk management regime. There are many complexities, risks and challenges involved in the project, however IGX is proactively identifying, managing and mitigating all risks. The Russian partner is very pleased with both the overall project and progress to date. Although there remain some uncertainties around the timing of completion of construction of the Kizner CWDF (including both the first and second main destruction buildings), there is strong assurance that the original objectives and scope of Canada's contribution to the Kizner CWDF will be achieved.

5.0 Lessons Learned and Recommendations

The following lessons learned have been identified and three specific recommendations provided based on these lessons and the evaluation findings:

5.1 Need for external expertise and supports

When IGX learned of the withdrawal of the UK partner in July 2008, IGX very carefully assessed what technical and other supports the IGX team would require to successfully take on the lead responsibility for overall project implementation. As a result, IGX established the MOU with the UK MOD for technical and project management support and entered into a contract with TBE for project monitoring, technical and other supports. As the project proceeds, the wisdom in establishing these arrangements and the increasing reliance by IGX on these external supports is becoming increasingly evident.

Lesson Learned: The early recognition by IGX that external technical expertise would be required to manage and safeguard the Canadian contribution to Kizner is proving to be well founded. For future IGX programming, based on the Kizner experience, IGX should ensure that the planning phase includes careful assessment of the potential need to supplement IGX expertise with external resources.

5.2 Risk-Based Monitoring

IGX has established and implemented a formal risk-based approach to monitoring the progress in providing Canada's contribution in support of the Kizner CWDF. The IGX monitoring regime includes: risk-based planning and scoping; development of assessment checklists and criteria, identification of the nature and extent of external technical expertise required for each monitoring activity, documentation of the findings of monitoring activities and prompt follow up on identified issues and concerns.

Lesson Learned: IGX's formal, well planned and risk-based approach to monitoring has, to date, been very effective in managing and safeguarding Canada's contribution to Kizner. Future IGX projects should consider utilizing the Kizner-related approach to risk based monitoring as a model.

5.3 The IGX Project Management Approach

The Kizner project management approach has been very effective to date.

Lesson Learned: The sound project management approach embraced by IGX increases the likelihood that GPP projects are successfully delivered.

Recommendation #1:

The Kizner project management approach should be well documented and applied, where appropriate, to future IGX and DFAIT projects.

5.4 Reporting and Communication

IGX ensured that detailed and formal reporting and communication mechanisms were stipulated in the various agreements. In instances where IGX felt the information reported would be not be timely and or sufficient, IGX established alternate and or informal mechanisms to fill those gaps.

Lesson Learned: The extent and frequency of reporting and the development of communication protocols between IGX and its partners have been instrumental in ensuring the continued progress and co-ordination of the Canadian contribution for Kizner. Future IGX projects should consider utilizing the Kizner-related approach of complementing formal reporting and communication protocols with informal and or supplemental approaches to address outstanding issues and gaps as they arise.

Recommendation #2:

IGX should ensure that, throughout the remainder of the Kizner project, sufficient focus remains on ensuring that formal and informal reporting and communication approaches are utilized, especially with regard to the construction status of the CWD facility.

5.5 Risk Management Regime

IGX has implemented a solid risk management regime that includes maintenance of several comprehensive risk registries that address strategic, overall project level and sub-project level risks.

Lesson Learned: A mature approach to risk management has been an integral element to the progress and success to date for Canada's contribution to Kizner. The integrated approach to risk management utilized within the Kizner project should be considered as a model for other IGX and DFAIT programming.

High risk areas for the Kizner-related project include: manufacture of the destruction process line equipment; transport, import and delivery of the metal parts furnaces to Kizner, storage of equipment and related supplies on site at Kizner, timely completion of construction and commissioning of the Kizner CWDF (including both destruction buildings); and foreign currency fluctuations.

5.6 Leveraging the IGX Experience and Assets

IGX representatives have acknowledged that insufficient planning and focus has been placed on ensuring that the considerable goodwill, expertise and networks gained by IGX through the provision of support for the Kizner CWDF, are fully leveraged within IGX, DFAIT and or across government.

Lesson Learned: If there is no formal plan for leveraging the goodwill, expertise and networks gained or established through each project, the opportunities for increased visibility and synergies within IGX, DFAIT and or GoC programming will be lost.

Recommendation #3:

IGX should consider the need for a more formal strategy and approach to leverage the goodwill, expertise and networks gained or established through the Kizner project within IGX, DFAIT and or across government.

Top of page

6.0 Management Response and Action Plan

Table 3: Management Response and Action Plan
RecommendationManagement Response and Action PlanResponsibility CentreTime Frame
Recommendation 1:

The Kizner project management approach should be well documented and applied, where appropriate, to future IGX and DFAIT projects.

Associated Findings: 9, 10, 12, 13

Agree. IGX/CWD will continue to document the Kizner project until completion as is currently being done. At the end of each project, a lessons learned final report will be prepared and shared within IGX and relevant DFAIT groups.IGXPMF: Three months after completion of the Kizner program.

Lessons learned final report: Three months after completion of each project (Catalytic Reactors, Metal Parts Furnaces, Destruction Process Lines, Green Cross Funding)

Based on the lessons learned, the IGX Project Management Framework (PMF) will be updated accordingly, if appropriate.IGX/CWD
Recommendation 2:

IGX should ensure that, throughout the remainder of the Kizner project, sufficient focus remains on ensuring that formal and informal reporting and communication approaches are utilized, especially with regard to the construction status of the CWD facility.

Associated Findings: 11

Agree. It is important to continue maintaining and building the network within the framework in place to ensure accurate access to formal and informal sources. This will be achieved through regular progress meetings and reports, meetings and conference calls with Russian officials and partners, the use of the diplomatic channels, and cooperation with the Canadian Embassy (MOSCO) and the Organisation for the Prohibition of Chemical Weapons (OPCW).IGX/CWDEntire duration of the project.
Recommendation 3:

IGX should consider the need for a more formal strategy and approach to leverage the goodwill, expertise and networks gained or established through the Kizner project within IGX, DFAIT and or across government.

Associated Findings: 14, 15, 16

Agree. A written strategy will be created to formalize how CWD reports and how leverage is achieved under the Kizner project, in coordination with the Geographic Branch.IGX/CWDTwo month after the Formative Evaluation is approved.

Top of page

Footnotes:

1 Total estimated costs, excluding IGX-related salary and operational costs, is anticipated to be in the range of $98 million CDN, as follows: DPL equipment in the range of $40 million, catalytic reactors for $16.45 million, metal parts furnaces in the range of $38 million, services and support from the UK MoD in the range of $400,000 and from Teledyne Brown Engineering in the range of $ 3.5 million. Total costs will vary depending on any future scope changes and on currency fluctuations since the DPL equipment agreement is based on Russian roubles and both the metal parts services agreement and the Teledyne Brown Engineering contract are in USD.

2 Green Cross International is an environmental non-governmental organization (NGO) which advocates the safe and environmentally sound destruction of chemical weapons. Green Cross runs a network of local public outreach offices in Russia, which aim to provide independent and objective information about Russia's CWD program to local population. In support of the Kizner CWDF, DFAIT has been providing funding to support establishment and operation of a Green Cross Public Outreach Office in Izhevsk (Udmurt Republic), near site of the Kizner CWDF.

3 DFAIT strategic outcome "Canada's International Agenda" is described as follows: The international agenda is shaped to Canada's benefit and advantage in accordance with Canadian interests and values.

4 The IGX risk registry includes, for each identified risk: the risk statement, an overview of the risk conditions, the potential effects the risk in terms of cost, schedule, scope, safety and public concern, mitigating actions, positions responsible for implementing the mitigating actions, and risk closure criteria.

5 The TBS Policy on Evaluation defines efficiency and economy. Efficiency is defined as "the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output." Economy is defined as "minimizing the use of resources. Economy is achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes."

6 See Footnote 1 in Section 1.1.

7 See further funding details in Footnote 1 in Section 1.1.

Office of the Inspector General


* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Footer

Date Modified:
2012-11-22