The purpose of the Internal Audit of the Global Partnership Bureau (IGX) was to review and provide assurance that the appropriate management control framework was in place and operating economically, effectively and efficiently and that IGX had complied with the terms and conditions for the Global Partnership Program (GPP). This internal audit also satisfies the requirement contained in the Treasury Board approved Results-Based Audit Framework (RBAF) for the GPP that an internal audit of IGX be conducted.
The documentation review and interview phase of the audit took place between March 11 and May 12, 2005. Methodology consisted of interviews with key personnel, file reviews and field visits to partners.
Since its standing start three years ago, the GPP has made significant achievements. In particular: IGX was created and staffed with highly skilled people; the necessary Treasury Board (TB) approvals obtained; a Project Management Framework (PMF) developed which provides a project delivery methodology; a Cooperation Agreement (Treaty) between Canada and the Russian Federation signed which provides the legal framework for the implementation and delivery of projects in Russia; and nine complex programs/projects have been developed, approved and are being implemented. All of this has been accomplished while working in a very difficult environment.
The following three factors have had a significant impact on the findings and recommendations contained in this report:
The Department needs to complete a full assessment of the human resource, training, financial, information management, systems and methods support, results management and project and program management requirements to meet the challenges of the current and new programming for which it is assuming responsibility. It needs to adopt a comprehensive approach and develop department-wide solutions.
Many of the strategic and critical management tasks (planning, risk management, reporting) and functional support services (finance, contract, human resource management) need to be enhanced to adequately support the new program and project execution responsibilities that the Department is undertaking.
The International Security Branch (IFM) is responsible for developing and delivering the most significant portion of the new programming mentioned above. In order to ensure a comprehensive and coordinated approach to programming and project delivery in IFM, a Strategic Management Bureau headed by a Director General reporting to the Assistant Deputy Minister should be created. This level of reporting relationship is necessary to ensure an effective coordinated approach to Branch management activities.
The new Bureau would be responsible for two tiers of management and support activities:
The Phase I submission for the GPP, which received TB approval in August 2003, authorized:
These initial efforts are being implemented through established multilateral institutions, including the International and Science and Technology Centre (ISTC), International Atomic Energy Agency (IAEA), European Bank for Reconstruction and Development (EBRD) and, in one case, through a G8 partner United Kingdom - Ministry of Defence (UK-MOD), all of which are already engaged in the project area.
Phase II of the GPP, which received TB approval in October 2004, authorized:
In addition to the approvals obtained in the Phase I and II TB submissions, in July 2004, IGX received approval from TB for the $120M nuclear submarine dismantlement project.
At the request of IGX, the Audit Division completed in June 2004 an Advise and Assist Review concerning the early implementation of IGX. The report recommended the creation of a centralized support service unit under the leadership of a Director reporting to the Senior Coordinator. This was partially addressed with the creation of the virtual Project Management Office (PMO). However, project staff do not have a clear understanding of the roles assigned to support staff such as communications, finance, contract and project support. During the audit, steps were taken to ensure that there is a clear understanding in the Bureau of the role of these support specialists. Other recommendations were made addressing the development of project management tools, a training program geared to the specific needs of the project management staff and a formal review of the approved RMAF and RBAF. These recommendations have been partially implemented.
IGX has evolved from a small group of individuals implementing a new and complex program to a more steady state of operations. Since December 2004, the number of staff has grown from 18 to 33 people and IGX management recognizes that the informal practices followed in the past must now be formalized. It is taking action to introduce more formal planning and budgeting processes, risk management, human resource planning, support and administrative practices.
Overall, IGX has made progress towards implementing the first four GPP Phase I projects. However, all four have experienced a number of implementation delays. All of the G7 partners have experienced similar delays in delivering their projects in the Russian context. The reasons for these delays have been well documented and in all cases IGX has taken appropriate action. The $120 M Nuclear Submarine Dismantlement project, approved by TB in July 2004, is on target and achieving the planned milestones.
The absence of a single point of service within the Department made the work of IGX far more complicated. Time was lost and false starts were made before IGX management appreciated fully the range of systems they needed to develop in support of the GPP programming.
IGX has made significant efforts to develop appropriate systems and procedures but gaps still remain. A number of the recommendations in this report are related to IGX formalizing some of the management processes it currently carries out on an informal basis. In particular, IGX should formalize its planning and budgeting, bureau level risk management, human resource planning and communications processes. As the number and value of projects grow, it is essential that a more formal approach be taken.
With regard to planning and budgeting, IGX should undertake a review of the original priorities and objectives established for the program. While reviews have taken place as part of the annual G-8 reporting process, IGX should develop a more in-depth three year strategic plan which articulates the results it intends to achieve and the resources it will require to achieve these results. This would provide a basis for IGX to measure its performance against planned results. Taking a longer term strategic planning perspective will ensure that all options are being considered and avoid having to make costly adjustments and disruptions to the organizations' operating environment.
Over the past year, the IGX project management staff's knowledge, experience and capacity to manage complex high risk projects has significantly increased. The development and application of the Project Management Framework (PMF) has contributed to sound project management practices being followed in IGX. However, there is still work to be done in this area. In concert with any Departmental efforts, IGX should take action to develop or enhance the necessary project management tools such as a project management system (e.g. project results, milestone tracking), information management systems (e.g. records management, by ensuring that the existing paper based filing system is being fully utilized and adopting the new electronic systems as they mature), standardized formats, and other project support mechanisms, that are required to effectively manage complex high risk projects.
IGX has taken an informal approach to Human Resource planning and should take steps to formalize its Human Resource planning with respect to succession planning, staffing, and training and development. As part of the plan, IGX should undertake a comprehensive review of the number and type of human resources it will require over the longer term. The development of the plan should be done in cooperation with the Human Resources Branch.
Although project level risks are regularly reviewed, there has been no formal review of the program level risks and mitigation strategies identified in the TB Submission approved in August 2003. IGX management told the Audit Team that they frequently discuss the program risks and take appropriate action; however, the process is not formally documented. IGX should immediately undertake a formal review to update the program level risks and mitigation strategies identified in the TB Submission and report accordingly. Conducting annual reviews of identified risks and mitigation strategies is an effective management tool, especially in the relatively high risk environment which IGX delivers its programming.
The IGX Program RBAF indicates that communications with members of Parliament, Cabinet, central agencies, and within the Federal Government generally and communications and outreach programming through NGOs, academia, business and the general public will continue to be the most effective strategy to keep the Weapons of Mass Destruction (WMD) issue in the forefront of the public consciousness. Such communications are proposed as key risk mitigation strategies. Given the importance attached to having continuing support for the Program, IGX should develop and have approved by the ADM a formal comprehensive communications strategy. The forthcoming annual report to Parliament, which is in the final stage of preparation, will be a key document in addressing the identified risks.
This report contains other findings and recommendations which will require the attention of senior management. While they are not highlighted in the executive summary, they are no less critical to the overall successful management of the GPP. These include recommendations related to operational and work planning, completion of performance appraisals, governance and information technology.
Implementing the recommendations contained in this report will require a significant commitment of resources to ensure their successful realization. It is recognized that not all of the recommendations can be implemented simultaneously and will need to be prioritized. Implementation of the recommendations will assist DFAIT and IGX to manage and deliver their projects and provide the financial and risk control environment required to demonstrate prudent and effective management of program funding.
1.1 The GPP was created to implement projects in Russia and other countries of the former Soviet Union, in cooperation with these countries and/or other G8 nations, to reduce the threat posed by weapons and materials of mass destruction (WMD) to Canadians and the international community. The GPP has been allocated a budget of $1 billion over a ten year period.
1.2 IGX is responsible for the overall management of the GPP, including program/project approval, implementation, monitoring and reporting.
1.3 The Phase I submission for the GPP, which received TB approval in August 2003, authorized:
These initial efforts are being implemented through established multilateral institutions (ISTC, IAEA, EBRD) and, in one case, through a G8 partner (UK-MOD) already engaged in the project area.
1.4 Phase II of the GPP, which received TB approval in October 2004, authorized:
1.5 In addition to the approvals obtained in the Phase I and II TB submissions, in July 2004 IGX received approval from TB for the $120M nuclear submarine dismantlement project.
1.6 Based on the approved Results-Based Audit Framework (RBAF), recipient audit strategies have been developed for the four Phase I projects and strategies are currently being developed for the two Phase II projects. The strategies place considerable reliance on the audit functions of the organizations responsible for the execution of the projects.
1.7 Initially an internal audit of the GPP was planned for FY 2003/04 with a follow-up audit in FY 2004/05. At the request of IGX, in FY 2003/04 an Advise and Assist Review was undertaken. It was felt this would be more helpful to the Management of IGX in its efforts to establish the appropriate results-based management control framework required to effectively deliver the GPP. This internal audit will satisfy the requirement for an internal audit contained in the approved RBAF for the GPP.
1.8 The purpose of ZIV's audit of internal management of the program was to review and provide assurance that the appropriate management control framework is in place and operating economically, effectively and efficiently, and that IGX has complied with the terms and conditions for the Program.
2.1 In order to establish the appropriate risk-based scope and objectives for the internal audit, interviews were held with key members of the management and staff of the IGX, Assistant Deputy Minister, International Security Branch and Political Director, the Treasury Board Analyst, the Director General of SMD and the Senior Program Analyst in SMPA.
2.2 The scope of the audit included the following eight management areas:
The criteria used to assess the areas included in the audit scope, and agreed to by IGX, are contained in Annex "A".
2.3 The scope of the audit excluded an in-depth review of financial transactions and contracting practices. These practices will be the subject of a compliance audit scheduled to take place in the summer of 2005. It also excluded a review and assessment of the activities carried out by the IGX staff located in the Moscow Embassy.
2.4 The purpose of the internal audit of IGX was to review and provide assurance that the appropriate management control framework was in place and operating economically, effectively and efficiently and that IGX had complied with the terms and conditions for the GPP. This internal audit also satisfies the requirement contained in the TB approved Results-Based Audit Framework (RBAF) for the GPP that an internal audit of IGX be conducted.
3.1 The audit methodology consisted of conducting interviews with IGX and other applicable departmental functional support staff, central agency staff, partner/donor organizations, reviewing documentation and carrying out appropriate analyses. The audit methodology included the following key elements:
3.2 The documentation review and interview phase of the audit took place between March 11 and May 12, 2005.
3.3 Twenty-eight individuals were interviewed including the management and staff of IGX, representatives from the functional support areas (finance and human resources), Area Management Office (IAM), International Assistance Envelope (IAE) management and the Treasury Board Analyst. In addition, representatives of ZIE and ZIV visited the organizations implementing the first four GPP projects: International and Science and Technology Centre (ISTC), International Atomic Engery Agency (IAEA), United Kingdom - Ministry of Defence (UK-MOD) and the general contractor (Bechtel) engaged to complete the railway construction project, European Bank for Reconstruction and Development (EBRD) and the Russian shipyard responsible for submarine dismantling. The team also visited the Moscow Embassy. A review of the key project documentation for the Phase I and Phase II projects was conducted.
4.1.1 The availability of key support services (e.g. finance, human resources, contracting/contributions and information systems) is essential to the effective delivery of programs and projects. As a policy Department, DFAIT does not have the required central corporate supporting infrastructure to deliver programs and projects. Support service issues are left to the individual manager to resolve with assistance from the functional groups. Both the managers and functional groups are limited because of inadequate resources, the absence of a single authority for program and project support and the lack of corporate program and project management standards to guide a new program. The lack of key support services (e.g. finance, human resources, contracting/contributions and information systems) impedes the smooth delivery of programs and projects and the roll-up of results against departmental and government priorities. The Department needs to develop better systems to support its increasing program responsibilities.
4.1.2 Given the wide range of strategic and critical management tasks (planning, risk management, reporting) and functional support services (finance, contract, human resource management) that do not receive the attention they require, there is a need to ensure a comprehensive and coordinated approach to programming and project delivery in the International Security Branch. A Strategic Management Bureau headed by a Director General reporting to the Assistant Deputy Minister should be created. Presently, Bureau Management agrees that there is a need for a support unit at the Branch level, led by a Director reporting to the ADM through the Bureaux DG's. The Audit Team believes that a unit led by a DG reporting to the Branch ADM would be close to the programs and would provide a more strategic focus at the branch level. Management also agrees that the focus of the unit will include strategic planning, results management, risk management, program/project delivery policy and guidelines, liaison with Treasury Board and internally in FAC on management and support issues, coordinated reporting and communication outreach.
4.1.3 In addition to the support unit, the functional areas must have the knowledge and experience associated with program/project management in order to effectively support IGX and new program/project delivery. Based on the interviews conducted with representatives from the functional areas, they do not have the necessary knowledge, experience or resources to provide effective services. This is a serious Corporate issue given the new programs (e.g. Peace and Security, Counter-Terrorism, Canada Fund) FAC will be undertaking.
4.1.4 The financial and non-financial information systems currently in place in FAC were not designed to facilitate the capture and reporting of information required in a program/project management environment. In order to overcome these weaknesses, some project management software has been acquired by individual project managers. This forces staff to maintain complicated and time consuming excel spread sheets and to manipulate the current financial system to obtain the information required to monitor and control their programs/projects. It has been agreed that a longer term approach, based on a comprehensive plan, will be taken to address deficiencies in the financial and non financial systems.
4.1.5 The Audit and Evaluation Divisions have received additional resources to support IGX and it will receive resources for the new programming. The functional areas, however, have not received additional resources as they were not requested in the Treasury Board submissions. In order to properly support program/project delivery, the functional areas will require additional resources and these should be requested in future Treasury Board submissions. Resources will be required for the development and maintenance costs associated with the upgrade of the financial and non-financial information systems currently in place in FAC in order to support a program/project management environment. Resources will also be required for Human Resources Branch to address the personnel demands associated with the new program/project environment.
4.1.6 IFM should establish a Bureau, led by a DG reporting to the ADM, with adequate resources that would provide two tiers of management and support activities:
4.1.7 Where appropriate, IFM should ensure that all future Treasury Board submissions contain a request for appropriate additional resources for the functional areas.
4.1.6 IFM agrees with the need to establish a separate unit to be responsible for strategic management at the Branch level and to provide support services to the GPP and other programs delivered in the Branch. Following extensive consultation with the managers of all branch programs, IFM formally established IXS as the Program Services Division in August 2005. The unit is based extensively on recommendation 4.1.6 but will be managed at the director level (already appointed) and for now, will leave communications and outreach activities to be implemented at the bureau level. IFM recognizes the need to bring the core functions of IXS fully on line quickly and consequently took steps to establish the unit prior to the audit report's completion. At the same time, however, IXS represents a significant change in the way support is provided for program delivery in IFM and it is important to integrate the model into ongoing operations carefully. Some elements of its ultimate structure and mandate will be subject to ongoing refinements as the unit evolves. Consequently, IFM does not exclude the possibility in the medium term of expanding the mandate of the Program Services Division as per recommendation 4.1.6 or elevating its management to the Director General level if program delivery experience indicates that such an action would be appropriate and beneficial.
a) Financial - provide the interface with IAM and SMD for all program delivery as well as advice, general support and quality control for financial tracking at the program level. In addition to the FI-03 in IXS, FI-02s are in place with IGX and START, a significant increase in financial management resources for the IFM Branch.
b) Legal Services - Covers legal requirements of all programs. In an agreement with JUS Legal Services, two JUS lawyers are provided for in IXS. At present, two experienced JUS lawyers are in place in IXS although one will retire in April 2006. A competition for the second position will be completed in April. An administrative support position has been filled.
c) Contracting - Agreement is in place with SPD to staff two positions that will work on a day-to-day basis in IXS but will be on loan from the Centre of Excellence for Contracting (SPPG). One PG-04 contracting specialist is in place in IXS. Consideration is being given to having the second position focus on grants and contributions and discussions are underway with the Centre of Excellence for Grants and Contributions (SMFH) based on the SPPG model.
d) General Project Management Services - Staffing completed in September.
e) Information Management and Systems support - A business analyst was engaged through Consulting and Audit Canada to provide options for enterprise-level information management solutions which would integrate financial and project management as well as provide for interoperability across programs and with OGDs, in particular CIDA. This study will be completed by March 31, 2006. In cooperation with CIDA, implementation of the recommendations would take place in FY 06/07 with the target date of having the new system operational by April 2007.
f) Human Resource management - Human resources issues continue to be a major challenge for all program delivery groups and other Bureaux in the IFM Branch. The IFM Branch has created an additional FTE in IXS to address the HR needs of the Branch as a whole and the position should be staffed by May 2006. Under an MoU, the Human Resources Branch will provide the IFM Branch with a senior PE who is familiar with the department and certified to provide advice on both the Public Service Modernization Act (PSMA) and the Public Service Employment Act (PSEA). The PE will assist in all aspects of rotational and non-rotational staffing and classification issues, provide advice to Branch managers on the implementation of the PSMA and PSEA and interface with the Branch corporate and strategic planners to ensure proper alignment of human resources planning with corporate and financial planning. The immediate priority for this individual will be to address some of the critical staffing challenges faced by the START and any remaining staffing challenges faced by the CTCB Program Secretariat in ICT ICX groups. The IFM Branch intends to have competitions underway by August for all remaining un-staffed positions in the program delivery bureaux.
For strategic services, IXS will be directly responsible for ensuring that program management principles, including RBM, risk management, strategic planning, and quality control are used extensively at the Branch level. It will also assist individual programs in each of these areas by organizing training courses, sharing best practices and providing counsel.
a) Program management training programs have been implemented by IGX.
b) Results-based management (RBM) is already being implemented at the level of individual programs in the Global Partnership bureau as well as the START and Counter-Terrorism Capacity Building Programs. This takes place through the PMA and RMAF processes, through the use of formal PMFs and via the integration of clear milestones and associated reporting into project implementation.
c) To expand the application of RBM principles throughout the Branch, the ADM's PMA has been modified to encompass an RBM provision and this will cascade down to the senior management levels in fiscal year 06/07. This would involve a course on RBM for senior managers to take place by the end of March 2006. RBM training for program managers will be implemented by June.
d) To expand the use of RBM and risk management principles throughout the IFM Branch, a formal integrated approach to RBM and risk management will be initiated by the ADM for International Security (IFM) before July 2006. Results will be tracked through a Results-Based Management Accountability Framework for the Branch that takes due account of the commitments of individual programs as approved by TB. The IFM Branch RMAF will be developed by June 2006 and integrated to both the PMA and strategic planning processes.
e) Risk management is in place at the project level in all program areas by virtue of the RBAFs included in the respective TB submissions. In IGX, risk registries are fully in place at the project level and the program level registry is in process. Appropriate risk monitoring mechanisms in other programs will be completed by November 1, 2006. Training on risk management for IGX was implemented in December. Risk management training for START and ICTC program managers will be completed by July 2006.
f) To achieve a more consistent and systematic approach to risk management at the project, program and Branch levels, the Director, IXS, will serve as the Branch Risk Officer and establish a Risk Management Committee for the IFM Branch. The committee will be made up of members from the program delivery Bureaux and other divisions, as appropriate. The first meeting of this group will take place in March 2006.
Corporate and strategic planning issues - A new FTE has been added to IXS to work on corporate and strategic planning issues at the Branch level. Key functions of this position include the development of IFM Branch inputs into FAC's corporate planning process in cooperation with all five bureaux as well as strategic planning responsibilities related to the direction of program delivery for the IFM Branch as a whole. Integration of corporate and HR planning at the Branch level as required by the PSMA will be a priority. A request to HR to staff this position was not fulfilled; another request has been submitted for the current assignment cycle.
4.1.7 Agreed. IFM already works on the basis that where appropriate, functional areas should be funded for the services provided directly to program management units and provisions are in place specifically to fund audit and evaluation activities directly from Global Partnership Program sources. IFM also notes, however, that there are three sources of funds for the functional areas: the existing A base, common service charges levied on new positions and program funds. FAC has introduced a new formula for the calculation of common service charges which is applied to the costing for all new positions. The levy represents a significant increase over previous charges and will impose a heavy cost on this Branch. IFM expects that these funds will be applied to program requirements and that they will be fully reflected in any request to Treasury Board for additional resources.
5.1.1 Sound planning over both the short and medium term will ensure that an organization has aligned the allocation of resources to the achievement of the results and outcomes it is accountable to deliver. It also provides the basis on which it can monitor and verify progress towards the achievement of the desired results.
5.1.2 In IGX, strategic planning is done on an informal basis, except for the input it provided for the 2005-2008 Corporate Business Planning process and in its TB Submissions. This input is at a very high level and does not provide sufficient detail to guide the medium and long term attainment of the results and objectives described for the GPP. More importantly, it does not provide a comprehensive base against which progress towards the desired results can be monitored.
5.1.3 The GPP was established in June 2002 and since that time there has been no formal review of the original priorities and objectives established for the Program. IGX management told the Audit Team that they have conducted informal reviews of program priorities and objectives, however, the process is not formally documented. There is no formal systematic review of performance against the medium and long term objectives of the Program. Reviews that do take place are at the project level and relate to the immediate progress or lack thereof for a specific project.
5.1.4 IGX has completed a formal ten year financial forecast. It clearly demonstrates that the original flat line forecast will not meet the programming needs of the GPP. A re-profiling of the planned expenditures has been requested.
5.1.5 Operational and work planning are generally informal, the exception being the performance objectives agreed to by the members of the EX group as part of their annual performance agreements. Work priorities are discussed at the weekly management and staff meetings. They are also discussed during bilateral meetings between staff and managers and one Senior Project Manager uses semi-annual staff retreats to establish priorities for his section.
5.l.6 As there is no formal business planning process in place, there is no documented evidence that regular performance monitoring is carried out and adjustments made as the need arises. In the initial stages of IGX, given its relatively small complement of staff, the informal planning and budgeting processes satisfied their needs. As the Program grows, planning and operating informally is not adequate. Taking a long term strategic perspective will ensure that all options and their impacts are being considered and avoid costly adjustments and disruptions to the organizations' operating environment.
5.1.7 Annually, IGX should undertake a formal documented review of the original priorities and objectives established for the GPP, taking into account the changing global and Canadian context in which the GPP operates. The review should be shared with and agreed to by the senior management of FAC.
5.1.8 IGX should adopt a results management approach by operationalizing its RMAF and RBAF. This would include a comprehensive schedule of planning, programming, monitoring and reporting for all its activities. Responsibility for delivering each component of the schedule should be stated and all staff informed.
5.1.9 Annually, when preparing the Corporate Business Plan, IGX should conduct a more in-depth analysis of the results it intends to achieve and the resources it will require to achieve these results over the ensuing three years. This analysis should include:
The results of this review should be formally documented and shared with FAC senior management. It should also serve as the basis to measure IGX's performance against planned results.
5.1.10 A detailed budget and work plan should be prepared prior to the commencement of each fiscal year. This plan should be monitored on a quarterly basis and the appropriate adjustments made. The plan and review should be results based.
5.1.7 Agreed. The priority review was completed in February and the conclusions have been approved by the ADM. An action memo will be submitted to the Minister for approval of the refined priorities by end of FY 05/06.
5.1.8 Agreed. IGX already follows a results based management approach across its portfolio of projects and maintains comprehensive risk registries based on the RBAF/RMAF undertakings and these are integrated into the two reports submitted to Treasury Board each year. The format of the next TB report will be modified to provide a clearer representation of the planning/programming cycles and related reporting. Schedules are also in place for program planning and reporting while monitoring schedules have been well defined for submarine dismantlement, the most active stream of the Global Partnership Program. IXS, the new program support unit, now has an RBAF/RMAF specialist on staff and IGX will work with her in the first quarter of FY 06/07 to finalize the operationalization of the RBAF and RMAF and evaluate refinements that would better reflect current realities.
5.1.9 Being done. IGX has completed and tabled its first two formal reports to TBS. In addition, the program's first annual report to Parliament was tabled in November. These processes include an analysis of progress/achievements, together with an analysis of the broader strategic considerations that have had an impact on the program.
As indicated in 5.1.7, the review of program priorities and objectives was completed and approved by the ADM in February. The strategic action plan for IGX will flow out of this process and will include a 3-year analysis of results GPP intends to achieve in conjunction with the annual FAC Business planning exercise. The plan will be completed by the end of FY 05/06. These undertakings have also been identified as a key commitment in PMAs for this FY. The results of this review will be shared with senior management and will be reflected in our statutory commitments to the Parliament and TB.
5.1.10 Agreed. A detailed budget has been developed for FY 2005/06 and will be done each year. Budgets are currently allocated by priority area according to profiles approved by TBS and a financial coding structure has been developed within the existing financial system. This uses Work Breakdown Structure (WBS) codes to support the allocation of budgets and expenditures to individual programs and projects and allows for detailed tracking of costs and financial reporting. ARLU provides original budget information based on existing TBS authorities, which are known at the beginning of each fiscal year.
When IXS is fully staffed by June of 2006/07, templates and processes will be developed for the creation of more detailed budgets and work plans. In addition, as part of the plan to introduce a more automated non-financial project management system during FY 2006/07, the WBS financial coding structure will be refined to align with the new system and to provide more robust financial reporting.
Current-year financial plans (budgets/actuals/forecasts) are already reviewed and updated six times per year in conjunction with quarterly reviews, the ARLU, Supplementary Estimates A and reprofiling exercise. Work plan monitoring will be integrated into this process starting in FY 2006/07.
Individual work plans are currently under development as part of the PMP process. This will be completed by the end of the first quarter of fiscal year 2006/07.
5.2.1 The development and subsequent approval by Treasury Board of a Project Management Framework (PMF) provides a sound guide for the development, approval and monitoring of IGX projects. All project management staff have indicated that they use the PMF. For the original four projects that were developed prior to the PMF being developed, the spirit of the PMF has been applied. The documentation for more recent projects clearly demonstrates that the PMF is being used and applied. The PMF could be written in a more user friendly fashion. It contains explanations and long narratives on project management theory which are not necessary. All staff interviewed have indicated that when applied with judgment, the PMF is a useful tool.
5.2.2 Based on a review of key project documentation and discussions with the project management staff, sound project management practices are being followed. However, the necessary project management support tools such as a standardized project management system (e.g. for tracking project results, common milestone tracking system), information management systems (e.g. records management, by ensuring that the existing paper based filing system is being fully utilized and adopting the new electronic systems as they mature), standardized formats, and other project support mechanisms, that are required to effectively manage complex high risk projects have not been developed. As a result, staff time is wasted looking for information, maintaining project records on stand alone spread sheets and developing their own personalized systems.
5.2.3 The project management staff's knowledge and competencies associated with operating in a project management environment have significantly improved over the past year. Program management training has been provided to staff; however, the project management training recommended in the Advise and Assist Report (June 2004) has not occurred and most staff still feel there is a need for this type of training.
5.2.4 The recipient organizations (EBRD, UK-MOD, IAEA, ISTC) received advances at the end of fiscal 2003-04 which at the time of the audit remained wholly or partially unused. It is recognized that TB approval was received to make the payments; however, the reasons for making such advances in the amounts that they were made are not adequately documented. IGX has carefully tracked these transactions and interest is being credited to Canada, when called for, in the accounts of the recipient organizations. Subsequent funding has not been provided or it has been adjusted taking into account the unused portion of previous advances. The amounts of the advances and the portions which remain unused as of June 2005 are as follows:
|Organization||Amount Of Advance||Amount Unused|
5.2.5 There have been delays in the projects. The reasons for these delays range from difficulties encountered with agreeing to the Russian sub-contractor for the UK-MOD project, delays in planning and approval of the strategies for the IAEA and EBRD project portfolios and start-up of projects at the ISTC. All of the G7 partners have experienced similar delays in delivering their projects in the Russian context. The Senior Project Managers (SPM's) have been carefully monitoring these situations and have taken appropriate action. The reasons for the delays are well documented and explained. In terms of achieving the stated objectives for the projects the planned evaluations of the Program and projects will provide the information required to make a judgement on the results being achieved.
5.2.6 In concert with any Departmental efforts that may take place, IGX should take action to develop the necessary project management tools such as a project management system, information management system and standardized formats.
5.2.7 Immediate action should be taken to develop and deliver a project management training program geared to the specific needs of project management staff in IGX and the new programs.
5.2.8 IGX should document the reasons and justifications for the year end advances to the recipient organizations. No further advances should be provided to these organizations until such time as the advances that have been made are fully utilized. IGX should continue to ensure that Canada receives appropriate interest on the unused portion of the advances, when called for.
5.2.6 Agreed. IGX will review what can be addressed at the Bureau level and in cooperation with IXS, will develop additional templates by end of the first quarter of FY 05/06. At the same time, IGX will take action, in concert with relevant bureaux of the Department, to develop project management tools that will apply across other programs. The time line for completing this exercise is beyond IGX's control.
5.2.7 Agreed. IGX has already organized a project management training program geared to the specific needs of project staff. Initial training sessions were completed in January 2006 and a follow up project management course was completed in March.
5.2.8 All payments were authorized by Treasury Board and were required under the legal framework established by agreements with the recipient organizations. IGX tracks all contributions and liaises with the recipient on an ongoing basis. IGX is satisfied that the delays in converting Canadian contributions into project implementation in Russia or the FSU are fully justifiable. In addition, IGX has taken action to reduce the actual flow of subsequent funding to reflect usage wherever permitted under our legal agreements. For instance, funding to the ISTC has already been reduced to reflect slower than expected expenditures on project implementation and supplemental programs.
To the maximum degree possible, IGX intends to adhere to the recommendation to withhold further advances to organizations until such time as the advances have been fully utilised. However, IFM notes that there are cases where universal adherence to this principle would place Canada in a position of noncompliance with its legal obligations under the agreements, arrangements or understandings. In addition, there will be instances where IGX has to transfer funds to an organization or country such as the UK that is undertaking several projects on our behalf using separate accounts. This also applies to the ISTC which requires an advance for its administration and operating budget irrespective of the status of funding provided for project implementation and to the EBRD which is managing several distinct international programs involving Canadian funding.
5.3.1 At the time of the audit, IGX's planned staff complement for 2005-06 was thirty-three. Seventeen of the staff are directly involved in project delivery, there are three management positions and the remaining thirteen provide support services such as legal advice, financial management advice, policy and communications support, and administrative support. Five of the staff are rotational and four are on secondment from other departments or agencies.
5.3.2 Human Resource Planning (requirements, staffing, training, succession) in IGX is informal. The Audit Team found that IGX has not completed a formal assessment of its human resource needs nor does it have a formal succession plan. The recent restructuring and decision to hire additional resources were carried out on the basis of discussions amongst the senior management and the managers and their staff. Options other than the one implemented were considered; however, there is no documentation demonstrating what these options were. It is recognized that succession planning in IGX will be difficult and the possible loss of staff is high risk for the following reasons: young mobile staff; possibility of significant amounts of maternity leave; specialized knowledge requirements; language skills; french training; rotational staff; attracting rotational staff to a project management environment; and retirements. All of these could significantly impact the long term capacity of IGX to deliver the Program.
5.3.3 Generally, project staff have a clear understanding of their immediate roles, responsibilities and authorities. However, staff do not have a clear understanding of the roles of staff assigned to support areas such as communications, finance, contract and project support. While the audit was underway, steps have been taken to ensure that there is a clear understanding in the Bureau of the role of these support specialists. The interface and the boundaries between the project staff and functional staff need to be better defined.
5.3.4 The Advise and Assist Report of June 2004 recommended that "IGX should consider, in collaboration with IAM, creating a centralized support service unit under the leadership of a Director reporting to the Senior Coordinator." In response to this recommendation, IGX has created what can best be described as a "virtual" Project Management Office, consisting of the Legal Advisor, Financial Management Advisor, Project Management Advisor and Special Projects Officer. Each of these individuals has a different reporting relationship in the organization structure. With respect to the Legal Advisor, who reports directly to the DG, this arrangement has worked well. Such is not the case for the FMA and PMA. There is misunderstanding of their roles by some of the staff and the FMA's reporting relationship through a Director has caused difficulties. In April, the reporting relationship of the FMA was changed so that the reporting relationship is direct to the DG. Overall, it cannot be said that there is effective coordination and delivery of the required support services.
5.3.5 Many of the strategic management tasks (planning, risk management, reporting) need to be enhanced to adequately support the new program and project execution responsibilities that IGX is undertaking.
5.3.6 All of the IGX staff interviewed indicated that they work long hours, though none of them felt this was a problem. However; having staff continue to work long hours over a long period of time could result in an eventual loss of productivity due to extreme fatigue and sick leave. The requirements to work long hours is not necessarily associated with the requirements of the PMF but rather other demands which are placed on the project management staff.
5.3.7 With the employment of PM 4 Program Officers in all of the project areas there is back-up in place for all of the Senior Project Managers.
5.3.8 The staff of IGX received only informal feedback on their performance. They have not received official performance appraisals.
5.3.9 A comprehensive formal Human Resource plan should be developed and IGX should seek the assistance of the Human Resources Branch in developing the plan.
5.3.10 In the context of the Planning and Budgeting process, IGX should undertake a comprehensive review of the number and type of human resources it will require to efficiently and effectively deliver the planned program.
5.3.11 To ensure all staff receive formal feedback on their performance, IGX Management should fully implement the new FAC Performance Management Program (PMP).
5.3.9 Work is well underway in the development of a formal HR action plan with a particular emphasis on succession planning and skills development. Succession plans for senior positions have been discussed with HR, submitted to senior management and are being implemented. Consultations have also been held with HR and management has made significant progress in moving forward with both the PMP process and the development of learning plans to underpin Project Management skills. In this regard, most program staff have undertaken the procurement and contracting for services course as well as PMP and project management training. To be completed by end of FY05/06.
5.3.10 The number and type of human resources we require will be incorporated into the HR action plan, strategic action plan and will be reflected in IGX contributions to the Department's business plans.
5.3.11 Underway. Most managers have completed PMP training. See also 5.3.9.
5.4.1 The RBAF for the Phase I TB submission contained the following statement: "The resultant risk profile will be regularly reviewed, in the light of implementation experience, and used to revisit program monitoring, recipient audit, internal audit and evaluation requirements." To date there has been no review of the risk and mitigation strategies presented in the RBAF at the Program level. To be an effective tool in the management and control of the program, the identified risks and the associated mitigation strategies must be formally reviewed and updated on a regular basis (at least annually).
5.4.2 At the project level, risk registries have been established and there are regular reviews of the risks and mitigation strategies. However, at the Program level this exercise has not been completed. This is the next logical step after project review to escalate the results to the program level.
5.4.3 IGX should immediately undertake a formal review to update the Program level risk and mitigation strategies identified in the RBAF for the Phase 1 TB submission. Subsequently there should be, as a minimum, an annual review and update.
5.4.3 Agreed. While the most serious risks facing the program occur at the project level and are formally reviewed and updated regularly, a similar process is needed at the program level. The first formal annual review of such risks as part of an ongoing process will be completed by end of FY 05/06.
5.5.1 In the approved RBAF for the Program IGX identified "communications with members of Parliament, Cabinet, central agencies, and within the Federal Government generally" and "Communications and outreach programming through NGOs, academia, business and the general public will continue to be the most effective strategy to keep the WMD issue in the forefront of the public consciousness" as key risk mitigation strategies. The forthcoming annual report to Parliament, which is in the final stage of preparation, will be a key document in addressing the identified risks.
5.5.2 A draft communication strategy has been prepared; however, at this point there is no approved communication strategy for the GPP and IGX. It is understood that a communication specialist is being hired and that one of the first tasks for this individual will be to prepare and have approved a communication strategy.
5.5.3 Given the importance of continuing support for the Program, IGX should ensure that as soon as possible after hiring the communication specialist a formal strategy is prepared and approved. The strategy should be approved by the ADM and, once such approval is given, shared with all of the staff in IGX. An implementation plan should form an integral part of the strategy.
5.5.3 A communications strategist has been engaged. Work on updating the communications strategy prepared in conjunction with Phase I and II submissions is completed and will be submitted to the ADM prior to April 1, 2006. A Client Services and Outreach specialist has also been engaged and work is also underway on an outreach strategy. Both activities will be carefully coordinated.
5.6.1 As described in the approved Program RBAF the Global Partnership Advisory Group (GPAG) plays an important role in the overall governance structure of the GPP. The GPAG is composed of senior officials from other government departments and agencies (OGDs). The role of GPAG is to provide broad policy advice on the direction and implementation of the GPP and ensure that relevant Canadian interests are reflected in the GPP's delivery. Until recently, the committee had not been as effective a governance mechanism as had been anticipated. The DG IGX recently undertook a number of discussions and visits with his colleagues both in FAC and other interested government departments with a view to ensuring that the original role envisaged for the group is still valid and that it will play an effective and value added role in the governance of the GPP.
5.6.2 It is important that IGX has built effective strategic relationships with its Russian counterparts, partners, central agencies and OGD's. Based on the recent visits conducted by members of the Audit and Evaluation Divisions to the ISTC, EBRD, IAEA, UK-MOD, SHIPYARD and Bechtel, IGX has established effective working relationships with its delivery partners and it is clear they understand FAC's role and responsibilities.
5.6.3 Once the DG IGX has completed discussions with colleagues in FAC and OGDs, new terms of reference for the GPAG should be developed and agreed to by the members.
5.6.3 Agreed. The new terms of reference were approved by the Global Partnership Advisory Group in February 2006.
5.7.1 In the Phase I and II Treasury Board submissions a number of commitments were made in the areas of reporting, evaluation and audit. A review of these commitments was undertaken in order to document the commitments and to determine exactly what the capacity of FAC is to deliver on these commitments.
5.7.2 The following summarizes the findings of the review in each of the critical areas identified in the Treasury Board submissions:
5.7.3 In order to fully comply with the commitments set forth in the Treasury Board submissions and the related RMAF and RBAF it is recommended that IGX formally review the original performance indicators to ensure their relevance. As well, IGX should ensure someone is assigned the overall responsibility for data collection, coordination and reporting.
5.7.3 Agreed. In cooperation with IXS, IGX will formally review the original performance indicators by end of the first quarter 06/07 and ensure that one officer has been assigned overall responsibility.
5.8.1 In June 2004 an Advise and Assist Review Report of IGX was issued. The report contained twelve recommendations that IGX agreed to implement. Three recommendations have been fully implemented, two partially implemented and seven have not been implemented. When responding to these recommendations IGX indicated that all of the recommendations, except for one, would be implemented, either immediately or in the very near future.
5.8.2 The two recommendations which have been partially implemented are as follows:
1. IGX should take immediate steps to implement the formal document management system as approved by the Senior Coordinator.
The filing system has been established; however, much of the material has not been placed on the files.
2. IGX should consider, in collaboration with IAM, creating a centralized support service unit under the leadership of a Director reporting to the Senior Coordinator.
5.8.3 The seven recommendations which have not been implemented are as follows:
1. Accelerate the development and implementation of the necessary project management tools.
Other than developing a suggested format for the preparation of concept papers there has been no progress in developing standardized project management tools.
2. Take immediate action to develop and deliver a project management training program geared to the specific needs of the project management staff.
The training program geared to the specific needs of the project management staff has not been developed. Some staff have attended standardized project management training.
3. A follow-up regime should be put in place, in collaboration with ZID, to ensure that the commitments made in both the RMAF and RBAF are met.
No follow-up regime was put in place.
4. The approved RMAF and RBAF should be formally reviewed and updated immediately and at least annually thereafter.
No action taken. Was to have been completed in the summer 2004.
5. IGX should ensure that the required data is available and assign the overall responsibility for data collection, coordination and reporting.
No action taken. Was to have been completed by late summer 2004.
6. IGX, in collaboration with ZID, should review the performance indicators listed in the RMAF to ensure that they are relevant to the successful management of the GPP.
No action taken. Was to have been completed by late summer 2004.
7. IGX should develop an action plan to deal with staff absences and turnovers in the foreseeable future.
IGX feels that it has done what can be done at the Bureau level.
Some of the recommendations outlined above have been restated in this audit report.
5.8.4 IGX should, in the context of this report, take action to fully implement the nine recommendations outlined above.
5.8.4 Noted. These recommendations have been subsumed in the recommendations resulting from this audit.
5.9.1 The responsibility for information systems and document management is both a corporate and IGX responsibility. At the time the audit field work was taking place, there was no formal strategy for the management of information and electronic systems in IGX. Software purchases were made on an ad hoc basis, and there were four systems being used to maintain files (CATS, INFO BANK, IGX Central Registry, Senior Project Manager's Office). Because there was no formal strategy for the management of information and electronic systems in IGX, the retrieval of some key documents was a challenging and time-consuming activity. Since the completion of the audit fieldwork, IGX has taken the initial steps required to develop a formal strategy that addresses information systems and document management.
5.9.2 In cooperation and in coordination with any activities or reviews that may take place at the Corporate level, IGX should undertake a complete analysis of its information management and systems requirements. The results of this review should be shared with the new programs and common solutions sought. Until such time as the review is completed, new software or equipment purchases should be restricted. A longer term approach, based on a comprehensive plan, should be taken in addressing the bureaus information management and systems requirements.
5.9.2 Agreed. IXS was established by IFM on 06 September 2005 to deal with a number of issues including information management and systems requirements for the three program delivery bureaus of the I Branch. IXS is presently working with SXD and other relevant bureaus of the Department to ensure the development of an adequate information architecture. IGX is taking an active role in this endeavour. The complexity of this task and the need to ensure that the new approach is compatible with other applications within the Department, as well as some whole of government requirements, means this activity is unlikely to be completed before the end of FY 2006/07.
|Audit Objective||Audit Criteria|
|1.0 Planning and Budgeting|
|2.0 Functional Support|
|3.0 Project Management|
|4.0 Human Resource Management|
|5.0 Risk Mitigation|
5.1. A review has been conducted of the risk and mitigation strategies in the RBAF and RMAF.
|5.1.1 There is documentary evidence that the risk and mitigation strategies in the RMAF and RBAF have been reviewed and updated on a regular basis (i.e. every six months).|
|7.0 Treasury Board Commitments|
7.1 IGX, ZIV and ZIE can meet the commitments contained in the Treasury Board Submissions and the related RMAF and RBAF's.
|7.1.1 Documentation exists demonstrating that the commitments specified in the Phase I and II RMAF and RBAF's have been reviewed and updated as appropriate and differences reconciled.|
|8.0 Advise and Assist Review|
8.1. The recommendations agreed to in the Advise and Assist Review have been implemented.
|8.1.1 IGX Management should have implemented their Management Action Plan in response to the recommendations contained in the Advise and Assist Report within the specified time frames.|