Foreign Affairs, Trade and Development Canada
Symbol of the Government of Canada

Foreign Affairs, Trade and Development Canada

international.gc.ca

Audit of the Secretariat of the Francophonie Summit

(March 2009)

(PDF Version, 126 KB) *

Executive Summary

During the winter of 2005–2006, the federal government and the Government of Quebec concluded an agreement on the Francophonie Summit to be held in Quebec City in October 2008. This agreement established a Steering Committee composed of two representatives of Canada, one acting as its chair; two representatives of Quebec, one acting as its vice–chair; and one representative of the Government of New Brunswick. The members of the Steering Committee were appointed by their respective governments.

The Organization Secretariat of the Francophonie Summit (hereafter "the Secretariat") was established to ensure that all activities leading up to the Summit, as well as the Summit itself, were appropriately supervised and carried out with no significant problems. All Secretariat employees were considered to be DFAIT employees. The Director General of the Summit Secretariat was also a member of the Steering Committee.

An internal audit of the Secretariat was required by the Treasury Board of Canada. The objectives of this audit were to ensure that the Secretariat had put in place the controls necessary for the appropriate operation of the organization and that those controls were working correctly. The audit was conducted between May 7 and August 15, 2008.

The details of the audit mandate, including the objectives, scope, methodology and criteria used for the audit, are presented in Appendix A – About the Audit.

The audit team was able to obtain reasonable assurance that the Secretariat put a control framework in place necessary for the effective operation of the organization. The Secretariat was successful in fulfilling their mandate as evidenced by the success of the Summit which concluded prior to completing this report.

The following are positive examples of the Secretariat's management framework:

  • The Secretariat had a well structured team of directors who were motivated to succeed in making the 2008 Francophonie Summit in Quebec City a resounding success.
  • The Secretariat put in place the controls necessary to ensure sound management of its activities.
  • The operational directorates developed and used management tools appropriate to the strategic and operational levels, in order to ensure appropriate and effective planning and monitoring of activities leading to the organization of the Summit.
  • The Secretariat used resources in compliance with the policies and directives of the Treasury Board of Canada and the Financial Administration Act and observed the terms and conditions of the Memorandum of Understanding signed between the governments of Canada, Quebec and New Brunswick.

Issues for Improvement

The Secretariat did not develop an integrated risk management framework at the time it was created. Such a framework would have served to provide assurance that risks were identified, managed and communicated within the Secretariat as a whole, in order for the organization to meet its overall objectives.

The following information was not included in the report on financial performance presented to the Steering Committee:

  • Expenses subject to commitment ("soft commitments") in order to show the adjusted available balance; and
  • Budget and subsequent amendments so that the reader is able to identify differences between budgeted costs and actual costs incurred.

A table of budgetary risks was not included in the financial performance report presented to the Steering Committee to provide an overall picture of the financial risks, operational costs and possible final balance. At the time of the audit, as of July 31, 2008, the probable deficit in the event that all potential risks materialized was estimated at $0.6 million.

The file of documentary evidence concerning the assessment and the decision–making process for the allocation of contributions related to the special projects financial assistance program was incomplete. Therefore it is not possible to verify that the process was conducted in accordance with Government of Canada policies and directives.

The report outlines improvements in the areas of budgeting, contracting, documenting the process of allocating contributions to special projects and greening. These improvements are written as lessons learned and cover the seven components of the audit. These lessons learned should be taken into consideration in order to improve the controls necessary for the operation of other organizations similar to the Secretariat. These lessons may also be of interest to other Bureaux in the Department that manage summits and conferences.

Main Lessons Learned

  1. To improve the governance of the Secretariat, risk management should be introduced in the first stages following the creation of an organization secretariat.
  2. To provide the Steering Committee with complete and value–added financial information on the situation of the Secretariat, it is recommended that:
    • all financial information be integrated to provide a consolidated picture of financial risks, operational costs and available balance; and
    • all relevant information (such as the initial budget and subsequent amendments, breakdown of expenditures by activity and expenditures subject to commitment, or "soft commitments") be presented in the financial performance report for the Summit.
  3. To provide Secretariat staff with appropriate control tools and to demonstrate transparency of the process, it is recommended that the formal documentation methodology established to support the evaluations and decisions made about the allocation of contributions to special projects be applied at all levels and maintained on file.

Management has reviewed this draft report and is in agreement with the observations and lessons learned. The actions recommended by management to address the lessons learned can be found in Appendix B. The complete list of lessons learned is also in Appendix B.

Top of page

1.0 Introduction

1.1 Background

The 12th Summit of Heads of State and Government of Countries Using French as a Common Language (hereafter "the Summit") was held in Quebec City in October 2008, bringing together many heads of state and their delegations. This important meeting demanded considerable organizational capability, and it was to that end that the Organization Secretariat of the Francophonie Summit – Quebec City 2008 (hereafter the Secretariat) was formed. This temporary organization, established jointly by the governments of Canada, Quebec and New Brunswick, was to ensure that all activities leading up to the Summit as well as the Summit itself were appropriately managed and conducted with no unforeseen events.

The cooperation between the three governments was structured by the following two agreements:

  • Agreement between the Government of Canada and the Government of Quebec on the 2008 Quebec City Summit of La Francophonie, reached in January 2006; and
  • Memorandum of understanding between the governments of Canada, Quebec and New Brunswick on the preparation, organization, conduct and administrative and financial details of the 12th Summit of Heads of State and Government of Countries Using French as a Common Language signed in June 2007.

The Secretariat was given an operating budget of $32.75 million. It was agreed that the Department of Foreign Affairs and International Trade of Canada (DFAIT) and the Government of Quebec would each contribute $16 million and the Government of New Brunswick $0.75 million.

As stipulated in the memorandums of understanding, the Secretariat was headed by an Organizing Committee (hereafter "the Steering Committee" as commonly referred to by the Secretariat) that was composed of two representatives of Canada, two representatives of Quebec, and one representative of New Brunswick. This committee was chaired by an official representative of the Prime Minister of Canada and vice–chaired by the assistant deputy minister for Bilateral Affairs and La Francophonie at the Quebec Ministry of International Relations (MRI). Under the memorandums of understanding, it was agreed that the general mandate of the Steering Committee would be to prepare for the Summit and to ensure it ran smoothly.

The mandate of the Secretariat was to implement and coordinate the preparation, organization and conduct of the Summit and the peripheral events connected to the Summit. The senior managers of the Secretariat were appointed by their respective governments of Canada, Quebec and New Brunswick, however for the conduct of business, the director general of the Secretariat reported directly to the Steering Committee, of which he was a member. The memorandums of understanding governing the Secretariat's operations, stipulated that they must comply with the policies and directives issued by the Treasury Board of Canada. In this regard, each government designated a representative to ensure that the administrative and financial terms and conditions of the Secretariat in relation to legal and financial obligations were applied.

1.2 Key Issues Impacting on the Secretariat

The governments of Canada, Quebec and New Brunswick financed the 12th Francophonie Summit. One of the main challenges the Secretariat faced was the complexity of the decision–making process of the tripartite Steering Committee. This committee, that included representatives of the different levels of government in order to reflect the structure of the public assistance granted, served as a mid–level decision making body between the Secretariat management and the political authorities. Consequently, there was a significant risk of the Secretariat facing complications in the operational planning of the activities necessary for the smooth operation of the Summit.

The following are examples of activities that awaited approval by the Steering Committee that had a significant impact on the Secretariat's operations:

  • the official Summit program – submitted in June 2007, still under discussion in July 2008;
  • the program of the Ministerial Conference of La Francophonie (CMF) – submitted in October 2007 published in April 2008;
  • the greening of the Summit, method of funding carbon credits, and target objectives (carbon zero trend or carbon neutral event) – submitted in November 2007; the calculation of carbon credits had not been completed as of December 2008;
  • the individual approval of each component of the communication plan prior to implementation – submitted in November 2007, approved May 2008;
  • the assembly of promotional bags for delegates – submitted in January 2008, still under discussion in July 2008;
  • the availability of liaison officers from the three governments – submitted in April 2008;
  • the imagery for ceremonial rooms and media galleries – submitted in June 2008; and
  • the language of communication of official documents – submitted in March 2008 (approved in July 2008 but in question until August 2008).

The following opportunities were not realized because of delays in the decision–making process:

  • The approval of the promotional video of the Summit was delayed until the spring 2008. As a result, the Secretariat was not able to produce the DVDs for the Journée de la Francophonie on March 20, 2007, and ship them on time to foreign embassies and delegations.
  • The absence of a logo until January 31, 2008, submitted for approval in October 2008, caused lost opportunities for visibility, notably at the Ministerial Conference of La Francophone (CMF) in Vientiane, Laos, in November 2007.
  • The launch of the special projects financial assistance program was delayed, causing the amount of $100,000 from the 2007–2008 fiscal year to lapse from the Secretariat's budget.

In 2003, the decision that Quebec City would be the host of the 12th Francophonie Summit was taken. In 2006, before the Secretariat was set up, contracts were signed with Quebec City hotels to reserve accommodation and restaurant services. This was done based on the experience of previous summits held in Quebec City in 1987 and 2001, taking into account the high number of delegates at those summits. It was essential to reserve a large number of hotel rooms, as Quebec City would also be celebrating its 400th anniversary during the same period and many visitors and events were anticipated. DFAIT was unable to make the necessary reservations at the time and the Government of Quebec agreed to make the reservations and pay the required deposits.

In order to meet the requirements of the Royal Canadian Mounted Police (RCMP)1 with respect to hotel security, the contracts for the hotel rooms had to be amended both before and after the creation of the Secretariat. For security reasons, the RCMP required that certain hotels be fully booked and selected hotels had to be accessible only to persons accredited for the Summit during a specified period.

1.3 Overall Assessment of Controls in Place

The Secretariat met their mandate by:

  • putting in place a set of controls that could provide for sound management of the activities;
  • developing and using appropriate management tools for the operational directorates;
  • complying with the terms and conditions agreed to in the memorandums of understanding, especially those of an administrative and financial nature; and
  • utilizing resources in compliance with the policies and directives of the Treasury Board Secretariat of Canada and the Financial Administration Act.

Although the overall assessment of the Secretariat identified no significant items that might cast doubt on the Secretariat's control framework, it identified certain findings that should be taken into account in the interest of improved management practices and performance. These findings based on the audit objectives, are presented in detail in the next section of this report.

Since the audit of the Secretariat ended on August 15, 2008, just over two months before the Summit was held, the recommendations have been approached from the perspective of lessons learned which may be of use in future summits or other similar events.

Top of page

2.0 Observations and Lessons Learned

2.1 Governance and Strategic, Operational and Contingency Plans

The Secretariat had a well structured team of directors who were motivated to succeed in making the 2008 Francophonie Summit in Quebec City a resounding success. The expertise and experience of the individuals selected to fill key positions within the Secretariat were major assets for the functioning of the operating directorates. The Secretariat management put in place the strategic and operational tools required to ensure appropriate and effective planning and monitoring of the activities involved in organizing the Summit. However, the following situations have been noted as requiring improvement.

Integrated Risk Management Framework

At the commencement of the Secretariat operation, an integrated risk management framework was not developed to ensure that risks were identified, managed and communicated throughout the Secretariat in the interest of meeting its overall objectives. The operational directors identified that risks were managed and communicated through the Secretariat management committee and other ad hoc meetings. As a result, use was made of the personal experience of every director. However, given the absence of a risk analysis at the outset, the operational directors stated that risk was managed as incidents arose.

The early design and implementation of an integrated risk management framework for all members of management would have had the benefit of providing the following:

  • a uniform risk management process between each operating directorate;
  • risk management throughout the Secretariat and all the directorates sharing the same vision and benchmarks;
  • all necessary controls in place to manage significant risks; and
  • possible significant risks be considered to achieve the organization's objectives communicated to the Steering Committee at the commencement of operations.

Lesson learned 2.1.1

To improve governance of the Secretariat, risk management should be introduced in the first stages following creation of an organization secretariat.

2.2 Budget

The Secretariat submitted an initial budget plan and a series of measures in order to balance the budget with the funds allocated by each government to the Steering Committee September 20, 2007 and November 9, 2007 respectively. At each Steering Committee meeting after January 2008, the Secretariat tabled a memorandum to present the budget situation, summarize the composition of expenditures and obtain any necessary spending authorities. Appended to this memorandum was a table showing the variance between budget and actual expenditures and a table of unforeseen costs that could create a deficit.

Budget Approval

The Secretariat's financial performance was rigorously monitored and questioned by the members of the Committee, as demonstrated in the Committee minutes. The Steering Committee was informed, at each one of the meetings, of the budget situation of the Secretariat and it systematically approved the budget choices that were presented.

Budgetary Oversight

Analysis of the documents related to budgetary oversight indicated the following:

  • Supplementary information was not included in the financial results table submitted to the Steering Committee to improve the presentation of the Secretariat's financial situation.
  • The forecasted expenditures subject to commitment (or "soft commitments") were not presented separately in order to show the actual available balance in the financial results report submitted to the Steering Committee.

Certain expenses presented in the budgetary risks table were more than likely to occur, and therefore should have been committed as soon as they were noted, to present a more accurate financial situation to the members of the Steering Committee. Examples include:

  • the cost of agreements with the Centre des Congrès de Québec for "green" services; and
  • compensation of overtime for the duration of the Summit, as confirmed by the operating directorates.

In order to track the budget situation, the Administration and Finance directorate issued detailed monthly financial reports to the operating divisions so that they could validate the transactions charged to their responsibility centre.

A budgetary risk table, although available, was not integrated with the report on financial results presented to the Steering Committee. Therefore, the possible deficit was not specifically shown in the report indicating the available balance.

On July 31, 2008, the financial results report showed a management reserve for contingencies in the amount of $2,803,736. The corresponding expenses were estimated at $3,254,000, which was $450,264 in excess of the budgeted management reserve. The financial results report also indicates a deficit of $185,462 related to wages and benefits. Therefore, the operating deficit could total $635,726 if all of the estimated expenses should materialize. If this situation occurred, the Secretariat would no longer have a financial margin to cover contingencies that might arise over the course of the Summit.

In order to provide future steering committees with complete and value–added financial information the following items should be taken into consideration.

Lesson learned 2.2.1

Ensure that all financial information (e.g. the initial budget and subsequent amendments, breakdown of expenditures by activity and expenditures subject to commitment, or "soft commitments" and unforeseen expenses) should be integrated into one financial results report for the activity in order to provide a consolidated picture of the financial risks, operational costs and available balance.

Lesson learned 2.2.2

Disclose all elements for which there is an acceptable basis of measurement and a reasonable estimate of the amount.2

2.3 Financial Controls

A financial authorities delegation grid for the Secretariat was available and in compliance with the Delegation of Authorities Policy and the DFAIT delegation matrix. The Secretariat employees were delegated the financial authorities appropriate to their job and responsibilities.

All financial transactions examined from the test sample proved to comply with sections 32 and 34 of the Financial Administration Act. The authority for payment under section 33 of the FAA was centralized at DFAIT headquarters in Ottawa. There was an appropriate segregation of duties within the Administration and Finance Directorate of the Secretariat regarding Sections 32 and 34 of the FAA.

The documentation supporting each transaction examined was complete and kept on file, and all purchases of depreciable physical assets were appropriately recorded and inventoried.

2.4 Contracting Process

The Secretariat utilized standing offers and competitive and non–competitive processes to purchase goods and/or services. Each process was used appropriately and in accordance with the Contracting Policy of the Treasury Board Secretariat of Canada. Minor deficiencies in the key controls of this process were found.

Three of the 12 contracts examined contained minor irregularities. In two cases, the delivery of goods or services began before the date the contract was signed. In one of the two cases, a local purchase order was initially used. However, when it was realized the monetary value of the agreement exceeded the permitted limit of $5,000, the situation was corrected and a contract was drawn up to replace the purchase order. The second case involved a contract established to hire an employee who started work two weeks before the contract was signed. In the third case, a contract was modified three months following the end date of the initial contract. The contractor had verbally agreed to deliver the additional service and began the work prior to the contract amendment. These minor observations in the contracting process were corrected by those responsible, and therefore did not present a risk to the Secretariat.

Lesson learned 2.4.1

While the Secretariat is an organization that must be able to respond promptly to the situations it encounters, to maintain the integrity of the process it is essential that all contracts appropriately reflect the date that goods and/or services are obtained and that any contract amendment be immediately supported by documentation entered in the file.

2.5 Process of Allocating Contributions to Special Projects

One of the components of the Secretariat's budget was the allocation of $1.1 million for the Special Projects Financial Assistance Program, which was intended to finance activities and events that promote La Francophonie. For purposes of management of this program, the Secretariat designed and published a reference document that defines but does not limit the nature of the program, the eligibility criteria, the evaluation criteria, the terms of funding and the application procedure.

The Special Projects Division of the Secretariat developed work tools based on models from the Francophonie Promotion Fund. The principal elements were:

  • the criteria for project eligibility and evaluation;
  • the terms of funding;
  • a grid for analysis and evaluation of special projects;3 and
  • a grid for calculating the contribution based on the project's budget estimates.

A special projects evaluation sub–committee was created, composed of one representative of DFAIT, one representative of the Ministry of International Relations (MRI) and staff of the Special Projects Division. The main role of this advisory committee was to facilitate decision making at the Steering Committee level by making recommendations based on the evaluation produced by the Special Projects Division.

Of the 31 files supporting applications for financial assistance reviewed, only one contained a project analysis and evaluation grid. The Secretariat's Special Projects Division confirmed that for most of the files, no project analysis and evaluation grid was formally completed. The only document that supported a detailed evaluation of the contributions allocation process was a summary sheet that provided the following:

  • general information about the Project;
  • information on the financing package indicating all the contributors;
  • a brief but not complete report on the evaluation and recommendations of the Special Projects Division and the evaluation sub–committee; and
  • the decision of the Steering Committee.

This document was not sufficient to corroborate the complete evaluation process and the recommendations made to the Steering Committee, as it did not contain a detailed evaluation of the project based on the criteria applicable to the Special Projects Financial Assistance program.

Employees of the Special Projects Division indicated that discussions about the detailed evaluations of the projects were held regularly in the division and with members of the evaluation sub–committee, and that these oral discussions concerned the program evaluation criteria. However, it was not possible to corroborate this with documentary evidence.

The minutes of Steering Committee meetings recounted the Committee's decisions with respect to the special projects submitted. However, the justifications for those decisions were not formally documented in the project files. Documentation for the rationale of all decisions is fundamental, particularly when a decision is counter to the recommendations submitted. The Steering Committee's decision was contrary to the advice issued by the evaluation sub–committee in 3 of the 31 projects examined. Two of the three projects had been recommended for approval by the evaluation sub–committee but denied by the Steering Committee; in the other case, the file was approved by the Steering Committee despite a negative recommendation. These decisions were taken in order to promote fairness in granting financial assistance.

It is not possible to provide reasonable assurance that the process of allocating contributions to special projects was conducted in accordance with acceptable government guidelines regarding grants and contributions. The absence of documented files increases the risk that corporate records, information and knowledge will not be available to support informed decision making and reporting.

In order to provide the Secretariat staff with appropriate assessment tools and guarantee the transparency of processes, we recommend that the following measures be introduced:

Lesson learned 2.5.1

Use formal documentation, at all approval levels that supports the evaluation and decisions made regarding the allocation of contributions to special projects.

2.6 Greening

Background

At the Francophonie Summit in 2004 in Ouagadougou, Burkina Faso, the Francophonie community identified a number of strategic initiatives, including that of developing cooperation for sustainable development and solidarity. La Francophonie's commitment to act in favour of sustainable development thus supported the Secretariat's initiative of setting up a greening program for its activities and for the planning and holding of the 2008 Summit. However, despite the will of all of the Summit stakeholders to produce a green event, there was no operating expenditure allocated for the Greening component in the initial budget of the Secretariat.

Observations

In keeping with the Steering Committee's will for a Greening component to the Summit, an advisory committee was formed within the Secretariat to assist the Steering Committee in selecting the elements of a greening program. This committee submitted various greening scenarios to the Steering Committee and the objective of "zero tolerance waste"4 was approved. At the time of the audit, the Secretariat was still awaiting a decision on the offsetting of greenhouse gases generated by the Summit for the zero carbon component. The amount of $325,000 was identified as the probable cost of the greening program; this was included in the table of budgetary risks.

The Secretariat put various work tools in place to define and structure the greening activities, whose principal elements were:

  • the definition of terms of reference for the advisory committee;
  • the establishment of a greening program outlining the actions to be taken and the persons responsible for follow–up;
  • the development of a communication plan for the greening program;
  • the analysis of the impacts of greening on the Secretariat's budget; and
  • the definition of environmental clauses to be included in contracts signed by the Secretariat.

The unit responsible for greening also produced an environmental policy proposal designed to describe the alignment, objectives and means for structuring the greening component. However this policy was not endorsed by the advisory committee or released to Secretariat employees. As a result, the Secretariat had no approved guiding principles that formally set forth the Secretariat's expectations with respect to greening and the behaviours to be adopted under the circumstances. This situation increases the risk that integration of the greening component in the Secretariat's routine and operational activities would be more complex and sensitive.

The impact of the greening component on the Summit cannot be evaluated until the end of the Summit, when qualitative and quantitative indicators can be compiled and analyzed.

Lesson learned 2.6.1

A greening component should be formally identified in the operating budget.

To provide the Secretariat with appropriate monitoring tools, the following measures should be considered.

Lesson learned 2.6.2

Have an approved environmental policy for the Secretariat at the start of operational activities in order to structure the management of those activities and achieve the targeted objectives.

Lesson learned 2.6.3

Adopt a zero tolerance objective, establish indicators and essential targets, in order to be able to conduct a variance analysis and report the results based on a measurement framework.

2.7 External Communications

To structure the external communication process, the Secretariat had developed the following tools:

  • a media relations policy;
  • a marketing policy for the Special Projects Program;
  • a communication plan;
  • an implementation plan outlining the activities to carry–out the communication plan;
  • a strategic schedule; and
  • a framework document on the communication roles and responsibilities of the stakeholders in the organization of the Summit.

The Secretariat created a Communications Coordination Committee (COCOCOM) to act as an ongoing forum for discussion and information sharing on communication initiatives between the governments and the Secretariat and to ensure that communication complemented the activities surrounding the Summit.

The Communications Division developed relevant strategies and activities to help achieve the objectives defined in the communication plan. However certain components of that plan had not been implemented and were awaiting approval by the Steering Committee at the time of the audit. Given the high visibility of the Summit, the effectiveness of this plan could not be evaluated until all the communication activities were implemented and the appropriate statistics compiled and analyzed.

2.8 Issues Regarding Hotel Reservations

As stated in the Introduction, hotel contracts were signed prior to the creation of the Secretariat in order to ensure that a sufficient number of rooms were secured for the Summit. The following problems with respect to the hotel reservation contracts were encountered:

  • The initial contracts were not awarded in compliance with the policies of the Treasury Board Secretariat of Canada and the Memorandum of Understanding. The Memorandum of Understanding was published subsequent to the signing of the hotel contracts.
  • The number of person–nights reserved and certain specific clauses in the initial contracts concerning restaurant services were not consistent with the actual requirements of the Secretariat, which were fewer than the projected requirements. As the Secretariat was not responsible for security at the Summit, the security requirements of the hotels had not been taken into consideration.

As of June 27, 2008, the Secretariat estimated that 6,224 surplus person–nights might arise from the signing of the hotel contracts, for an estimated budget impact of $1,894,000. The security requirements of the RCMP did not allow for the surplus person–nights at certain hotels to be transferred to other, unaccredited clients.

The Secretariat implemented the following action and initiatives to normalize this situation:

  • analysed the hotel contracts and obtained an opinion from Legal Services at Public Works and Government Services Canada (PWGSC);
  • analysed the updated requirements of the Secretariat and identified the contract clauses to be amended;
  • PWGSC prepared a purchase plan and standard contract in keeping with the requirements of the Secretariat and those of the Treasury Board Secretariat of Canada;
  • negotiations were conducted with the hotels to amend certain clauses of the contracts and reduce the contractual commitments of the Secretariat;
  • 810 rooms were transferred to the Sûreté du Québec and the contract for two hotels were transferred to the Royal Canadian Mounted Police; and
  • the resale of the surplus rooms.

In this situation, the Secretariat took measures that permitted it to discharge its responsibilities appropriately and chose an appropriate course of action under the circumstances.

Top of page

3. Conclusion

The audit team was able to obtain reasonable assurance that the Secretariat put a control framework in place necessary for the effective operation of the organization. The Secretariat was successful in fulfilling their mandate as evidenced by the success of the Summit which concluded prior to completing this report.

The report outlines improvements in the areas of budgeting, contracting, documenting the process of allocating contributions to special projects and greening. These improvements are written as lessons learned and cover the seven components of the audit. These lessons learned should be taken into consideration in order to improve the controls necessary for the operation of other organizations similar to the Secretariat.

Appendix A – About the Audit

Objectives

The Treasury Board of Canada required that an internal audit of the Secretariat be conducted midway through the period preceding the holding of the Summit and that it involve representatives of the three levels of government.

It is therefore in this context that DFAIT's Chief Audit Executive Bureau, in consultation with the internal audit and evaluation directorate of Ministère des Relations Internationales (MRI), mandated Audit Services Canada (ASC) to coordinate and participate in the internal audit of the Secretariat.

The objective of the audit is to provide reasonable assurance to the co–signers that the internal controls necessary for the sound management of the organization were in place and working properly. Another goal of this internal audit is to produce a report distributed to the Secretariat and the Steering Committee.

Scope

A preliminary review concerned with risk assessment and identification of the key issues of the Secretariat was conducted from May 7 to 9, 2008. During that period, the DFAIT and MRI auditors met with the director general, deputy directors general, directors and deputy directors of the Secretariat to conduct interviews and a risk assessment session. Based on this exercise, the joint audit team was able to target the key objectives in order to define the scope of the internal audit and the issues to be evaluated for the Summit.

Following this preliminary review, the following seven objectives were adopted as issues that should be covered during the audit:

  1. Governance
    • review of processes for making strategic and operational decisions
    • assessment of the effectiveness of communications within the Secretariat
  2. Budget
    • existence of processes for managing and controlling the budget in an environment of changing priorities
    • evaluation of the effectiveness of budgetary controls and accountability mechanisms
    • review of the existence of budget estimates established by each division
    • knowledge and awareness of the operational impact of the budget allocated on each division
  3. Strategic, operational and contingency plans
    • integration and interrelation of elements on the strategic schedules of all divisions
    • monitoring of operational plans and identification of the impact of delays on the effectiveness of performance of tasks
  4. Financial controls
    • evaluation of the appropriate application of signing authorities and compliance with the Government of Canada's Financial Administration Act
  5. Contracting process and process for allocating contributions to special projects
    • review of existing controls and compliance with procedures for contracting and for allocating contributions to special projects
    • evaluation of impartiality and use of selection criteria
  6. Greening
    • evaluation of the impact of the greening component on the Summit
    • consequences of the lack of defined parameters for the greening component on the overall operations of the Secretariat
  7. External communications
    • effectiveness of the communications plan, given the visibility of the Summit and the need to clearly explain to the public the objectives and benefits of the Summit

The scope of the audit is confined to the organization funded jointly by the three governments, and hence excludes the departments and agencies that also contribute to the success of the Summit, that is, National Defence Canada, Justice Canada, Health Canada, Public Works and Government Services Canada and the Royal Canadian Mounted Police.

Given the limited lifespan of the Secretariat, based on the holding of the Summit, the period covered by this audit started on the date it was set up, June 5, 2007, and ended on the date of completion of the field audit work, August 15, 2008.

Top of page

Methodology

The audit was conducted in accordance with the internal audit standards generally accepted in the Government of Canada. The audit also took account of the operational context of the Secretariat and the particular nature of its operations. The auditors therefore took the following steps to meet the audit objectives:

  • interviews with members of the Secretariat management and with the chair and vice–chair of the Steering Committee,
  • review and analysis of processes and of the management practices and controls in place,
  • analysis of key documents and files maintained by the Secretariat to support its activities, and
  • evaluation of control measures to ensure that the Secretariat's activities are in compliance with the memorandums of understanding and the policies and directives of the Treasury Board Secretariat of Canada, using audit tools developed on the basis of those policies and directives.

Composition of the Work Team

This audit was carried out under the responsibility of Audit Services Canada (ASC). The audit team was composed of ASC auditors, but also one auditor appointed by each of the three governments ***

Audit Sample

For the auditing of the contracting process, a judgmental sample of 12 goods and services contracts was selected from a population of 150 contracts. More specifically, the audit team selected all the contracts over $25,000 for the period before April 14, 2008, that is, four contracts, and eight signed contracts for the period of April 14 to July 17, 2008. The latter contracts were chosen on the basis of their monetary value and their representation of cost centres of the different divisions. The total monetary value of the contracts audited was $1.41 million.

For the auditing of financial controls, a random sample of 52 financial transactions was selected based on a confidence interval of 95% and a 5% probability of error. The total population audited was $2.69 million, representing 1,783 transactions for the period of June 5, 2007, to July 7, 2008. Eight transactions were added to this sample on a discretionary basis, for a total of 60 transactions.

Finally, to audit the process of allocating contributions to special projects, the audit team did not employ sampling but considered the entirety of the population. Hence all 31 special projects submitted to the Secretariat were audited.

Audit Criteria

Governance and strategic, operational and contingency plans

  1. Identify and analyze the processes and controls put in place by the Secretariat to comply with the following items:

    1. definition of the strategic directions;
    2. definition of an operating framework;
    3. definition of an integrated risk management framework (systematic identification, assessment and management of risks);
    4. the directions and the operating framework and integrated risk management framework are clearly defined and communicated;
    5. consistency between the strategic directions, operating framework and decisions made by the Secretariat;
    6. definition of the operational plans for each division (objectives, implementation schedule, critical path, use of resources and budget);
    7. the division has determined the expected results based on organizational objectives;
    8. the division compares the results obtained against targeted objectives, identifies and monitors variances, analyzes the impacts of all deviations from the expected results, and puts corrective measures in place;
    9. coordination of the schedules and strategic activities of all divisions;
    10. operational plans take changes in risks into account;
    11. communication between the decision–making parties (Organisation Internationale de la Francophonie (OIF), Steering Committee, Executive Committee, division directors) is clear and carried out regularly; and
    12. all strategic decisions are appropriately approved by the required hierarchical level.
  2. Determine whether the activities identified in the critical path can be carried out in the period of time remaining before the Summit with the human and financial resources available.

  3. Determine whether the processes and controls identified function as planned and whether they are appropriate and effective.

Budget

  1. Existence of mechanisms to manage and control the budget in an environment with changing priorities

    1. pre–commitment approval,
    2. periodic budgetary oversight and variance analysis,
    3. spending reduction measures and implementation of same, and
    4. budget reallocation mechanisms.
  2. Evaluation of effectiveness of budget controls and accountability mechanisms

    1. Pre–approvals are obtained and respected.
    2. Variance projections are estimated and reported.
    3. Spending reduction measures are approved, applied and evaluated and their impacts are managed.
    4. Budget reallocations are in line with changes in priorities.
  3. Review of the existence of budget estimates established by each division

    1. Each chargeable unit has established a spending program and cost forecasts, and has estimated the degree of certainty of forecasts, risks of over budgeting and opportunities for savings.
    2. Each unit has chosen the most economical means of realizing its objectives.
    3. The budget allocated to each unit is adequate and realistic.
    4. A contingency reserve has been created and is not compromised by predictable, unavoidable expenditures.
  4. Knowledge and awareness of the operational impact of the budget allocated to each division

    1. Available budgets are sufficient to implement all essential activities planned, with the exception of potential changes during the course of the Summit; and
    2. Cost reduction measures are applied without compromising the activities essential for achieving objectives.

Financial controls

  • Delegation of financial authorities is in compliance with the federal Financial Administration Act (FAA).
  • Financial transactions comply with the FAA and the laws and policies specific to certain types of transactions (travel, training, procurement).
  • The segregation of duties with respect to financial transactions is appropriate and optimizes control of financial resources.
  • Approved financial transactions are corroborated by a complete audit trail.

Contracting process

  1. Transparency and Value for Money

    1. Contracting processes are open, fair and transparent; and
    2. Contracts are awarded via competitive processes so as to obtain the best value, and in cases of non–competitive contracts, the reasons cited are in compliance with the rules.
  2. Controls and Impartiality

    • The decisions applied during the contractual process were made by duly authorized incumbents, and
    • The standard conditions of the federal government were applied.
  3. Procurement

    • Procurement files are documented so as to ensure a complete audit trail.

Process of allocating contributions to special projects

  1. Development of criteria for evaluating proposals

    • The selection criteria developed internally conform to the criteria communicated to bidders.
    • The evaluation scale for each criterion in the special projects analysis and evaluation grid is based on a reasonable and justifiable methodology.
  2. Controls on evaluation of proposals

    • A project analysis and evaluation grid is complete and available in the file for each project received within the prescribed timeframe.
    • There is a decision report for each project submitted to the special projects evaluation and approval committee.
    • For each criterion evaluated, a full and appropriate justification of the score obtained is provided. There is substantiating documentation in the file where appropriate.
    • The projects approved meet the requirements of the special projects financial assistance program.
    • The projects rejected do not meet the requirements of the special projects financial assistance program. Reasons for the denial of approval are clearly documented.

Greening

  1. Knowledge and documentation of the risk and impacts of the greening component on the Secretariat;
  2. Existence of a green policy and guidelines;
  3. Existence of an action plan and communication plan for the Secretariat and identification of legitimate greening targets;
  4. Existence of a mechanism to coordinate the greening action plan with the other divisions of the Secretariat;
  5. Existence of a measurement, variance assessment and reporting framework to report on results obtained with respect to greening; and
  6. Existence of a structured analysis mechanism to establish the budgetary and operational assumptions of the greening component.

External communications

  1. Note the relevance of the Secretariat's communication plan in terms of the following:

    • giving the Summit the necessary visibility for an event of this scale;
    • taking account of the population's needs, that is, the necessity of clearly explaining the objectives and benefits of the Summit; and
    • establishing an appropriate, effective and approved communication strategy for each stakeholder.
  2. Determine whether the Secretariat achieved the objectives laid out and approved in the communication plan.

  3. Ensure that the Secretariat has put a clearly defined process in place for adhering to all the items defined in the communication plan.

Top of page

Appendix B – Summary of Lessons Learned and Actions Recommended by Management

2.1 Governance

Level of Significance

Medium

Lessons Learned

2.1.1. To improve governance of the Secretariat, we recommend that risk management be introduced in the first stages following creation of an organization secretariat.

Agree/Disagree

Agree

Action(s) Recommended by Management

The Secretariat supports this recommendation and feels that a risk management committee should be set up at the outset. Also, administrative services should be in place at least one year before the event.

Timeline for Completion

Administrative services should be in place at least one year before the event.

Responsibility

In most cases, the responsibility for addressing the recommendations will rest with the group responsible for the next event of this nature.

2.2 Budget

In order to provide the Steering Committee with complete and value–added financial information on the situation of the Secretariat, we recommend that the following items be taken into consideration:

Level of Significance

Low

Lessons Learned

2.2.1 Ensure that all financial information (e.g. the initial budget and subsequent amendments, breakdown of expenditures by activity and expenditures subject to commitment, or "soft commitments" and unforeseen expenses) should be integrated into one financial results report for the activity in order to provide a consolidated picture of the financial risks, operational costs and available balance.

Agree/Disagree

Agree

Action(s) Recommended by Management

A monthly memorandum to the Steering Committee accompanied by the table of budgetary risks and financial results for the previous month provides an inclusive picture of the financial information.
The Secretariat presented all relevant information to the Steering Committee. Furthermore, at the auditors' suggestion, the Secretariat has included fiscal risks ("soft commitments") in the "Expenditures and Commitments to Date" column of the financial performance table since July 30.

Timeline for Completion

Complete

Responsibility

Director of Administration – Finance and Human Resources

Level of Significance

Medium

Lessons Learned

2.2.2. Disclose all elements for which there is an acceptable basis of measurement and a reasonable estimate of the amount.

Agree/Disagree

Agree

Action(s) Recommended by Management

From the outset, it must be stated that the most substantial costs were estimated based on historical data provided by the International Organization of the Francophonie that, for the Quebec City Summit, did not hold true. However, the Secretariat considers it made a significant effort to estimate its costs. The difficulty of this kind of exercise is increased by the unique and non–cyclical nature of a Summit, the unpredictability of some activities and, in the case of the Quebec City Summit, the interaction between three governments. For future events, the Secretariat feels that priority hiring of the people responsible for financial services could lead to better cost estimates and, consequently, more realistic budget planning.

Timeline for Completion

Administrative services should be in place at least one year before the event.

Responsibility

ADM of the group responsible for the next event of this nature.

2.4 Contracting process

Level of Significance

Medium

Lessons Learned

2.4.1 While the Secretariat is an organization that must be able to respond promptly to the situations it encounters to maintain the integrity of the process it is essential that all contracts appropriately reflect the moment that goods and/or services are obtained and that any contract amendment be immediately supported by written records entered in the file.

Agree/Disagree

Agree

Action(s) Recommended by Management

While the Secretariat, because of the very nature of its mission, must be able to act promptly, often within extremely short deadlines, every contract (goods/services) that the Secretariat awards is duly authorized under the delegation of authority plan in force, contains information justifying the need and specifies the procurement method. Nevertheless, we recognize that at the beginning of operations of the Secretariat, and before we were able to provide contracting training to the directorates, a contract amendment and a standing offer were done after the confirming order. These situations did not occur afterwards.

Timeline for Completion

Complete.

Responsibility

Director of Administration – Finance and Human Resources

2.5 Process for allocating contributions to special projects

In order to provide the Secretariat staff with appropriate monitoring tools and to guarantee the integrity of the process, we recommend that the following measures be introduced:

Level of Significance

Medium

Lessons Learned

2.5.1 Establish and implement a formal documentation methodology, at all levels, that supports evaluation and all decisions made regarding the allocation of contributions to special projects.

Agree/Disagree

Agree

Action(s) Recommended by Management

The Secretariat does have a methodology; it is based on the following 10 steps contained in official documentation:

  1. Receipt of the funding application
  2. Verification of the form
  3. Confirmation of eligibility
  4. Analysis and recommendation
  5. Submission to evaluation sub–committee (SPP–MRI–DFAIT)
  6. Submission to Steering Committee and decision
  7. Preparation of the agreement
  8. Implementation
  9. Follow–up
  10. Final processing

The Secretariat acknowledges that some documents were not available at the time of the audit but we confirm that the missing documents are now on file.

Timeline for Completion

Complete

Responsibility

Director Special Events

2.6 Greening

Level of Significance

Low

Lessons Learned

2.6.1 A greening component should be formally identified in the operating budget.

Agree/Disagree

Agree

Action(s) Recommended by Management

There were no provisions for a greening component in the initial budget allocated to the Secretariat. For the next Summit, it is important to identify a budget for the greening component ***.

To show what steps the Secretariat took, it is important to identify which budgets were allocated to greening in its operating budget, such as the budget allocated for the following:

  • food service at the airport using porcelain dishes;
  • serving fair trade coffee to delegates;
  • renting a water fountain and recyclable glasses;
  • the contract with the Centre des congrès de Québec (CCQ) for recycling and composting;
  • additional identification for recycling bins at the CCQ;
  • renting additional recycling bins for the CCQ;
  • printing the volunteers' identification badges;
  • creating the CCQ greening kiosk and purchasing items for the kiosk;
  • hiring a specialist in greening a large–scale event;
  • hiring the greening program coordinator and funding her operations (travel, training, cell phone, office supplies).

Timeline for Completion

Very early in the planning phase of the next event of this nature.

Responsibility

ADM of the group responsible for the next event of this nature.

Furthermore, to provide the Secretariat with appropriate monitoring tools, the following measures should be considered:

Level of Significance

Low

Lessons Learned

2.6.2 Have an approved environmental policy for the Secretariat at the start of operational activities in order to structure the management of those activities and achieve the targeted objectives.

Agree/Disagree

Agree

Action(s) Recommended by Management

An article on the objectives of the greening program, signed by the greening program team, was disseminated to all employees in September in the internal newsletter of the Summit's Organization Secretariat.

On October 1, the Deputy Director General sent a memo on the greening program to all employees to urge them to participate in the program.

The Steering Committee ratified an environmental policy in July, but since it was awaiting a response from the federal government on the offsetting of greenhouse gases, there was no point in disseminating the policy.

Timeline for Completion

Very early in the planning phase of the next event of this nature.

Responsibility

ADM of the group responsible for the next event of this nature.

Level of Significance

Low

Lessons Learned

2.6.3 Adopting a zero–tolerance objective, establish indicators and essential targets, in order to be able to conduct a variance analysis and report the results based on a measurement framework.

Agree/Disagree

Agree

Action(s) Recommended by Management

In the greening action plan, there are indicators that will be documented in the final report.
As for the targets, the Steering Committee chose to hold a zero–waste, carbon–neutral summit even though it had no previous benchmarks for a Francophonie Summit greening program. The thrusts of this program were first and foremost aimed at raising awareness among various Summit stakeholders about adopting best practices.

Also, the decision was made to concentrate greening efforts at the Centre des congrès de Québec (CCQ) between October 15 and 19 because of limited financial resources. Recycling and composting efforts at the CCQ included not only the above dates but also the set–up and tear–down periods at the CCQ. The efforts at the CCQ made it possible to recycle and compost 80% of the waste generated during the Summit. As regards greenhouse gases, on September 5, 2008, the federal government decided to fully offset the greenhouse gases emitted as part of the Summit. The Secretariat will provide DFAIT with the available information so it can proceed with calculating the greenhouse gases.

The next summit will therefore be able to benefit from the experience of the Organization Secretariat of the 12th Francophonie Summit and identify specific targets.

On October 1, the Deputy Director General sent a memo on the greening program to all employees to urge them to participate in the program.

The Steering Committee ratified an environmental policy in July, but since it was awaiting a response from the federal government on the offsetting of greenhouse gases, there was no point in disseminating the policy.

Timeline for Completion

Complete

Responsibility

Assistant Director General Administration and Operations


1Agency responsible for the security of the heads of state and government during their visit to Canada.

2Reference: Chapter 1000.55 of the CICA Public Sector Accounting Handbook.

3This analysis grid is based on the evaluation criteria: it defines indicators for each criterion in order to appropriately evaluate each of the projets. In addition, the grid specifies an evaluation scale that weights the criteria according to importance. This allows a rating of 100 to be assigned for each project.

4 A zero tolerance objective imposes no obligation on the organization to certify that it has produced an event with no environmental footprint.

Office of the Chief Audit Executive


* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Footer

Date Modified:
2013-01-03