In July and August 2000, the Internal Audit Division (SIV) conducted an audit of the Research & Conference Fund (RCF), a program element of the Asia Pacific Initiatives (API) Fund. The objective of the RCF is to support initiatives in Asia and Canada that will enhance the knowledge base vis-à-vis the Asia Pacific, strengthen linkages with the region, and improve the capability of Canadians to interact with the region. The fund is managed by the Asia-Pacific Regional Planning and Coordination Unit (PNSP). The total dollar value of program funding for the 1999/2000 fiscal year was $434,000.
The RCF provides funding to the North Asia and Pacific Bureau (PND) and the South and Southeast Asia Bureau (PSD) for projects that meet program criteria. Funds are not directly available to the public. Funding is based on managerial assessment that the project involves several parties with shared ownership and participation, and thus qualifies as a Contribution Agreement. PNSP recommends project approval based on project proposals provided by PND and PSD. Although PNSP recommends project and funding approval, desk officers within PND and PSD are primarily and directly accountable for the projects funded.
Prior to the audit, PNSP realized that deficiencies existed in the information-gathering and filing process. As a result, PNSP created individual project files that included program documents that existed within PNSP. This process was conducted for the 1995/1996 to 1999/2000 fiscal years. The resulting files and associated spreadsheet (which identifies basic project information) is a good initiative that, if maintained, will enable PNSP to maintain better control over projects and identify documentation that remains outstanding from the recipient and/or the originating division, as well as the overall project status.
The audit identified several areas in need of improvement:
Despite an absence of formal practices in certain areas, the program has been capably managed by PNSP.
1.1.1 The Asia Pacific Initiatives (API) Fund comprises seven elements (Research and Conference Fund; Asia Pacific Fund; Canadian Education Centres Network; APEC Central Funds Contribution; Korean Peninsula Energy Development Organization; Centre for Dialogue, Simon Fraser University; and the Canadian Chamber of Commerce), for a total annual funding value of $2.6 million.
1.1.2 The Fund was initially approved in December 1989 ***, and renewed in November 1995 ***. In August 2000, PNSP provided a revised Submission to Treasury Board, requesting a five-year renewal of the authority.
1.1.3 Program funding for all API programs was $2.6 million for the 1999/2000 fiscal year. Of this amount $434,000 was provided to the RCF. RCF funding essentially comprises funds remaining once the other organizations have been funded. One Hundred Percent of API funding falls under Vote 10 (Grants and Contributions).
1.1.4 The API is administered by 3 FTEs. The Deputy Director (PNSP) is the primary liaison to PND and PSD (the sources of project proposals), and is responsible for reviewing and assessing proposals and recommending project approval. Final approval for RCF projects lies with the Director General, PND and the Director General, PSD who sign Contribution Agreements.
2.1.1 The overall objective of this audit was to review the PNSP administrative and operational frameworks surrounding the RCF element of the API program, and to identify potential improvements in practices that would assist PNSP reach optimum achievement of its operational objectives. PSND operations were reviewed to:
2.2.1 The audit comprised a review of 11 projects from the 1995/1996 to 1999/2000 fiscal years.
2.2.2 RCF expenditures for the audit sample totalled $676,531 (38% of the total RCF project value for the scope years reviewed). The following table provides a breakdown of RCF funding by year for sample projects reviewed. The table was developed based on information provided by PNSP management. The Audit Team was unable to reconcile this information to the overall API budget to ensure completeness of the data, as final budget information surrounding the RCF was not available.
|Fiscal Year||Vote||Total Projects for Year||Sample Projects||Total Value of RCF Projects||Total Value of Sample||% of Total RCF Value for Year|
2.3.1 The methodology comprised:
3.1.1 The following findings are based on a review of a sample of project files and interviews with PNSP officers.
3.2.1 The audit found several challenges that affect the ability to manage the program at optimum effectiveness. The Audit Team was informed that frequent staff turnover and absence of corporate memory within divisions has frustrated efforts at program management. This emphasizes the need for processes that can easily be understood, followed and transferred. While support for payment is on file, there is no information as to benefits derived from a program perspective. The following are a summary of findings surrounding the management and administration of the program:
3.2.2 The impact of inadequate processes (calls for proposals, monitoring, guidance) may include confusion about the program, missed opportunities, and a lapse of funds.
Recommendation for PNSP
3.2.3 PNSP, in conjunction with PND, PSD and PAM, develop program "guidelines" that define and describe key program management processes; particularly those required by Treasury Board policy.
3.2.3 With guidance from SIV, PNSP developed the "Guidelines for the Research and Conference Fund: Application, Review, Approval, and Payment Process" (attached). These guidelines are included in PNSP's formal call for RCF proposals. Divisions have been informed that this document is also available in the P-Branch shared network drive.
To ensure that projects from the FY 2000/2001 and onward are systematically assessed and approved, PNSP created a Project Proposal and Assessment Template (Annex A). The template includes two assessment forms: one to be completed by the responsible geographic division and the other by PNSP. Also included is a section for the responsible Director General's approval.
Furthermore, PNSP prepared a project/recipient evaluation guide (Annex A) for geographic divisions to ensure that projects/recipients are evaluated upon project completion.
3.2.4 Prior to the 1999/2000 fiscal year, a central filing system did not exist. The result is the inability to maintain complete project documentation in a single location and within a primary project file. In mid-2000, at the request of Departmental senior management, PNSP created individual project files that included documents that existed within PNSP. This process was conducted for the 1995/1996 to 1999/2000 fiscal years. The resulting files and associated spreadsheet (which identifies basic project information) is a good initiative that, if maintained, will enable PNSP to retain knowledge of documentation that remains outstanding from the recipient and the originating division, as well as the overall project status. The creation of a centralized filing system would facilitate the document collection process. It should be determined where the central files should be located, and ensure that stakeholder divisions forward copies of all project documentation to PNSP for inclusion in the project file. A checklist of documents required could be included in the process "guidelines".
Recommendation for PNSP
3.2.5 PNSP collaborate with PND and PSD to develop a centralized program filing system.
3.2.5 During the second quarter of FY 2000/2001, PNSP developed a centralized program filing system for projects under the Asia Pacific Initiatives Program. Project files under this new system date back to the FY 1995/1996.
3.2.6 All files reviewed were missing key process and project documentation. Fundamental information, particularly surrounding the project proposal, evaluation and approval process, is absent or so minimal that it is not possible to determine whether a complete assessment of the eligibility of the applicant against program criteria was conducted. The absence of this type of information does not allow a project officer unfamiliar with the project to easily assume responsibility for the file.
Recommendation for PNSP
3.2.7 Ensure the review of all RCF project files for completeness of project documentation.
3.2.7 PNSP, with the assistance of SIV, developed a project documentation checklist (Annex A) against which to verify RCF project files. Before closing a project file, PNSP now ensures that project documentation is complete.
3.3.1 Actual budget figures for the audit scope years were not available. PNSP management provided a summary of program funding (1999/2000) to the Audit Team. Original program funding for all API programs was $2.6 million for the 1999/2000 fiscal year. Of this amount, $1 million was provided to the Asia-Pacific Foundation (APF); $1.124 million was provided to the CECN, $434,000 was provided to the RCF; and the balance provided to the remaining four recipients under the API fund. RCF funding essentially comprises the funds remaining once the other API programs have been funded.
3.3.2 From an audit perspective, there is uncertainty surrounding the budget development process. The Audit Team was unable to locate final approved budget documentation for the RCF for any of the fiscal years reviewed. PNSP often does not acquire a confirmed budget for RCF until late in the fiscal year. For the 1999/2000 fiscal year, this contributed to approximately $75,000 in lapsed funds. By the time the final budget was confirmed, it was too late in the fiscal year for PNSP to return to PND and PSD to request additional project proposals. The Audit Team was informed that there have been "significant changes" in the budget process for the 2000/2001 fiscal year. Information reviewed by the Audit Team, including the new Treasury Board Submission, does not provide detailed information regarding the budget process.
3.3.3 The Audit Team found that final, approved budget documentation for RCF was not available in PNSP, PND, PSD, the Central Processing Unit (CPU) or the Local Information Services Offices (LISO). The Audit Team was informed that this is largely due to the fact that RCF funding is from money "left over" after the Asia Pacific Foundation and the Canadian Education Centres Network (CECN) have been funded. The primary challenge faced by PNSP is the uncertainty over the final budget amount. This varies at different points throughout the year, largely due to the uncertainty surrounding the "in kind" contributions provided to the CECN. The development of a comprehensive budget would help divisions plan projects and avoid lapsing available contribution funds at year end.
Recommendation for PNSP
3.3.4 A comprehensive, approved budget for RCF be developed and approved by the appropriate officers at the beginning of the fiscal year.
3.3.4 The new Treasury Board Submission for a five-year renewal (FY 2000/2001 to FY 2004/2005) of the authority for Asia Pacific Regional Programs was approved in August of the FY 2000/2001 ***.
A maximum of $1 million per annum is committed to the Asia Pacific Foundation of Canada.
A residual and decreasing amount of funds is committed to the Canadian Education Centres Network (CECN). Given the expectation that the CECN will achieve full financial self-sufficiency in five year's time, contributions will be reduced each year from the $300,000 in the FY 2000/2001 to $100,000 in the FY 2004/2005 (at which time contributions will terminate).
A minimum of $250K per annum is committed to a contingency fund. As the commitment to the CECN decreases by $50K each fiscal year, the contingency fund will increase by $50K until the FY 2004/2005.
A minimum of $450K per annum is committed to the RCF. In the FY 2000/2001, the contingency fund was used towards the RCF and PNSP expects it be used towards the RCF in the next four fiscal years.
3.4.1 Proposals for project funding are submitted to PND and PSD by NGOs, the Office of the Minister of Foreign Affairs (MINA), the Office of the Secretary of State (SSAP) and other DFAIT sectors. PNSP solicits project proposals from PND and PSD at two or three periods over the course of the fiscal year. PNSP does not put out a formal call for proposals at regular intervals throughout the year. Divisional priorities, which drive the type of project proposals desired, are described in the annual divisional business plans.
Recommendations for PNSP
3.4.2 To facilitate the submission of the project proposal process, PNSP should put out a systematic call for standard format proposals at regular intervals.
3.4.3 To facilitate the proposal process, PNSP should define and communicate key program processes.
3.4.2 PNSP developed a Project Proposal and Assessment template (Annex A) and has issued two formal calls for proposals since the fall of the FY 2000/2001.
3.4.3 To communicate key program processes, PNSP includes with its formal call for proposals, the "Guidelines for the Research and Conference Fund: Application, Review, Approval, and Payment Process" (Annex B).
As part of PNSP's efforts to communicate its activities to the Asia Pacific Branch, a brief presentation on the RCF process will be made to each division on an ongoing basis.
3.4.4 Determining project eligibility and assessment of the recipient is a critical step in the contribution process. The primary weakness of sample files reviewed was in the evaluation of project eligibility. There is little documentation in project files which indicates that program management has assessed the recipient against program criteria. The application form/project proposal is on file in 8 of 11 RCF sample files reviewed. In 4 of 11 cases, there is no evidence on file that the project has been reviewed against general and specific program criteria.
3.4.5 In 6 of 11 cases, there is evidence of consultation with project stakeholders (such as divisional representatives, SMFG) regarding project feasibility and approval. Consultation usually took the form of discussions, via e-mail.
3.4.6 Ten of 11 files reviewed demonstrated that the project was approved by the appropriate officer; that is, the Director General, PND or the Director General, PSD whose signature appears on the Contribution Agreement.
3.4.7 None of the files include a project evaluation form by which the project proposal was assessed by the originating division or by PNSP. The Audit Team was informed that this is largely due to the fact that the PND/PSD desk officer responsible for developing the proposal (in conjunction with the recipient), and the Deputy Director, PNSP, are knowledgeable about program objectives and recipients and therefore "know" whether a project and/or recipient is feasible prior to receipt of the project proposal. Despite this knowledge, a written record of decision would improve project tracking and facilitate the development of a track record of the recipient. Such documentation would also alleviate some of the challenges faced by new staff by enabling them to determine project eligibility and status. This information may need to be prepared in consultation with the PND/PSD desk officer who is directly responsible for the project.
3.4.8 Supporting documentation, particularly information relevant to eligibility assurance, is important to ensure project relevance and the ability to justify funding. In addition, PNSP is scheduled to present a new Submission for funding to Treasury Board in the fall, 2000. An increase in program documentation will facilitate building the PNSP "business case".
Recommendation for PNSP
3.4.9 That PNSP prepare a written summary or project evaluation form regarding the final decision as to project approval or rejection, and include the value of project funding.
3.4.9 To ensure that projects from the FY 2000/2001 and onward are systematically assessed and approved, PNSP created a Project Proposal and Assessment Template (Annex A). The template includes two assessment forms: one to be completed by the responsible geographic division and the other by PNSP. Also included is a section for the responsible Director General's approval.
3.5.1 Project files were reviewed to consider the mechanism by which they were funded; i.e., grant, contract or a contribution. Although all project files reviewed were funded with Vote 10 (Grant and Contribution) money, in two cases reviewed, it could be argued that a "product" was delivered to the Crown. For example, a research project to identify the political and economic climate of a South-East Asian nation and potential for economic investment was funded by the RCF. As part of the recipient's narrative report, a copy of the research paper was provided to DFAIT. The argument in favour of contribution funding in this case, stems from the terms and contributions approved in the Treasury Board Submission, which indicate that projects should "fill a knowledge gap". In this case, it could be argued that such a gap was filled. In addition, the project was commissioned by a Canadian Non-Governmental Organization (NGO). The work was conducted by a partnership comprising Canadian and Japanese academics. In addition, the report served as proof that the project was conducted.
3.5.2 PNSP program management indicated that there is, at times, a "fine line" between contribution and contract funding. The terms and conditions of the approved Treasury Board Submission are broad, therefore, the Audit Team determined that for sample projects reviewed, the contribution was a legitimate funding mechanism. In addition, PNSP seeks approval from the Deputy Director, Contracting (SMFG) for each Contribution Agreement. As of August, 2000, Agreements will be vetted by the Manager, Payment Services (SMFP).
3.5.3 The review by SMFP ensures that the project has been reviewed against the principles of what constitutes a contribution (versus a grant or contract), and that the contribution as a form of project funding, has been determined to be a feasible funding mechanism. The audit found that in 7 of 10 cases, evidence that the agreement was approved by SMFG was on file. As of August, 2000, this responsibility has been transferred to SMFP.
Recommendation for PNSP
3.5.4 Ensure that a copy of SMFG approval is included in all project files. For projects commencing after August 2000, ensure that a copy of SMFP approval is included in all project files.
3.5.4 The responsibility for approval of Contribution Agreements has been transferred to SMFH.
PNSP now verifies files against a project documentation checklist developed with SIV and ensures that among other required documentation, a copy of SMFH approval is on file.
3.6.1 The originating division is primarily responsible for managing project progress. It may therefore be more feasible for the originating division; i.e., PND or PSD to prepare an assessment as they are knowledgeable of specific project objectives, and how they fulfill overall program requirements. As PNSP is required to obtain primary program files, it could ensure that assessments are prepared.
3.6.2 It is primarily the responsibility of the originating division (i.e., PND or PSD) to monitor the recipient as per the terms and conditions of the Contribution Agreement. The Contribution Agreement contains clauses outlining the responsibilities and requirements of the recipient and the donor, and enables PNSP to track specific project requirements, recipient obligations, reporting requirements and payment schedules, and to demonstrate that projects funded concur with program objectives.
3.6.3 The Audit Team found that although systematic and consistent program monitoring practices have not been implemented within PNSP, final reports prepared by the recipients provide a summary of the effects that projects have had with respect to program objectives. These reports are key to ensuring that funding is used to further program objectives (and thereby ensure justification for funding) and obtaining information on the value-added derived from the project.
3.6.4 The audit found that in 10 of 11 cases, the recipient provided activity, progress or project reports to the project authority at DFAIT. In no case, upon closing the project file, was an assessment of the project or the recipient conducted by PNSP, PND or PSD. Such assessments are beneficial to the Department as they can be used to develop "track records" of the recipient, which could in turn be useful if the recipient subsequently applies for funding.
3.6.5 Despite gaps in or absence of processes, the audit found that in 8 of 11 cases, there is evidence that the project produced the intended results. This is apparent from the narrative reports and/or invoices provided by the recipient as per the terms and conditions of signed Contribution Agreements.
3.6.6 Monitoring could take the form of a "note to file" summarizing the project, amount of funding provided, original objectives and actual results. This information could form the tombstone data of a system that tracks eligible recipients over time and mitigate the risk of funding applicants with unfavourable track records. This will enable program management to respond to senior management queries about the program and add to the program's overall tracking system. In addition, it will enable those unfamiliar with the program to gain knowledge of ongoing projects.
Recommendations for PNSP
3.6.7 PNSP, in conjunction with PND and PSD, to obtain missing project reports from all recipients and include a copy in the project file.
3.6.8 PNSP, with input form PND and PSD, to prepare periodic program assessment reports that include program budget, expenditures, lapsed funds, project listing, project summaries, and whether program objectives have been met.
3.6.7 PNSP is currently gathering project reports for the FY 2000/2001. Most of the missing project reports from the FY 1999/2000 have been obtained and PNSP continues to pursue responsible divisions for the missing ones.
3.6.8 Periodic program assessments have been performed and a spreadsheet containing all of the above mentioned information has been created. Furthermore, PNSP has prepared a project/recipient evaluation guide (Annex A) for geographic divisions to ensure that projects/recipients are evaluated upon project completion.