Evaluation of the Asia Pacific Foundation of Canada

Evaluation Report
Prepared by the Evaluation Division Global Affairs Canada
December 2021

Table of contents

• Acronyms and Abbreviations
• Executive Summary
• Evaluation Scope and Methodology
• Program Background
• Findings
• Conclusions
• Considerations and Recommendations
• Annexes

Acronyms and abbreviations

ABAC

APEC Business Advisory Council

ABLAC

Asian Business Leaders Advisory Council

Act

Asia-Pacific Foundation of Canada Act

APEC

Asia-Pacific Economic Cooperation

APF Canada

Asia Pacific Foundation of Canada

CEO

Chief Executive Officer

CGA

Conditional Grant Agreement

CIGI

Centre for International Governance Innovation

CPI

Consumer Price Index

FOIP

Free and Open Indo-Pacific

FY

Fiscal Year

GAC

Global Affairs Canada

GIC

Governor in Council

HR

Human Resources

ISED

Innovation, Science and Economic Development Canada

PECC

Pacific Economic Cooperation Council

PGRF

Post-Graduate Research Fellows

OGDs

Other Government Departments

Executive summary

This evaluation examined the operations and activities of the Asia Pacific Foundation of Canada during the period from 2014-15 to 2019-20. The objective was to assess the alignment of the Foundation's activities with the requirements of the Conditional Grant Agreement (CGA), along with the effectiveness of the Foundation's governance structure, progress toward outcomes, and the state of communication with Global Affairs Canada (GAC). This report presents the evaluation findings, conclusions and considerations as well as recommendations.

Starting in 2014, the Foundation's business model evolved toward a more action-oriented approach serving broader audiences. This new approach led to changes in programming and the adoption of a new business development strategy, as well as organizational changes. This evolution of the Foundation over the last five years set the backdrop for the evaluation and led to findings in three areas: programming, governance and management, and government relations.

The programming conducted by the Foundation produced solid results that aligned with, and were supportive of, the mandate outlined in the CGA. The broad mandate provided the flexibility required for the Foundation to undertake new and relevant programming that responded to changing contexts. Stakeholders found value in the Foundation's activities, particularly improved awareness of the Asia Pacific region through the publication of a large number of products. In addition, the Foundation played a positive convenor role between peoples and institutions of Canada and the Asia Pacific region through the fostering of various networks. An area identified to help strengthen research programming is a better utilization of the Distinguished Fellows network.

The evaluation found that the Foundation was well managed with a strong governance and organizational structure. The $50-million Endowment Fund, established through the CGA, provided a stable funding base. The Board has done an effective job of governing the Foundation and has made prudent financial decisions over the years. The Foundation has also been successful in mobilizing external funding sources. The organizational structure has evolved to reflect the needs of the current business approach and is well suited for the Foundation's activities. The grant requirement embedded in the CGA has impacted the Foundation's staffing strategy, resulting in the research team comprising a small number of senior staff and a large cadre of junior researchers. While the model provides flexibility for broad-based programming, the grant requirement was found to limit the Foundation's ability to change the staffing model if needed.

While examining government relations, the evaluation found a few areas in need of further clarification. There is a partial misalignment between the Act and the CGA in terms of independence and oversight. Although granted independence through the Act, the Foundation must still comply with a number of CGA and other government policy requirements. Global Affairs Canada's oversight role, and the extent to which the Foundation's work should be consistent with the Government of Canada's priorities, is unclear. In addition, the evaluation found that the Government of Canada has not consistently fulfilled its obligations to the Foundation under the CGA, such as appointing Governor in Council (GIC) Board members on a timely basis. Lastly, communications between the Foundation and the government have largely been ad hoc and transactional, causing an expressed need for more formal communication channels.

Evaluation scope and objectives

Evaluation rationale

The Asia Pacific Branch of Global Affairs Canada (GAC) recommended that an evaluation of the Asia Pacific Foundation of Canada (APF Canada) be conducted in line with the provisions of the Conditional Grant Agreement (CGA) signed between the Foundation and the Minister of Foreign Affairs in 2005. As per Section 9(2) of the Agreement, the Minister of Foreign Affairs may undertake, "at the expense of the Department, evaluations of the Foundation by evaluators of his or her choosing". This is the second department-led evaluation of the Foundation; the first was conducted in 2008-09by the Diplomacy, Trade and Corporate Evaluation Division at GAC.

The evaluation process followed the Treasury Board Policy on Results and Directive on Results. In the Government of Canada, evaluation is defined as "the systematic and neutral collection and analysis of evidence to judge merit, worth or value. Evaluation informs decision-making, improvements, innovation and accountability. Evaluations typically focus on programs, policies and priorities and examine questions related to relevance, effectiveness and efficiency."

Evaluation scope

The evaluation covered the Foundation's operations and activities between April 1, 2014, and March 31, 2020. The evaluation also examined the effectiveness of the communication and engagement between the Foundation and the Government of Canada—notably, through GAC. Finally, the scope included a review of GAC's activities in support of APF Canada and examined opportunities for improvement with regard to fulfilling the department's responsibilities under the CGA.

Evaluation approach

The evaluation was conducted in-house by the Evaluation Division of Global Affairs Canada. The team was supported by an independent consultant with extensive evaluation experience and knowledge of GAC, including participation in the 2008-09 evaluation of the Asia Pacific Foundation of Canada.

An Evaluation Advisory Committee comprising representatives from GAC and APF Canada, including members of the Foundation's Board of Directors, was established as a consultative mechanism to provide the benefit of a transparent evaluation process. The Committee was engaged throughout the evaluation process and provided advice and comments on deliverables.  

Concurrent with the evaluation, the firm KPMG conducted an external review of the CGA. The purpose of the review was to update and renew the Terms and Conditions of the CGA. The evaluation and review were coordinated to leverage synergies and avoid duplication.

Evaluation questions

Methodology

The evaluation team applied a mixed-method approach, using a range of data collection and analysis tools to assess each of the proposed evaluation questions. Five main methods were used to conduct this evaluation. See Annex A for additional methodology details and Annex G for a summarized list of sources per section.

Evaluation limitations and mitigation measures

Program background

The Asia Pacific Foundation of Canada

The Asia Pacific Foundation of Canada is an independent, not-for-profit organization established by the Asia-Pacific Foundation of Canada Act in 1985 as a result of the Government of Canada's identified need for stronger relations with the Asia Pacific region. The Act laid the basis for APF Canada's purpose, mandate, governance and reporting.

In 2005, the Minister of Foreign Affairs signed a Conditional Grant Agreement (CGA) with the Foundation. The CGA provided a $50-million conditional grant for the establishment of an Endowment Fund to ensure predictability of APF Canada's finances and to allow for longer-term planning. The CGA was also intended to enhance the Foundation's ability to develop a granting program and maintain several formal affiliations with regional organizations to help strengthen Canada's relations across the Asia Pacific region. To ensure that the funds were managed in accordance with the Policy on Transfer Payments, the CGA contained a number of requirements for governance, reporting, programming, and the management and use of the grant.

Mandate

The Act and the CGA clearly outline the Foundation's purpose and mandate, which is in essence to build networks and develop closer ties between the peoples and institutions of Canada and the peoples and institutions of the Asia Pacific region. Figure 1 illustrates the Foundation's theory of change using language from the Act and the CGA.

Figure 1: Theory of change

Organizational structure and finances

APF Canada's organizational structure and finances have evolved considerably since the previous evaluation conducted by the department in 2008-09, shortly after the Foundation was hard hit by the financial crisis. The Foundation underwent major staffing cuts in 2007 and saw the market value of the Endowment Fund fall below $50 million by October 2008. Since that time, the Foundation has recovered, and various management and structural changes were made between 2008 and 2014, which were not reviewed as part of this evaluation.

As of 2020, the Foundation is governed by a Board of Directors and managed by the President and Chief Executive Officer (CEO), who provides additional oversight and leadership. There are currently 39 full-time equivalent staff within the organization. Although the organization's main office is based in Vancouver, British Columbia, the Foundation has since expanded to include a satellite office in Toronto, Ontario. See Annex B for an adapted organizational chart as of May 2020.

The Foundation is funded through the proceeds of the Endowment Fund and through outside contracts and grants. The yearly budget for the Foundation is approved by the Board and has steadily increased over the last five years as new revenue sources were added. During the evaluation period, the Foundation's expenditures have grown from $4.8 million in FY 2014-15 to a high of $6.7 million in FY 2018-19.

Figure 2: Revenue and expenditures

The revenue figures include net investment income for the year from the Endowment Fund along with revenues from other sources, such as government contracts and the private sector. The yearly shortfall is covered by the generation of capital gains on the investments during the year or through draws from the unrestricted funds—investments over $50 million. Investment income fluctuates each year depending on market returns. Even with yearly withdrawals from the unrestricted funds, the investments overall have continued to grow.

The majority of external funding generated has been from government sources. Global Affairs Canada and other Canadian government departments have contributed approximately 60% of the government funds to the Foundation, with the Kakehashi Project, funded by Japan, contributing most of the remaining government funds. The Foundation's two largest government contracts are the $2.3-million GAC-funded Asia-Pacific Economic Cooperation (APEC)-Canada Growing Business Partnership and the $2-million Innovation, Science and Economic Development (ISED)-funded Women's Business Missions through the Women Entrepreneurship Fund.

Introduction to findings

Starting in 2014, the Foundation began to change its business model. With the arrival of a new President and CEO, a decision was made in conjunction with the Board to move the Foundation toward a more action-oriented approach serving broader audiences. Over the evaluation period, changes were made in programming, financing and management of the Foundation to put this model in place.

While major reports and policy briefs continued to be published, new formats and content were developed for products including shorter, more timely analyses such as dispatches, blogs and podcasts. For example, the Asia News Service was transformed to Asia Watch, focusing on providing analysis of news, trends, and issues in Asia.

As part of the transition, APF Canada developed new programming for a range of stakeholders, such as pilot programs on curricula with British Columbia as well as internships and the fostering of new networks.  

At the Board's direction, the Foundation adopted a business development strategy that focused on generating additional funding primarily from government sources and on monetizing products and services to match the research and programming agenda, such as the Investment Monitor where subscribers can pay to access specialized data and information.

In addition, the Foundation made a series of organizational changes. Human resources (HR) processes were put in place including an employment manual. The Communications Division was established to develop and manage communications strategies to reach key audiences including through social media. The Foundation reorganized itself to align with the pillars of the new Strategic Plan in 2019.

This evolution of the Foundation over the last five years set the backdrop for the evaluation. The changes made over the period were integrated into the analysis of the findings.

Figure 3: Timeline of key events

Programming

1. The Foundation has adapted to an evolving context, developing new and relevant programming aligned with its broad mandate.

The Foundation's purpose, outcomes, and expected results are outlined in the Act and the CGA. Together, these two documents shape the Foundation's mandate, which covers a number of areas including education, capacity building, economic relations and network-building. Although providing flexibility, the mandate provides limited guidance on areas where the Foundation should focus. This has meant that the evolution of priorities by the Foundation over the last five years has resulted in a wide range of pertinent projects in areas such as energy policy, national security, public health, education, trade promotion, development programming, and more. See Annex C for the Evolution of Strategic Plans.

The evaluation conducted an analysis of 25 of the Foundation's activities undertaken over the last five years, known as points of impact. The analysis demonstrated consistency and alignment with the Act and the CGA. Virtually all activities aligned with more than one area of the Foundation's outcomes, with a weight toward promoting economic ties and collecting and disseminating information. For example, the Canada-China Track II Energy Dialogue, a dialogue series that convened government officials and environmental activists and organizations on Canada-China engagement surrounding energy issues, aligned with numerous areas of the mandate. The initiative promoted dialogue and understanding of foreign policy issues related to the region, promoted collaboration among organizations in private and public sectors, and promoted closer economic and commercial ties. 

The broad mandate has provided the Foundation with the flexibility to undertake new and relevant programming that would have been otherwise not possible with a more rigid mandate. It has also allowed the Foundation to rapidly respond to changes in the Asia Pacific region. An example is the new "Mapping the Asia Pacific's COVID-19 Response" project launched in 2020, which tracks developments around COVID-19 policy responses in the region. This project aligns with the outcome of collecting and disseminating information and also aims to inform Canada's domestic pandemic response by identifying policy best practices.

The Foundation continues to have a clear niche in Canada as the only organization dealing with regional issues and Canadian relations. While its activities align with its mandate, a range of stakeholders wondered whether the breadth of the areas currently covered was too large given the budget. Was there a trade-off between breadth and depth to having an impact?

Figure 4: Analysis of activities

2. The Foundation is valued for its activities and programs that have contributed to increased awareness and understanding of the Asia Pacific region.

To assess the Foundation's contribution to promoting capacity development on Asia Pacific issues, the evaluation examined 25 activities or points of impact highlighted in the Foundation's 5-Year Organization and Activities Review.

One such program is the Kakehashi Project, which was established in 2013 and was financed by the Japanese government and administered locally by APF Canada. This program provided youth exchanges between Japan and Canada to promote mutual trust and understanding. In total, over 1,000 exchanges have been conducted during the evaluation period.

The Foundation also conducted educational activities such as the development of curricula for middle- to high-school students in British Columbia. The curricula are well regarded as a resource to build Asia competencies for youth. Due to this success, the Foundation is interested in further expanding the curricula to other provinces.

In addition to activities focused on youth, the Foundation has a number of signature products such as National Opinion Polls, Asia Watch, and the Investment Monitor. Feedback on these products indicated that they were valuable tools for a range of stakeholders. Different platforms and products attracted different audiences. For example, a recent survey of Asia Watch readers revealed that a vast majority of the readers were from the private sector (26%), government (21%) and academia (23%). On the other hand, Twitter followers and visitors to the website had less representation from these three groups and more by the general public. LinkedIn provided the highest engagement rates and catered to professionals. The evaluation conducted a backlink analysis to determine the types of stakeholders referencing or linking to the Foundation's web pages on their websites. The analysis found that the online resources made available by the Foundation were primarily linked to by the general public, followed by business and academia. Overall, stakeholders interviewed viewed these products as useful and providing value.

Lastly, the Foundation's activities included contracted policy work by government and business stakeholders that provided insightful information and analysis on the region's markets. For example, one of the points of impact highlighted by the Foundation was Sectoral Market Development where the Foundation researched and published papers on topics such as liquefied natural gas markets in Asia and artificial intelligence.

Overall, the evaluation found that the Foundation's activities were successful in contributing to the promotion of capacity development in persons and in entities sharing an interest in the Asia Pacific region.

3. The Foundation's numerous publications have not translated into a high global think tank ranking.

According to the Act and the CGA, only part of the Foundation's mandate is directly related to research—to promote dialogue on and understanding of foreign policy issues and to collect and disseminate information. With a shift to the new business model, the Foundation stated that it aims to provide high-quality research as well as actionable policy considerations and business intelligence by working with stakeholders. While these objectives are in line with the work of global think tanks, the Foundation does not rank high when compared to other think tanks.

A think tank can be defined as an organization that engages in public policy research, analysis, and engagement to provide advice on issues to policy-makers and the public to help them make informed decisions. The evaluation conducted a comparative analysis between APF Canada and nine other think tanks of similar size and focus to assess the Foundation's outreach and influence. The comparison included metrics such as social media reach, website traffic, as well as publications and citations. While the Foundation ranked tenth for academic citations of its work on Google Scholar, it ranked eighth for social media content and fifth for website traffic, demonstrating its growing outreach efforts. It is important to note, however, that there were limitations in this analysis such as finding comparable organizations with similar funding models, objectives and activities. See Annex D for additional details on the comparative analysis.

The Global Go to Think Tank Index, based on an international survey of scholars, public and private donors, policy-makers, and journalists, ranks more than 6,500 think tanks using a set of 18 criteria. Metrics include peer ranking, public nomination, quality and reputation of leadership and staff, as well as academic performance and reputation including number and reach of publications. The 2020 report ranked APF Canada 38th out of 44 Canadian and Mexican think tanks. The ranking could be linked to the Foundation's focus and capacity to conduct more action-oriented policy work rather than developing academic, peer-reviewed research papers.

The Foundation has acknowledged its lower ranking among global think tanks, but does not believe the ranking is a true representation of its value. For example, the benchmarks used in the indexing model do not align with the Foundation's strategic plan and its priorities to provide more action-oriented research and business intelligence. The Foundation therefore did not see any value-added in focusing efforts on improving its ranking.

An option to increase the reputation of its research could be to better engage Distinguished Fellows. The Foundation has a network of Distinguished Fellows in place, which represents an important pool of expertise. The Fellows are subject-matter experts in various aspects of Canada-Asia relations and are available to provide a range of support including participating in dialogues and public events, providing commentary, responding to domestic and international media requests, and informing research projects. However, wide variations were seen in the involvement of the Fellows in various activities since the Foundation often relied on the Fellows to be proactive in how they wanted to engage. This has meant that the Foundation has not always leveraged the expertise of the Fellows effectively to provide support and guidance for the Foundation's research and policy work.

The evaluation found that while the Foundation's comparative think tank ranking is low, the Foundation has chosen to focus its efforts on producing a large amount of varied content, which is well aligned with its strategic plan and the Board's desire to produce content relevant to Canadians. Nonetheless, more effective approaches could be employed to better engage the Distinguished Fellows to support the Foundation's policy and research work.

Figure 5: APE Canada think tank comparative ranking

4. The Foundation has developed a positive convenor role through networking activities intended to contribute to developing closer ties between the peoples and institutions of Canada and the Asia Pacific region.

Over the past five years, APF Canada conducted several activities that contributed to building networks and developing closer ties. One such initiative was the Women's Business Missions, which has led, to date, to 12 deals between Canadian and Asian women entrepreneurs and businesses. These missions were piloted in Japan in 2019 as the first gender-based mission to Asia and have now been expanded to other economies such as South Korea and Taiwan. Female entrepreneurs were provided the opportunity to network with Asian partners while promoting their products or services to an international market.

Another set of network-building activities conducted by the Foundation as part of the CGA requirements was providing secretariat services to the Canadian members of the APEC Business Advisory Council (ABAC) and the Pacific Economic Cooperation Council (PECC). As part of the services provided, the Foundation supported the Canadian members in meetings and provided reports and summaries as needed. Both councils provided recommendations and promoted dialogue and cooperation between the members of the international network. The support provided by the Foundation to the Canadian members was highly regarded and was seen as contributing to policy and business development for Canada. While the CGA stipulates that the Foundation should provide secretariat services to these councils, it does not specify what costs the Foundation should cover for its secretariat work from the Endowment Fund and what costs are outside of the agreement. This lack of clarity has led to confusion and to several requests for additional resources to GAC and other sources to cover costs associated with conferences convened, but not considered by the Foundation as part of the secretariat work funded by the Endowment.

Figure 6: Results from the women's business missions to date

Another initiative by the Foundation was the inauguration and leading of the annual Asia Business Leaders Advisory Council (ABLAC) meetings since 2016. These high-level meetings brought together approximately 30 Canadian and Asian business and government leaders annually to create dialogue on strategic policy and build stronger relations and opportunities between Canada and Asia. The meetings touched on the economic relations between Canada and Asian nations relating to issues such as trade, technology, and sustainability. The Council is regarded as important by participants and the Foundation.

The CGA also called for the Foundation to act as the APEC Study Center of Canada. There are over 50 APEC Study Centers around the APEC region. Until 2019, the Foundation undertook few activities related to this requirement. However, since 2019, it is actively engaging with a number of APEC study centers and establishing work plans.

To better understand its reach, the Foundation conducts its own extensive media and social media monitoring and analysis. Although the Foundation's mission is to be Canada's catalyst for engagement with Asia and Asia's bridge to Canada, most of the Foundation's content was created in English, therefore all social media platforms, except for LinkedIn, are currently not accessible in many Asian countries without the use of a virtual private network. A communication audit in 2017 identified methods to increase the Foundation's reach, which the Foundation has been applying. The latest figures provided by the Foundation demonstrated that its reach has expanded over the evaluation period. Web visits and page views increased by 26% and 68%, respectively, while social media engagement numbers grew across various platforms including Twitter (+75%), Facebook (+86%) and LinkedIn (+9%). In addition, newly adopted platforms such as Instagram and YouTube have also experienced growth.  

While the Foundation could put further effort into expanding its reach to become Asia's bridge to Canada by translating content into other languages or joining Asian-based social media platforms, the evaluation found that, overall, the Foundation's activities have contributed toward the building of networks and the development of closer ties between the peoples and institutions of Canada and the Asia Pacific region.

Governance and management

5. Over the past five years, the Foundation has managed the Endowment Fund well and made prudent financial decisions.

The Board, and specifically the Investment Committee, oversees the management of the Endowment Fund. A Statement of Investment Policy and Procedures is in place covering all the requirements listed in the CGA. In 2018, an independent firm was retained to assess the performance of the Fund and provide reports to the Board and the Investment Committee. These reports contain detailed analysis of the investment portfolio including its overall performance, analysis of returns compared to the market and commentary on market fluctuations that may impact the portfolio.

Figure 7: Rates of return against objectives
(4-year rolling average)

Two benchmarks are used to track the portfolio—Consumer Price Index (CPI) +4% and a custom benchmark based on the Fund's asset mix. On a four-year rolling basis, the Fund has outperformed both benchmarks. The Fund was also able to quickly recover from the 2020 market adjustment and has grown from $63.7 million in 2014 to over $72 million in 2021.

The Endowment Fund provided the Foundation with a base of funding that paid for operational and some programming costs as well as administering the grants program and secretariat work. The amount of annual distribution coming from the Endowment Fund has fluctuated between $3.4 million and $3.96 million per year during the evaluation period, based on the level of expenditure. The Board reviewed the returns on the Endowment and the potential future market volatility and decided to cap annual spending from the Fund to preserve the capital base. Caps of $3.5 million in 2020-21 and $3.25 million going forward were set, with further funding having to be mobilized from external revenue sources.

The CGA requires that 25% of the revenue of the Endowment Fund goes toward funding a grants program. The CGA was vague, however, on what "revenue" meant or how it was to be calculated. The Board decided in 2015 to move to an accrual system of accounting and set the grant level per year at $360,000 based on a rolling three-year average of income. This rolling average was established following changes in asset allocation in the Foundation's Endowment, which created unexpected revenue spikes that would originally have to be spent on grants in a single fiscal year. In April 2019, the Board sought to clarify the definition of "investment revenue" in relation to the CGA. The Foundation put forward a recommendation to GAC to exclude realized gains on investments from the calculation to allow these to be used to recapitalize the Endowment Fund. GAC approved that change in 2020.

Overall, the Endowment Fund has provided a stable base of funding, and the Foundation has managed the Endowment Fund well over the past five years. The changes to the grant allocations and calculations allowed the Foundation more flexibility in its funding model while maintaining a strong grants program. However, the evaluation found a lack of clarity in the CGA on the process for making and approving amendments such as those for the grant calculations.   

6. The Foundation has been successful in mobilizing funding outside the Endowment Fund since 2014-15. 

The latest 5-year Strategic Plan published in 2019 projected that the annual expenditure levels would continue to be approximately $6.0 million going forward. To support this level of expenditure required a continuing strong performance by the Endowment Fund and an increasing diversity of outside revenue from government and other sources.

During the evaluation period, an emphasis had been placed on expanding outside revenue sources. The Foundation was successful in this endeavour with the total non-Endowment revenue growing by over 130%. An independent study in 2018 on the potential for fundraising from the private sector indicated that there were limited opportunities. Most of the private sector funding received has been related to co-funding specific events.

The business development approach then focused on obtaining resources primarily from government sources. The proportion of total revenue coming from government sources (Canadian and foreign) grew from 10% in 2015 to a high of 53% in 2018. The funding was received from a wide range of government departments and agencies including GAC. For example, over the past five years, GAC has funded 19 projects for a total of $3.15 million.

Figure 8: Sources of revenue

However, many of these government contracts were of low value with a median value of approximately $25,000. These small contracts resulted in high transaction costs for both the Foundation and the government departments, making the contracts less efficient to administer. The 2016 Strategic Review indicated that small-scale proposals were valuable when they created a runway for future work, but otherwise could have a low return on investment by consuming significant management capacity. In addition, as the scope of Canada's interests and business needs within the region evolves based on emerging priorities, government stakeholders highlighted the lack of flexible funding mechanisms for GAC to swiftly task and fund APF Canada to undertake projects and activities that may not be explicitly supported by the CGA.   

On the other hand, the multi-year contracts sourced by the Foundation provided funding stability for a particular programming area and longer time frames to achieve results. The Kakehashi Project has been in place since 2013; the APEC-Canada Growing Business Partnership has been in place since 2016, and the Women's Business Missions since 2019. For the latter two, counterpart funding was required under the contracts, and only certain costs were covered. This meant that the Foundation needed to use part of the $3.25‑million Endowment Fund allocation to cover the projects. The Foundation's overall budgets for the last three years indicate that contracted projects took from 4% to 12% of in-kind distributions from the Endowment Fund. Projects that are pursued should therefore be strategically selected and provide value to the Foundation beyond simply adding incremental revenue, otherwise there could be opportunity costs where less funding may be available for core products and services or for other initiatives that may be more important to the Foundation.

While the Foundation has been able to generate substantial outside revenue, which supports the intention of the CGA, some trade-offs are seen with the sources and scale of some of the funding.

7. The Foundation has established a strong governance structure, and the Board has been active in key decision making.  

Figure 9: Board membership by sector and gender

The evaluation found that the Foundation's Board members were highly engaged. The Board met biannually with strong attendance. The CGA's requirements for committees and composition have been fully met with the establishment of four committees—Executive; Governance, HR and Compensation; Investment and Revenue; and Audit and Evaluation. There was consistent reporting by the four committees to the Board. In addition, to improve efficiency, the Foundation had made changes to the committee structure, such as combining the Compensation and Governance committees in order to better balance committee membership and offset the gap in government appointments to the Board.

The Board has been active in the development of the new business model, supporting the push for outside funding and approving the operational and programming approaches and the new Strategic Plan. In 2016, an external strategic review was undertaken that provided Board members an opportunity to review feedback from stakeholders and better understand trade-offs in implementing the new approach. The move to the new business model was seen as a way to make the Foundation more sustainable and enhance its impact.

To select new Board members, a Board matrix was developed and implemented. This served to improve representation, diversity, and obtain a spectrum of people on the Board. The resulting Board members are prominent, well connected individuals who are able to promote the Foundation and its work in various circles. Some 50% of the Directors in 2020 were former senior officials of the federal and provincial governments including ex-ministers and an ex-premier. Some 20% were in financial management, and the remaining 30% were from the private sector and academia. Women comprised 38% of the Board membership.

While there have been improvements toward gender and geographic representation on the Board, the Board itself recognizes the potential for further improvement. There is an opportunity for change, such as adding civil society representation or more individuals with Asian backgrounds, as the terms of several Board members will end in 2021.  

Overall, the governance structure has worked well. The Board members provided both guidance and expertise that has proven valuable to the Foundation and its achievement of outcomes to date.   

8. The Foundation's organizational structure has evolved to reflect the new business model, resulting in four high-functioning divisions.

As the business model evolved over the last five years, changes were made to the mix of staff, how they were organized, and the HR policies. A Business Development Team was organized, and a Communications Manager was hired. A new Vice President of Research was selected in 2019 with geopolitical skills to support the broadening focus. HR policies were developed, and an HR Manager engaged. This evolution culminated in a new organizational structure in 2019 that reflected the needs of the current approach. The new structure saw the removal of the Vice President of Operations and Networks position and the shift to four divisions—Office of the President; Research Division; Toronto Office; and Communications Division. Feedback from the staff indicated that, after an initial adjustment to the new structure, it is now working well.

Figure 10: Organizational chart by functions based on May 2020 chart

Office of the President

The President and CEO remains the decision maker for both the strategic planning and the daily operations. The flat organizational structure adopted means that one third of the staff now directly report to the President. Besides the three Division heads, the President also directly supervises the activities around business development, networks and secretariats, grants program, HR and finances.

Research Division

The Research Division is structured in teams based on the Strategic Plan pillars or projects that meet the needs of a specific stakeholder. The senior research staff are responsible for the continuity of the teams' programming. This structure has allowed for flexibility to operate projects independently and create expertise and efficiency within the individual teams. However, the evaluation found that building a research team around the expertise of a specific senior researcher may be difficult for long-term planning when an employee with the specific expertise leaves the Foundation for other opportunities. For example, the research under the Sustainable Asia pillar is an important and relevant area of focus for the Foundation, but major projects under this pillar have slowed down since 2019 as the program manager position remains vacant.

Toronto Office

The Vancouver and Toronto offices run separate programs with Toronto handling two of the largest projects—Kakehashi Project and the Women's Business Missions—as well as other activities. Until 2020, there was coordination between Vancouver and Toronto around specific events or meetings such as ABLAC, but more limited collaboration between the offices in general. With the pandemic, weekly virtual staff meetings were instituted, which staff have described as improving communication and coordination between the two offices.

Communications Division

The Communications Division has developed a brand for the Foundation and rapidly expanded its social media presence. In 2015, the approach to communications was relatively ad hoc and relied on written documents and pamphlets. Since that time, the Division has developed a wide range of communications tools that have been tailored to different audiences and platforms, making the information more accessible. The Foundation has put in place a detailed monitoring system to review performance of a wide range of indicators on a monthly basis.

In general, the evaluation found that the revised organizational structure is well suited to manage and undertake the Foundation's wide range of activities.

9. The grant requirement in the CGA has impacted the Foundation's staffing model, increasing the reliance on young professionals.  

The Foundation's approach to staffing its research program has relied on a small number of senior full-time researchers as well as a large cadre of junior and young professionals who have not done substantive research outside of an academic environment. This was partially driven by the grant program requirement in the CGA. While the turnover of permanent staff has been low, the fixed-term contract and grant recipients have been hired for short terms. Over the evaluation period, the grants program was used to hire 29 Post-Graduate Research Fellows (PGRF) and 56 Junior Research Fellows. While some of those receiving grants have stayed with the Foundation, a majority were there for short periods, sometimes as short as a few months.

The Foundation's grants program has provided the PGRFs with an opportunity to undertake research and gain experience. The program has been popular as demonstrated by the frequency of website traffic on the grants program web page, social media engagement, and the number of applications received. The PGRFs are required to provide program support in addition to their research and were chosen based on the extent to which their proposed research topics complemented the Foundation pillars and products. Prior to the pandemic, the frequency of travel each could undertake decreased. This, combined with the workload of program support, has sometimes resulted in some of the PGRFs being unable to finish research projects. In addition, current and former staff drew attention to the limited opportunities for career progression, prompting their move to the public sector or other organizations with stable, long-term job opportunities.

The staffing model of hiring many young researchers has suited the broad-based approach the Foundation has taken with respect to its programming. These researchers were able to support ongoing programming as well as pursue new areas of research. The Foundation was highly regarded by current and former staff, as well as by other stakeholders, as a training ground for the next generation of Asia Pacific experts. However, with the rotation of numerous young researchers hired only to work on specific projects, the Foundation has struggled with building longer-term institutional capacity and leveraging expertise developed in one project to other areas of the organization. Moreover, with the CGA requirement of $360,000 going to grants, it would be difficult to move to a different staffing and programming model if the Foundation chose to. For example, another option would be hiring more senior experts for specialized programming with fewer junior staff, as seen with other think tanks. With the grants program embedded in the CGA, the staffing options become more limited making it difficult to move to a different staffing model.

The evaluation found that the staffing model generally supports the current approach to programming and provides a unique opportunity for young professionals interested in gaining research experience in Canada-Asia relations. However, the CGA grant requirement makes changes to the staffing model difficult if the Foundation sought to hire additional senior researchers to focus on developing more specialized programming.  

Government relations

10. There is a partial misalignment between the Act and the CGA in terms of independence and oversight.

The Foundation's independence was established in the Act, while the CGA provided the Minister of Foreign Affairs with a number of oversight responsibilities. The evaluation found inconsistencies between stakeholders' understanding of roles and responsibilities with respect to the independence of the Foundation and the oversight responsibilities of Global Affairs Canada.

The Endowment Fund provided through the CGA is subject to the Policy on Transfer Payments, which aims to ensure that "transfer payments are managed in a manner that respects sound stewardship and the highest level of integrity, transparency, and accountability." Under the Directive on Transfer Payments, there are a number of provisions for upfront multi-year funding for which departmental managers are responsible for addressing in the funding agreement. The CGA includes these necessary provisions and provides the Minister a certain amount of oversight responsibilities to ensure compliance with a number of requirements.

Both the Act and the CGA indicate that the Minister must be provided copies of annual reports, strategic plans, corporate plans, audits and evaluations that may be tabled before Parliament. However, neither the Act nor the CGA reference the responsibility of the department to ensure compliance, which would allow GAC to advise the Minister whether or not CGA conditions are being met. The Policy and the Directive on Transfer Payments also do not clearly outline the department's responsibility to ensure compliance with specific provisions. The CGA requires that the Foundation's annual reports include 12 specific areas of information. However, the department has raised questions as to whether GAC can request changes to the annual reports to better comply with the CGA, or if this type of request conflicts with the independence provided in the Act.

The evaluation found that, in a few cases, the Foundation had only partially met the CGA requirements. This may be a result of duplicated or overlapping requirements listed in the CGA. For example, Section 5.11 of the CGA states that the Foundation agrees to carry out an evaluation of its activities by an independent third party every five years, to be approved by the Board, while Section 9.1 requires the Foundation to conduct follow-up evaluations every three to five years. Furthermore, Sections 9.1 and 9.2 provide the Minister with the authority to undertake evaluations at the expense of the department. See Annex E for a list of evaluation requirements as per the CGA.

In response to these requirements, the Foundation has prepared and submitted internal annual evaluations to the Board, as well as a 5-Year Organization and Activities Review published in 2020, meeting some of the CGA requirements. However, the evaluation standards used for internal reports did not meet the Government of Canada's definition of evaluation. In addition, the Foundation developed terms of reference for the Audit and Evaluation Committee. The terms do not clearly define evaluation, instead refer to performance evaluation of the Committee itself rather than the performance of the Foundation's activities. Lastly, the Foundation hired an independent third party to undertake a strategic review in 2016 that looked at areas such as governance, management and performance; however, the final report was not published.

The evaluation also found that the Foundation has only partially met requirements for the content of its strategic plans and annual reports. Both Foundation and GAC stakeholders interviewed identified a need to clearly define the requirements listed in the CGA as well as determine if certain requirements are still relevant and whether they should be embedded within the CGA.

11. The Government of Canada has not consistently fulfilled its commitments to the Foundation under the CGA.

As outlined in the Act and the CGA (and Annex F), the Minister of Foreign Affairs has a number of responsibilities related to governance and reporting. Over the five years covered by the evaluation, the Government of Canada did not fill the designated Governor in Council (GIC) appointments to the Foundation's Board of Directors in a timely manner. Staff and organizational changes at GAC meant that recommendations by the Minister of Foreign Affairs for the appointments were not put forward for approval, which had several implications.

The delayed appointments left gaps in various committees, affecting the Foundation's compliance with the CGA. This led to the Foundation appointing several Board members, which resulted in a large Board once the GIC appointments were made. In July 2019, the Government of Canada made appointments to replace the Chairperson and three Directors through the new open, transparent and merit-based approach to GIC appointments.

Other areas where the department did not effectively fulfill its commitments to the Foundation were related to the appointment of Canadian representatives to certain councils. The chair of the PECC has been in place since 2008 without an official appointment by the Government of Canada. Furthermore, GAC had not appointed all three members to the ABAC during the evaluation period, which has raised questions about Canada's commitment to these councils and general interest in the region. Despite the fact that the department has recently begun the process for appointing additional ABAC members, only one member has been appointed as of July 2021.

The evaluation found that over the past five years, GAC's engagement with the Foundation has been inconsistent. The liaison or oversight role in the department has changed over time, moving from one branch to another, based on individual interest or experience with the Foundation, rather than on a defined role. GAC staff were therefore not well versed on the conditions of the Endowment Fund and the number of contracts into which the Foundation has entered. Moreover, since 2011-12, GAC no longer reported on the upfront multi-year funding provided to the Foundation in its annual Departmental Results Report.

Lastly, there was no internal Government of Canada coordination regarding the funding strategy for the Foundation. There was no mechanism within the Government of Canada or particularly within GAC to collectively monitor the contracts granted to the Foundation. Government stakeholders interviewed were unaware of the number or total value of government contracts the Foundation had, despite the information being publicly available through government open-source data as well as through the Foundation's annual reports. The absence of a whole-of-government funding strategy for the Foundation has resulted in the funding of multiple one-off projects. At the same time, several GAC employees experienced barriers when trying to arrange for additional funding to the Foundation, outside of the CGA, to swiftly finance new opportunities within the region. Those interviewed expressed a need for more flexibility in the funding arrangement with the Foundation to allow for the expedient support of new and emerging priorities.

12. The extent to which the Foundation's work should be consistent with the Government of Canada's priorities is not clearly understood and may have impacted the Foundation's independence.

A wide diversity of needs for policy research and advice exists within the Government of Canada. While the Foundation has found success in delivering some policy products, especially under specific government contracts, the overall lack of clarity regarding the expectations for the Foundation's policy work made it difficult to assess the extent to which the government's and other needs were being met.

In order to better meet needs, the CGA specifies that the "Foundation's Strategic Plans must be consistent with the Government of Canada's priorities with regard to the Asia Pacific region." The evaluation found an expectation by government stakeholders that the Foundation undertake research and provide policy advice for their consideration. However, federal government stakeholders have expressed either unfamiliarity with the Foundation's recent policy products or views that the products had not met their needs and that they were likely to consult other think tanks working on Asia Pacific issues.

Stakeholders interviewed, including APF Canada and GAC staff, noted the difficulty with striking a balance between being consistent with Government of Canada priorities and developing policy work that may or may not align with the government's current views. There was an understanding by stakeholders that the Foundation was independent and able to present policy options that differ from the current government's views. However, the Foundation has experienced a critique of its work when an opposing view was presented.

Both the Foundation and the Government of Canada have made their strategic priorities publicly available to a certain extent. The CGA includes a provision in Section 5.6 for the Foundation to develop and implement a results-based Strategic Plan in consultation with its clients and stakeholders. However, the evaluation found that the Foundation's strategic plans were developed internally without significant consultations with GAC. Government stakeholders interviewed stated a lack of understanding of or familiarity with the Foundation's current strategic plan and priorities exists, despite the fact that the Foundation published its priorities, including overall strategic pillars, on its website and within annual reports in both official languages. By comparison, the Government of Canada's priorities for the Asia Pacific region are available through public documents such as Mandate Letters and annual Departmental Plans. However, GAC management and staff acknowledged that the department's priorities for the region have been dynamic and could change rapidly, making them difficult to align with. The Foundation has expressed a desire for more input from the Government of Canada regarding the need for policy work.

13. There has been no formal mechanism for communication, consultation or coordination between the Foundation and the Government of Canada or Global Affairs Canada.

The Foundation maintained bilateral relations with various funding agencies and departments based on individual contracts. Working level communication lines were well established for contracted programs, such as the APEC-Canada Growing Business Partnership, as well as for specific working relationships such as ABAC and the Canada-China Track II Energy Dialogue. Interviewees reported particularly strong relationships between the Foundation and other government departments and agencies such as ISED and Invest in Canada.

In regard to the management of overall consultations between the Foundation and the Government of Canada, the evaluation found no formal mechanism for communication. Strategic communications surrounding priorities and program development over the past five years were described as ad hoc, transactional and informal. There are currently a number of former GAC employees working for the Foundation or on the Board, which has led to strong relations between the executive levels based on past, professional connections, which in turn has helped to facilitate some consultations and disseminate publications.

Some government stakeholders expressed a desire for better coordination and alignment of priorities. For example, some interviewees expressed concern over the Foundation using its own connections to obtain endorsements for a project before approaching GAC or submitting a proposal. There was also concern over potential duplication of efforts and an overreliance on GAC resources. These concerns have caused tension at times between the department and the Foundation. For example, the Women's Business Mission pilot project in Japan was funded by the Foundation but relied heavily on GAC's Trade Commissioner Service for implementation, requiring mission resources to assist the Foundation, which was not initially planned for by GAC. Additional Virtual Women's Business Missions in South Korea and Taiwan were funded by ISED and had received varying assistance from the Trade Commissioner Service.

The Foundation's senior management and Board have articulated a desire to have a GAC ex-officio member on the Foundation's Board to help improve coordination, collaboration and communication with the Government of Canada. While there are provisions in the Act and the CGA permitting federal public servants to serve on the Board as non-voting members, stakeholders interviewed expressed mixed views. On one hand, there were views that having a GAC representative on the Foundation's Board would improve the department's ability to exercise oversight and report to the Minister. Some felt that it would be beneficial in terms of better understanding the government's priorities. In fact, there was a GAC representative on the Board up until 2016. However, the department has recently decided not to permit a representative to be on the Board due to a potential or perceived conflict of interest. Some interviewees had concerns that the presence of a government representative could indirectly influence Board decisions and not allow an arms-length relationship to be maintained. It should be noted, however, that the department's decision was inconsistent as there are instances of GAC representatives on boards of organizations that are at arms-length and receiving GAC funding, such as the Centre for International Governance Innovation. To help clarify any issues, the department has recently issued an internal directive for assessing participation by employees on the committees and boards of external entities in 2021.

Despite the varying forms of engagement and differing views on communication mechanisms, there was a desire on both parts to improve overall communication and establish clearer roles and responsibilities.

Conclusions

With the arrival of a new President and CEO in 2014, a business model was developed with the Board that moved the organization toward a more action-oriented approach that could serve a larger audience. The Board has done an effective job of governing the Foundation including being actively involved in the changes. The organizational structure now reflects the new approaches being taken including the establishment of a business development team and research staff organized around the new pillars of the Strategic Plan 2019.

During the evaluation period, the Foundation produced solid results that fully align with and were supportive of the CGA outcomes. Its work remains relevant to a broad range of stakeholders including government, business, academia and the general public. The Foundation implemented a series of activities and programs that contributed to increased awareness and understanding of the Asia Pacific region, including education curricula, youth exchanges, National Opinion Polls, Asia Watch, and the Investment Monitor. It has played a positive convenor role through the creation of ABLAC, the secretariat support provided to ABAC and PECC, and the series of women's business missions. These, and other initiatives, have contributed to building networks and ties between Canada and the Asia Pacific region.

With six years of implementation of the business model, a number of trade-offs need to be considered going forward. First, the programming covers a wide variety of areas. Given the inherent limitations of the budget, many stakeholders wondered whether the breadth of the areas being covered was too large for the budget. Second, many of the contracts sourced have been small with a high transaction cost related to their management. On the other hand, while multi-year contracts can provide stable funding, they often require counterpart funding or additional funding from the Endowment revenue. This means that these projects should be strategically selected to ensure they provide a value to the Foundation beyond simply incremental revenue. Third, the CGA requires that 25% of the Endowment revenue generated is used for the grants program. During the period covered by the evaluation, these funds were used primarily for hiring a large number of Post Graduate Research Fellows and Junior Research Fellows. With the grants program embedded in the CGA, changing the staffing model to hire senior researchers and experts was found to be a limitation for the Foundation. However, the Foundation has established a network of Distinguished Fellows, which if fully utilized, could provide a strong pool from which the Foundation could draw expertise and further strengthen its research programming.

While the CGA provides a stable base of funding and outlines a broad mandate that allows for adaptability in programming, there remain areas of ambiguity that require more clarity. This includes clarifying the difference between the Foundation's independence and the Government of Canada's oversight responsibilities. A question also arises in terms of the type of policy support the government is expecting from the Foundation as per the CGA, as well as the ability for the government to quickly provide funding to the Foundation for the execution of activities and projects that are not within the provisions of the CGA. The lack of a flexible funding mechanism within the Asia Pacific Branch was seen as the reason for missed opportunities for Canada to be involved in certain initiatives or get more visibility. In the absence of more flexible funding options, the department uses small contracts, which in turn lead to high transaction costs and increase the administrative burden for both sides.

Another area identified for improvement was the need for the Government of Canada to fulfill its commitments to the Foundation under the Act and the CGA. The GIC Board appointments have been slow and, at one point, left a one-year gap in the Chairperson position. The ministerial appointments of the ABAC representatives, for which the Foundation provides support under the CGA, have also been slow. The liaison or oversight role at GAC has changed branches and staff a number of times. New staff who have taken over were unfamiliar with the CGA and their role with regard to the Foundation. In addition, no internal Government of Canada mechanism exists to coordinate additional funding to the Foundation. 

Lastly, the communications between the Foundation and various government departments have been informal or pertain to specific contracts or programming. Strategic communications regarding each other's priorities or program development over the past five years were described as ad hoc and transactional. Both GAC and the Foundation have expressed a desire to establish more formal communication channels.

Considerations

The Asia Pacific Branch at GAC launched a concurrent review of the Conditional Grant Agreement conducted by the external firm KPMG. The purpose of the review was to update and renew the Terms and Conditions of the CGA and to offer greater funding flexibility and reflect the arm's length relationship between the Foundation and the Government of Canada as stipulated in the Asia-Pacific Foundation of Canada Act. The evaluation and review were coordinated to the extent possible to leverage synergies and avoid duplication. The following considerations and recommendations are therefore to be actioned in conjunction with those arising from the CGA review.

1. The evaluation suggests that during consultations to revise the CGA, the Foundation and GAC also discuss the value of continuing to embed the grants program in the CGA. The CGA requires the grants program to be funded through 25% of the revenue of the Endowment Fund. This has implications for both the Foundation's staffing model and the availability of funds to undertake other activities and projects. As part of the CGA Review process, the issue of continuing the requirement for the grants program should be considered. Linked to Findings 5 and 9.
2. The evaluation suggests that when making recommendations for upcoming Board appointments, both the Foundation and GAC consider applying a diversity and inclusion lens to their respective Board or GIC matrices and aim to increase the diversity of Board composition, including more civil society representation and individuals with an extensive knowledge in Canadian-Asian relations. Linked to Findings 7 and 11.

Recommendations

APF Canada

1. Undertake a strategic review of the current business model. In the context of the CGA review, a new President and CEO appointed in 2021, and new Board members expected to be appointed, a stock-taking opportunity exists to determine whether changes should be made to enhance the overall impact of the Foundation. Linked to Findings 1, 3, 6, 8, 9.
2. Be proactive in engaging the Distinguished Fellows. The cohort of Distinguished Fellows associated with the Foundation offer a wide range of expertise and regional experience. Methods should be developed to ensure that the Fellows are proactively engaged in supporting the Foundation's work. Linked to Finding 3.

Global Affairs Canada

3. Clarify the department's oversight roles and responsibilities, or authority framework, with respect to the Foundation. Linked to Finding 10.
4. Develop a funding framework for GAC to allow more flexibility and timely support of APF Canada to respond to additional departmental needs that fall outside of the CGA. As the department's priorities for the region evolve, flexibility in the funding arrangements is needed to facilitate contracting processes and reduce the transaction costs and administrative burden for GAC. Linked to Findings 4, 6 and 11.

APF Canada and Global Affairs Canada

5. Develop an operations and communications plan to address requirements of the Act and the CGA, in particular:
   • Develop a plan to address the legal requirements of the Act, particularly to address GAC's duty to consult the Board and ensure an appropriate timeline for the process of GIC appointments, as well as other Ministerial appointments. The current four GIC appointments for the Foundation's Board will be up for renewal or replacement by July 2022. By establishing a suitable timeline for the appointment process, new GIC Board members could be appointed as soon as the terms of the current members end, ensuring a smooth transition. Linked to Finding 11.
   • Develop a communication and consultation mechanism between the Foundation and the Government of Canada. For example, Board materials could be shared, a Memorandum of Understanding could be developed, or annual consultation meetings could be held with key Government of Canada departments and the Foundation to supplement the current informal channels of communication. The purpose is to agree on appropriate methods or forums for both sides to regularly exchange information and discuss plans and priorities. Linked to Findings 12 and 13.

Annex A: Detailed methodology

The following annex provides additional information on evaluation methods used.

Interviews

In total, 64 individuals were interviewed from a variety of stakeholder groups. A breakdown of engagement is illustrated below.

Interviews conducted by stakeholder group*

*Some individuals fell into more than one category but were only included in a single category for this chart.
**The Other category includes APF Canada Distinguished Fellows, private sector representatives, and other experts.
***APF Canada Board and Staff categories include both current and former members.

Document review

• The document review consisted of examining over 300 documents. See Annex H: List of Sources for a summarized list of documents.

Case studies

• "Building Blocks for a Canada-Asia Strategy" and "Canada and the Indo-Pacific: 'Diverse' and 'Inclusive', not 'Free' and 'Open'" were used as representative publications of APF Canada research.
• The case studies examined the background, impetus, target audience, timing, messaging and issuing framing, communications strategy and impact of the paper. The overall goal was to determine the degree of policy influence that can be attributed to the Foundation.

Comparative analysis

See Annex D for a detailed explanation of this methodology.

Review of APF Canada social media analytics

• Social media analytics were provided by APF Canada, which tracked social media statistics, such as the number of followers and engagement rates over time.
• Tracking was conducted for the Foundation's website, Facebook, Twitter, LinkedIn, YouTube and Instagram accounts as well as breakdowns of subscribers to Asia Watch.

Backlink analysis

• The backlink analysis was conducted through Ahrefs (www.ahrefs.com). A report was created from the Foundation's web page (www.asiapacific.ca) to list all websites that linked to APF Canada pages.
• Each linking website was categorized by stakeholder type such as Academia, Business, Government, General Audience, Think Tank, Social Media, Media, or Unknown. A pivot chart was created to summarize the stakeholder breakdown.
• Through this methodology, insights were gleaned regarding stakeholder demographics as well as the reach and influence of the Foundation.

Annex B: Organizational chart

This organizational chart was adapted from the Foundation's internal organizational chart from May 2020. Names of individuals have been removed.

Annex C: Evolution of Strategic Plans

Annex D: Comparative analysis detailed methodology

The think tanks were chosen for the analysis based on the following characteristics:

• some focus on the Asia Pacific region
• similar keywords as per the Ahrefs Competitive Analysis Report
• suggestions from interviewed individuals

Once comparable think tanks were selected, the following indicators were chosen:

Due to variation in each think tank's organization size and budget, the results were scaled to make comparisons. The annual operating expense was chosen as the indicator for organization size and was taken from each think's annual report, web page, or third-party website such as Charity Navigator. Two of the identified think tanks did not publish their annual operating expenses, resulting in them being removed from consideration in the final analysis.

Results of Comparative Analysis (data from January 2021)

Annex E: Evaluation requirements table

Annex F: List of government responsibilities as per CGA

Annex G: List of finding statements

1. The Foundation has adapted to an evolving context, developing new and relevant programming aligned with its broad mandate.
2. The Foundation is valued for its activities and programs that have contributed to increased awareness and understanding of the Asia Pacific region.
3. The Foundation's numerous publications have not translated into a high global think tank ranking.
4. The Foundation has developed a positive convenor role through networking activities intended to contribute to developing closer ties between the peoples and institutions of Canada and the Asia Pacific region.
5. Over the past five years, the Foundation has managed the Endowment Fund well and made prudent financial decisions.
6. The Foundation has been successful in mobilizing funding outside the Endowment Fund since 2014-15. 
7. The Foundation has established a strong governance structure, and the Board has been active in key decision making. 
8. The Foundation's organizational structure has evolved to reflect the new business model, resulting in four high-functioning divisions.
9. The grant requirement in the CGA has impacted the Foundation's staffing model, increasing the reliance on young professionals.
10. There is a partial misalignment between the Act and the CGA in terms of independence and oversight.
11. The Government of Canada has not consistently fulfilled its commitments to the Foundation under the CGA.
12. The extent to which the Foundation's work should be consistent with the Government of Canada's priorities is not clearly understood and may have impacted the Foundation's independence.
13.  There has been no formal mechanism for communication, consultation, or coordination between the Foundation and the Government of Canada or Global Affairs Canada.

Annex H: List of Sources

Program Background

• Amendment to the Conditional Grant Agreement between the Minister of Foreign Affairs and the Asia Pacific Foundation of Canada. 2010. (CGA Amendment)
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Annual Reports and Annexes from FY 2014/15 to FY 2019-20
• APF Canada Annual Activities and Work Plans from FY 2014/15 to 2019-20
• APF Canada Audit Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada staff meeting presentations 2020
• APF Canada website
• Asia-Pacific Foundation of Canada Act (R.S.C., 1985, c. A-13). 1985. (APF Canada Act)
• Conditional Grant Agreement between the Minister of Foreign Affairs and the Asia Pacific Foundation of Canada. 2005. (CGA)
• Financial information provided by GAC including emails and contract information
• Interviews with APF Canada staff and Board, GAC and OGD staff and management
• Policy on Transfer Payments. 2018. Treasury Board of Canada
• Public sources for contracts with OGDs
• Summative evaluation of the Asia Pacific Foundation of Canada. 2009. DFAIT

Introduction to Findings

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Act
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada Framework for a Strategic Plan (2014-19)
• APF Canada staff meeting PPTs 2020
• CGA and CGA Amendment
• Communications Audit: APF Canada, prepared by Weber Shandwick in 2017
• Compton Study on Private Sector Philanthropic Fundraising 2018
• Draft Asia Pacific Foundation Strategic Review, prepared by McKinsey & Company in 2016
• Interviews with APF Canada staff, Board, and GAC staff and management

Finding # 1

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review: 2015-2020 – Main Report
• APF Canada Act
• APF Canada Framework for a Strategic Plan (2014-19)
• APF Canada website
• CGA
• Interviews with GAC staff and management, APF Canada staff and Board, Distinguished Fellows, Grantees and external stakeholders

Finding # 2

• APF Canada 5-Year Organization and Activities Review: 2015-2020.
• APF Canada Annual Reports from FY 2014/15 to FY 2019-20
• APF Canada's Asia Watch survey 2020
• APF Canada's media analysis 2014-20
• APF Canada website
• Case study:
  • Building Blocks for a Canada-Asia Strategy. 2016.
  • Building Blocks Report communications, presentations and media outreach information
  • Gardiner, A., and Brindis, C., 2017. Advocacy and Policy Change Evaluation: Theory and Practice. Stanford, California: Stanford University Press
  • Influencing Public Policy: An Analysis of Published Op‐Eds by Academics. 2008.
  • Interviews with GAC staff and management, APF Canada and outside stakeholders
  • "Now is the right time for Canada's return to Asia," Hugh Stephens and Deanna Horton, Toronto Star, published on May 25, 2016.
  • Stewart Beck's 2018 statement at the Foreign Affairs and International Development Committee
  • Mayne, R., et al. "Using evidence to influence policy: Oxfam's experience." Palgrave Communications volume 4, Article number: 122 (2018).
• Evaluation media analysis and SEO tool on Ahrefs.com
• Investment Monitor Reports 2019 and 2020
• Interviews with GAC staff and management, APF Canada staff and Board, Distinguished Fellows, and external stakeholders
• Kakehashi Project Annual Report 2019-20
• National Opinion Polls 2014, 2016, 2018, 2020 and specialty reports

Finding # 3

• Global Go to Think Tank Index Report, 2020
• APF Canada's media analysis 2014-20; Evaluation comparative analysis, SEO tool on https://ahrefs.com/ and http://scholar.google.com
• Evaluation media analysis
• Interviews with AFP Canada staff, Fellows, GAC staff and management, and external stakeholders

Finding # 4

• APEC Business Advisory Council. 2017. Founding and Structure. Retrieved from https://www2.abaconline.org//page-content/2521/content
• APF Canada 5-Year Organization and Activities Review: 2015-2020
• APF Canada Annual Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada's media analysis
• APF Canada website
• Communications Audit: APF Canada, prepared by Weber Shandwick in 2017
• Draft Asia Pacific Foundation Strategic Review, prepared by McKinsey & Company in 2016
• Evaluation's media analysis
• Interviews with GAC, APF Canada staff and Board, Fellows, and outside stakeholders
• Media coverage for the ISED-funded Women's Business Missions

Finding # 5

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Act
• APF Canada Annual Reports and Annexes from FY 2014/15 to FY 2019-20
• APF Canada Annual Activities and Work Plan from FY 2014/15 to 2019-20
• APF Canada Audit Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada Staff meeting PPTs 2020
• CGA
• Compton Study on Private Sector Philanthropic Fundraising, 2018
• Draft Asia Pacific Foundation Strategic Review, prepared by McKinsey & Company in 2016
• Ellement Consulting Group Portfolio Reports
• GAC internal documents and emails
• Interviews with APF Canada staff and Board and GAC staff and management
• Investment and Revenue Committee Terms of Reference
• Investment Monitoring Report as at March 31, 2015, by Marsh and McLennan
• Statement of Investment Policies and Procedures – 2019

Finding # 6

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Act
• APF Canada Annual Reports and Annexes from FY 2014/15 to FY 2019-20
• APF Canada Audit Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada staff meeting PPTs 2020
• CGA
• Compton Study on Private Sector Philanthropic Fundraising, 2018
• Financial information provided by GAC including emails and contract information
• Interviews with APF Canada staff and Board, GAC and OGDs
• Public sources for contracts with OGDs

Finding # 7

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Act
• APF Canada Activity Reports 2015-18
• APF Canada Annual Reports and Annexes from FY 2014/15 to FY 2019-20
• APF Canada Audit Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada Bylaws
• APF Canada Terms of Reference for Board Committees
• CGA
• Compton Study on Private Sector Philanthropic Fundraising, 2018
• Draft Asia Pacific Foundation Strategic Review, prepared by McKinsey & Company in 2016
• Interviews with APF Canada staff and Board and GAC staff and management

Finding # 8

• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada Annual Activities and Work Plan from FY 2014/15 to 2019-20
• APF Canada Annual Reports and Annexes from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2014 to 2020
• APF Canada Media and Social Media Reports and data
• APF Canada Staff meeting PPTs 2020
• Communications Audit: APF Canada, prepared by Weber Shandwick in 2017
• Compton Study on Private Sector Philanthropic Fundraising 2018
• Draft Asia Pacific Foundation Strategic Review, prepared by McKinsey & Company in 2016
• Interviews with APF Canada staff and Board, Fellows, Grantees and external stakeholders

Finding # 9

• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Audit Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2015 to 2020
• Interviews with APF Canada staff and Board, GAC staff and management, Grantees and external stakeholders

Finding # 10

• 5-Year Strategic Plan for the Asia Pacific Foundation of Canada, November 2019
• APF Canada 5-Year Organization and Activities Review – 2015-2020 – Main Report and Annexes
• APF Canada Act
• APF Canada Annual Reports from FY 2014/15 to FY 2019-20
• APF Canada Biannual Board Books from 2015 to 2020
• APF Canada Framework for a Strategic Plan (2014-19)
• CGA
• Interviews with AFP Canada staff and Board and GAC staff and management
• Treasury Board Transfer Payment policy and directive. 2009
• Treasury Board Policy on Results. 2016

Finding # 11

• APF Canada Act
• CGA
• GAC internal memos, directives and emails
• GAC Departmental Results Reports (2009 to 2020)
• Governor in Council and Ministerial Appointments: Global Affairs Canada web page
• Interviews with APF Canada staff and Board and GAC staff and management

Finding # 12

• APF Canada 5-Year Organization and Activities Review – 2015-2020, Main Report and Annexes
• APF Canada Act
• Case Study:
• APF Canada website
• APF Canada. "Canada and the Indo-Pacific: 'Diverse' and 'Inclusive', not 'Free' and 'Open'". 2020
• Jeffrey Reeves. 2020. "Canada and the Free and Open Indo-Pacific: A Strategic Assessment." Asia Policy, v. 15 n.4.
• The Free and Open Indo-Pacific: Charting a Common Approach, Conference Notes. 2020.
• Interviews with APF Canada staff and Board, GAC staff and management, and external stakeholders
• CGA
• Interviews with APF Canada staff and Board and GAC staff and management
• Minister of Foreign Affairs Mandate Letters (2019, 2021)

Finding # 13

• APF Canada Act
• CGA
• GAC internal memos, directives and emails
• Interviews with APF Canada staff and Board, GAC and OGD staff and management

Considerations

• KPMG. 2021. CGA Amendment Business Case, Rationale and Recommendations
• KPMG. 2021. Terms of References for Review of Conditional Grant Agreement