Foreign Affairs, Trade and Development Canada
Symbol of the Government of Canada

Foreign Affairs, Trade and Development Canada

international.gc.ca

Summative Evaluation of the Asia Pacific Foundation of Canada

March 2009

(PDF Version, 513 KB) *

Table of Contents

Abbreviations and Acronyms

ABAC
APEC Business Advisory Council
APEC
Asia Pacific Economic Cooperation
APF
Asia Pacific Foundation of Canada
ASEAN
Association of Southeast Asian Nations
BoD
Board of Directors
CAPRN
Canada-Asia Pacific Research Network
CCBC
Canada China Business Council
CIC
Canadian International Council
CIDA
Canadian International Development Agency
CIGI
Centre for International Governance Innovation
CEO
Chief Executive Officer
CGA
Conditional Grant Agreement
CPRN
Canadian Policy Research Networks
DFAIT
Department of Foreign Affairs and International Trade
FY
Fiscal year
GoC
Government of Canada
IT
Information Technology
OIC
Order in Council
PECC
Pacific Economic Cooperation Council
US
United States
ZIE
Evaluation Division, Office of the Inspector General

Executive Summary

Context and Program

The Asia Pacific Foundation (APF) of Canada was established by an Act of Parliament in 1984. The original mandate was to develop closer ties between the peoples and institutions of Canada and Asia-Pacific and to promote capacity development and network building among persons and entities that share an interest in the Asia Pacific region.

In 2005, the Government of Canada (GoC) agreed to a $50 million conditional grant for the establishment of an endowment fund. The intention was to allow the Foundation to have a more predictable funding stream and a more arms-length relationship with the GoC.

A Conditional Grant Agreement (CGA) was put in place that targeted two results.

  • Canadian businesses, scholars, and citizens have a good understanding of, and strong relations with, people and institutions of the Asia Pacific Region.
  • Organizations, institutions and associations in private and public sectors in Canada and the Asia Pacific region collaborate, forming strong networks and relationships.

A Strategic Plan 2005-2008 was developed by the Foundation which set out three priorities.

  • Producing Knowledge that Matters - The Foundation would continue to focus on value-added information and analysis on Canada-Asia relations. While starting with a broad-based approach, a shift was made in 2007 to focusing more specifically on three research and policy themes: Asia Pacific Gateway; Canadians Abroad; and the Impact of Global Asia on Canada.
  • Building Networks in Canada and Across the Pacific - Under the Strategic Plan, the Foundation targeted a broad promotion of networks. According to the CGA, support was to be provided to existing networks such as the Pacific Economic Cooperation Council (PECC), APEC Business Advisory Council (ABAC), and the APEC Study Centre. A grants program was also to be developed which was to take an increasing proportion of revenue from the endowment. The networks and grants programs were brought into greater alignment with Foundation priorities starting in 2007. In addition to these mandated programs, the Foundation has placed priority on seeking new innovative networks with which to work in North America and Asia.
  • Increasing Public Awareness - The awareness objectives were originally very broad based, targeting the general public. In recent years, awareness activities have focused on providing value added information to Asia practitioners -- individuals who have a professional interest in Canada-Asia relations. A wide range of products and services operate here from a redeveloped website, publications, media coverage, roundtables, speeches, conferences, and public opinion surveys.

With its new positioning, the Foundation now functions as a knowledge broker, bringing together people and knowledge to support Canada's transpacific relations.

Evaluation

One of the conditions of the CGA was that an evaluation be undertaken by DFAIT within three years of signing the Agreement. The Evaluation was carried out by the Evaluation Division (ZIE) between December 2008 and February 2009.

The Evaluation looked at a wide range of issues. It assessed the relevance, efficiency and economy of APF's programs and activities in terms of meeting the priorities identified in the Conditional Grant Agreement. It also assessed the achieved results and impact of APF's activities in promoting strong relations and networks with people and institutions of the Asia Pacific Region.

Findings

Relevance

APF continues to fill a unique role within Canada on Asia Pacific relations. While there are a growing number of institutions globally dealing with Asian issues, the Foundation provides an important link between these issues and Canada.

The shift to an endowment and the establishment of the Conditional Grant Agreement has allowed the Foundation more flexibility to fulfill its mandate. Longer-term programming is now possible. The relationship between the Foundation and GoC has become fully arms-length, allowing the Foundation more independence.

The Foundation's activities and programs continue to be consistent with the priorities of the Government of Canada and DFAIT. While the APF now weighs a series of factors in developing its programming, it continues to provide direct and indirect support to the GoC's interests in Asia Pacific.

The recent narrowing of focus by the APF is seen by stakeholders as important for improving its effectiveness in the future and strengthening the ability to support its mandate. The original broad-based mandate was no longer feasible given the shifting nature of information and methods for policy influence. A more focused approach was required.

Effectiveness

Some of the new initiatives under the research themes have been highly effective in influencing policies and issues around Canada-Asia Pacific relations. The support to the Gateway initiative, for example, has produced solid results in terms of facilitating greater coordination among stakeholders and expanding the policy issues under consideration.

A balance has been struck in the efforts to support the various Secretariats and networks mandated under the CGA, with participants positive about APF contributions. The restructuring of the grant arrangements is providing more strategic support for the Foundation's work. By more closely linking the grant research to Foundation priorities, the overall effect has been greater coverage of the issues and greater leverage of the research in policy dialogues.

Specific activities such as conferences, roundtables, and speeches are seen to be highly effective. Overall, however, there is limited understanding and visibility of APF's programs. While a narrowing of APF's focus to Asia practitioners is appropriate, there is a need to better define what that means and how best to reach the market. This will include a better understanding of the groups being targeted, greater insights into issues that are of interest to them, and the development of new approaches to facilitate interaction. This could also include greater collaboration with other organizations.

Governance

The expansion of the Board, and its new composition, has improved the overall governance structure. However, some issues remain in terms of how to better engage the Board members in decision-making and APF activities. The potential contributions of the Board to the Foundation's work are not being maximized.

The new management structure, including greater use of outside resources, makes sense in terms of ensuring flexibility for the organization. However, it will be important to maintain a critical mass of in-house resources to effectively fulfill the vision of a knowledge-based organization.

Economy and Efficiency

The endowment has not provided the expected predictability of funding due to the current financial crisis. This has also impacted contributions from outside sources. However, the Board and management are dealing well with the issues and streamlining programs and activities to match the short term financial reality.

The Foundation has developed an interesting funding model for its research themes. By sourcing funds from a range of stakeholders, the initiatives can be more comprehensive and the funders have a greater stake in supporting policy and dialogue results. The next few years should determine its broader feasibility.

The Foundation has not placed enough emphasis on measuring performance and feeding this back into decision making. The focus in this area has been on compliance, not on developing methods for assessing performance that can assist decision making on priorities and future focus.

Conclusions

This Evaluation has provided an opportunity to review the work of the Foundation and the changes it has begun to implement. APF has taken major steps in its organizational and programming transformation over the last several years. While the transition is still a "work in progress," the initial indications are positive for the future. With some adjustments, the new approach has potential for increasing the Foundation's influence and funding opportunities.

Recommendations

Recommendation #1:

The Foundation should undertake a review of its approach to the Asia practitioner "market7"and develop a new strategy for effectively building networks and awareness. To implement such a strategy may involve changing elements such as the Website to make it more interactive and developing new approaches to networking.

Recommendation #2:

Enhanced collaboration with other groups in Canada and Asia Pacific should increasingly be considered as a method to increase influence and outreach. Greater collaboration will allow leverage of not just funding but ideas and networks that should contribute to raising the profile of the Foundation.

Recommendation #3:

The Foundation should continue improving the composition of the Board with this next round of appointments. This includes increasing the number of members who are specialists in areas where the Foundations works. The Board also needs to develop new methods for engaging the members in decision making and supporting the work of the Foundation in terms of networking or financial leverage.

Recommendation #4:

A new approach to assessing performance needs to be developed that is more closely linked to decision-making processes within the Foundation. Monitoring performance is an important aspect for testing whether the approaches being taken are reaching the right audiences and having the type of influence anticipated. Performance measurement can be done in a way that continues to meet the requirements of the CGA while providing more useful information for decisions.


1.0 Introduction

1.1 Program Background

1.1.1 Original Mandate

The Asia Pacific Foundation of Canada (APF) is an organization focusing on the provision of value-added information and analysis on contemporary issues in Asia that affect Canada. It was established by an Act of Parliament in 1984. The Foundation is a registered charity and has its headquarters in Vancouver, British Columbia.

The original purpose of the Foundation, as stipulated in the 1984 Act, was to: develop closer ties between the peoples and institutions of Canada and the Asia Pacific region; and to promote capacity development and network building among persons and entities that share an interest in the Asia Pacific region. This was to be accomplished through:

  1. Promoting mutual awareness and understanding of the cultures, histories, religions, philosophies, languages, life styles and aspirations in the Asia Pacific region and Canada, and their effects on each other's societies;
  2. Promoting dialogue on, and understanding of, foreign policy issues as they relate to Canada and the Asia Pacific region;
  3. Supporting development cooperation between organizations, institutions, and associations in Canada and the Asia Pacific region;
  4. Promoting collaboration among organizations, institutions, and associations in private and public sectors in Canada and in the Asia Pacific region;
  5. Promoting closer economic and commercial ties between Canada and the Asia Pacific region;
  6. Promoting, in Canada, scholarship in and expertise on economic, cultural, social and other subjects relating to the Asia Pacific region, and in the Asia Pacific region, scholarship in and expertise on economic, cultural, social and other subjects relating to Canada; and
  7. Collecting information and ideas relating to Canada and the Asia Pacific region and disseminating such information and ideas within Canada and the Asia Pacific region.

1.1.2 Endowment Support from the Government of Canada

In 2005, the Government of Canada committed to the renewal of the Foundation by providing a $50 million conditional grant for the establishment of an endowment fund. By providing this grant, the Government reiterated its recognition of the APF's ability "to engage a broad cross section of Canadian society, including academia, the business community and the general public." Note 1.

The rationale for the establishment of the Endowment was to ensure some predictability to the Foundation's finances and allow for longer-term planning. The aim was to provide the Foundation with the opportunity to sharpen its focus on priority issues in the Canada-Asia relations, strengthen networks across Canada, build institutional capacity, and develop new modes of delivery and dissemination. The conditional grant agreement (CGA) also set a number of changes to the focus, operations, and governance of the Foundation.

Specifically, the CGA was intended to enhance the Foundation's capacity to achieve two results.

  • Canadian businesses, scholars, and citizens have a good understanding of, and strong relations with, people and institutions of the Asia Pacific Region.
  • Organizations, institutions, and associations in private and public sectors in Canada and the Asia Pacific region collaborate, forming strong networks and relationships.

1.2 Current Strategic Approaches

One of the conditions of the CGA was the development of a Strategic Plan that would outline the priorities for three to five years, results being targeted, and the risks facing the Foundation. The Strategic Plan 2005 to 2008 was developed and has been updated since that time through yearly corporate plans and work plans.

In the Strategic Plan, three priorities were established for the Foundation. Since 2005, the areas of focus within these three priorities have shifted and become more streamlined. A summary of the original approaches and the current focus are outlined below.

1.2.1 Producing Knowledge that Matters

In the Strategic Plan 2005-2008, the Foundation indicated that it would continue to focus on value added information and analysis on Canada-Asia relations. The initial intent was to focus on the totality of relations including economic, political-security, and people to people linkages. The initial geographic priorities were set on Japan, China, and India, with a secondary emphasis on Korea and Association of Southeast Asian Nations (ASEAN) countries.

In 2007, the focus of the activities was narrowed in order to maximize the influence the Foundation could have with its current level of funding. In particular, a number of specific changes to the original Strategic Plan were proposed by the Co-Chief Executive Officers (CEOs) of the Foundation and approved by the Board.

  • The focus for the major research would now be on three specific areas: Asia Pacific Gateway; Canadians Abroad; Note 2. and the Impact of Global Asia on Canada.
  • In addition to the three areas, the Foundation would retain flexibility to respond to fast-breaking policy issues and "hot topics."
  • The Foundation would use Senior Fellows, contract researchers, and other partnerships to maximize its impact in terms of research and knowledge sharing.
  • The Foundation would also use its funds as a base for larger contributions from external sources.

The focus of the research on the Asia Pacific Gateway was a series of questions such as: What policies are needed to advance the Asia Pacific Gateway and Corridor Initiative? How does the Foundation create value added within and around the Gateway? What about the Atlantic Gateway? The specific topics that were researched include: sustainable Gateway; building a Gateway economy and industries (in conjunction with Western Economic Diversification); and security dimensions of the Gateway.

The Canadians Abroad component is focusing on issues such as: What is the size and scope of the Canadian community abroad, especially in Asia? What are the benefits and costs associated with transnational Canadians? How should governments organize themselves to deal with issues related to Canadians abroad? A set of studies on the policy implications of the estimated 2.7 million Canadians living outside Canada is being undertaken. A policy roundtable was also organized. Currently, country profiles of Canadian citizens in key destinations are being developed.

The Impact of Global Asia has been focused on areas such as: Canada-India policy dialogue; Canada-India Education Initiatives; Canada-Asia two-way investment; and Canada-Korea Forum. This component was discontinued as a research theme in the 2008-09 Work Plan.

1.2.2 Building Networks in Canada and Across the Pacific

In the Strategic Plan 2005-2008, the Foundation set out four areas where it could strengthen networks in Canada and across the Pacific:

  • Consulting more closely with stakeholders of key networks on their Canada-Asia information and research needs;
  • Working with elected groups in Canada and Asia on joint projects;
  • Providing direct support for the strengthening and development of Asia Pacific research networks through a grants program; and
  • Positioning Canadian experts in selected activities of regional policy and business networks.

The component has included working with groups such as the Canada-Asia Pacific Research Network (CAPRN) and the Asia Pacific Economic Cooperation (APEC) Study Centre. The Foundation also acts as the Secretariat for the Pacific Economic Cooperation Council (PECC) and the APEC Business Advisory Council (ABAC).

This element was redefined in 2007 as well. Four changes were made.

  • A greater focus was placed on Canada-Asia policy issues.
  • A priority was placed on seeking out new innovative networks in North America and Asia that align with the Foundation's priorities. Work was begun with new groups such as the Canadian International Council (CIC) and the Shastri Indo-Canadian Institute.
  • A review was undertaken of in-house networks.
  • The secretariat support to PECC and ABAC was better aligned to the Foundation's priorities.

The CGA stipulated that a granting program was to be developed to build capacity through research, conferences or initiative funding. The program was to be accessible on a competitive basis and open to academics, researchers and organizations across Canada and internationally.

The CGA also specified that the granting program should take up an increasing proportion of the Foundation's revenue. Starting from a base of 10%, the target was to have annual increases until a minimum of 25% of the revenue was reached in FY2011/2012.

In the Strategic Plan 2005-08, the Foundation set the parameters of the Grants Program and identified six (6) major grant categories:

  • Commissioned Papers: The Foundation would invite proposals from senior scholars to write policy papers on topics related to Canada-Asia relations, to be subsequently published in the Foundation's Canada-in-Asia series.
  • The Policy Research Grants fund: The Foundation would support research on themes selected by the Foundation and having policy implications that are of particular importance in the Canada-Asia relationship.
  • Conference Funding: This program would provide incremental funding support to academic institutions, think tanks, industry associations, and NGOs for conferences in Canada, dealing explicitly with Canada's relations with Asia.
  • The Post-Graduate Research Fellowships Program: This program would provide top graduate students from across Canada with practical experience in policy research.
  • Media Grants: This program would provide travel grants for editors and senior journalists to meet with counterparts in Asia or to develop in-depth stories on major issues in Asia.
  • Special Initiative Funding: This program was designed to support unique research, conference, or public education activities that have a broad constituency and outreach potential.

In 2007, some changes were made to the approach to the grants program. A decision was made to increase the size of the grants, alter the selection process to target potential recipients with proven track records, increase the use of commissioned research, and build a closer alignment between the grants and the Foundation's research priorities.

1.2.3 Increasing Public Awareness

The Strategic Plan 2005-08 indicated that the Foundation would place greater emphasis on public outreach and exposure by leveraging access to the media, supporting selected Asian awareness activities with a broad reach, and producing analysis on issues that have immediate interest to the general public. The initial focus of the Foundation was on building broader awareness of Asia Pacific by the general public in Canada.

This approach was streamlined in recent years. The audience for the Foundation's work was more narrowly focused on "Asia practitioners"--individuals who have a professional interest in Asia/Canada-Asia relations. The primary approaches now are in support of the following activities:

  • Generating high quality media coverage including having the Foundation provide op-eds;
  • Focusing on strengthening activities outside British Columbia in other parts of Canada;
  • Launching of a new website focused on Asia practitioners;
  • Continued use of smaller roundtables and private consultations;
  • Public opinion surveys on the attitudes of Canadians toward Asia; and
  • Expansion of the Asia practitioner database.

1.3 Governance Structure

According to the Act, the affairs of the Foundation are to be managed by a Board of Directors (BoD), comprised of the following members/directors:

  1. The Chairperson and up to six other directors appointed by the Governor in Council after consultation with the Board by the Minister of the Department of Foreign Affairs and International Trade (DFAIT);
  2. Up to eighteen directors appointed by the Board after consultations with the governments of the provinces and any interested individuals, corporations and organizations; and
  3. The President of the Foundation.

The term of office for the directors is three years. According to the Act, federal public servants can serve on the Board as non-voting members only but can have access to any information and be called to any Directors' meeting.

The CGA also called for a series of Committees to be put in place to develop more transparent processes. The BoD appoints: an Investment Committee; an Audit and Evaluation Committee; a Compensation Committee; a Governance Committee; and an Executive Committee.

1.4 Management Approach

The supervision and the direction of the activities and staff of the Foundation are the responsibilities of the President or CEO of the Foundation. The President is appointed by the Board of Directors.

The Foundation has gone through two organizational changes since the CGA came into effect. The first focused on ways to meet one of the conditions of the CGA that the expenses going to overhead and administration should be decreased by 5% per year in each of the first three fiscal years after 2004/05. The Foundation implemented a series of cost-cutting measures including finding new premises and reorganizing the way it approached its work. This included the release of seven staff from the Foundation.

The most recent reorganization made a number of changes in terms of the organizational structure.

  • There is now a single President, who also fulfills the function of a Chief Executive Officer of the Foundation, instead of the previous arrangement with two co-Chief Executive Officers. As of September 2008, one of the former co-CEO positions has been transformed to a Senior Advisor to the Board. The responsibilities of the Senior Advisor are to assist the Foundation on research direction, publications, and representation, including the Canada-Korea Forum.
  • A new Executive Director position has also been created as of September 2008. This position is responsible for corporate affairs, Board relations, fundraising, administration, and finance. The Director will also present a fundraising strategy and work plan to the Board in early 2009.
  • The responsibility for research themes has been transferred to external, part-time research directors (where funding is available), who will work closely with the internal project managers.
  • Senior Fellows have been appointed who will assist the Foundation in their specific areas of expertise. The Senior Fellows support the work of the Foundation through their advice, networks, research projects and publications.
  • Support services such as information technology (IT) maintenance and the website have been outsourced.

The current organization chart is shown on the following graph.

Organizational Structure

1.5 Funding

The Foundation is funded primarily by the proceeds of the endowment of $50 million from the Government of Canada starting in 2005/06. The Endowment Fund, as per the CGA, is to be "prudently invested and the principal is not to be drawn down." While the Foundation can only use the returns, and not the principal to support its activities, the Foundation is allowed to supplement the Endowment at any time with additional amounts raised from non-government sources or through private or other government donations.

The revenue generated from the Endowment, particularly in the current market, places a constraint on the ability of the Foundation to achieve its targeted results. As a result, the Foundation has placed emphasis on raising funds from additional sources and has been increasing its revenue in this area. The new Executive Director has responsibility for developing these efforts further.


2.0 Evaluation Objectives and Methodology

2.1 Rationale for the Evaluation

The Evaluation Division (ZIE) within the Office of the Inspector General is managing this Evaluation, following the requirements of the 2005 Conditional Grant Agreement. According to Article 9.1 of this Agreement, "An evaluation of the Foundation, at the expense of the Department of Foreign Affairs, including but not limited to, a review of its corporate governance, management, transparency, accountability, activity and success, will be conducted three years from the signature of this document."

2.2 Objectives of the Evaluation

The overall objectives of the Evaluation are:

  • To assess the extent to which the activities of the Foundation continue to be consistent with its mandate, with the priorities of the Government of Canada (GoC), and those of DFAIT in the Asia-Pacific region;
  • To determine the progress made and results achieved against stated objectives and strategic outcomes since 2005;
  • To assess the effectiveness of the current governance structure and determine whether appropriate management controls are in place to support the timely, efficient and effective administration and delivery of the Foundation's programs and activities; and
  • To identify lessons learned from the experience to date.

This Evaluation assessed the relevance and efficiency and economy of APF's programs and activities in terms of meeting the priorities identified in the Conditional Grant Agreement. It also assessed the achieved results and impact of APF's activities in promoting strong relations and networks with people and institutions of the Asia Pacific Region.

2.3 Evaluation Methodology

The Evaluation took place between December 2008 and February 2009. It was managed by DFAIT's Evaluation Division. The Evaluation team consisted of two ZIE Evaluation Managers and an external Evaluation consultant.

The conduct of the Evaluation was overseen by an Evaluation Advisory Committee comprised of representatives from DFAIT and the APF staff and Board. The Committee convened twice during the course of this Evaluation. A conference call by the Committee was held in November 2008 to review the detailed Terms of Reference and draft Work Plan and make any adjustments prior to the Evaluation start. A meeting of the Committee was held in February 2009 to review and provide comments on the preliminary findings of the Evaluation.

2.3.1 Document and Data Review

Relevant documents were reviewed to examine the Foundation's performance and achieved results. Some of the documents covered included: the CGA; Strategic Plan 2005-08; Corporate Plan and Budget 2006-07; Annual Work Plans; Annual Reports to DFAIT; APF Evaluation survey; basic financial information; minutes of Board meetings; quarterly reports; listings of speeches and events at which the Foundation participated; and the Criteria and Approach to the Grants Program.

Some of the products of the APF were also reviewed including commissioned research, policy and analytical papers, conferences and symposium reports, and documents on the specific initiatives such as the Gateway Initiative and Canadians Abroad. Some of the databases of the Foundation were accessed such as the Directory of Asia Pacific Experts, Canada-Asia Bilateral Business Directory, and the statistics on Canada-Asia relations.

2.3.2 Key Stakeholder Interviews

Telephone and in-person interviews were conducted with a wide range of individuals. These included: APF staff; APF Board members; APF advisors and Senior Fellows; government, university, private sector, and association partners; and DFAIT. The purpose of these interviews was to gain a better understanding of the role of the Foundation as a provider of value-added information and analysis of the Canada-Asia relationship and to obtain the stakeholders' opinion of the effectiveness of the Foundation and the extent to which it provides valuable support to its partners and audience. In addition, a short questionnaire was sent to the Ambassadors and High Commissioners in the Asia Pacific region to obtain their feedback on the APF products, services, and activities. A 45% response rate was received.

2.3.3 APF Evaluation Survey

Instead of undertaking a separate survey as part of the Evaluation, it was agreed that the survey administered by the APF in 2008 provided a solid basis on which to obtain information on major Evaluation issues. The objectives of the APF survey were to measure and track: the importance of, satisfaction with, and level of interest in the core APF activities; the usefulness of the products, services and activities APF delivers; and the frequency of use. Information from this survey was used as background to the Evaluation discussions that were undertaken to better understand the results emerging.

2.3.4 Comparative Analysis

One of the areas for analysis was a comparison between the APF and other groups within North America having similar mandates and scope of activities. The comparison was to focus on the communication strategies and tools used by these organizations, (e.g. Web sites, multi-media techniques, e-mail newsletters, blogs and interactivity tools), to disseminate information or to build awareness of various issues. The purpose was to compare their approaches to APF's outreach activities.

A wide range of information on groups was reviewed by the Team. This included groups in the United States (US) having similar mandates such as the East West Centre, National Bureau of Asian Research, and Asia Society. In Canada, no groups have similar mandates related to Asia Pacific, but there are organizations with similar objectives of policy influence, networks, and awareness including groups such as the Centre for International Governance Innovation (CIGI), the Canadian Policy Research Network (CPRN), and the Public Policy Forum. This comparative analysis approach provided some information to support the evaluation findings but it also became clear that this approach needed to be supplemented by already existing studies and analyses of outreach practices, influence and impact of various policy and research groups.

The Foundation's masthead on the new website calls the APF "an independent think tank on Canada's relations with Asia." While APF's function now extends beyond this to include awareness building, the primary literature available is mainly on assessing think tanks Note 3. However, since the definition of think tanks has expanded greatly in recent years to reflect the wide variety of groups that now come under this label, many of the available studies were deemed relevant and useful for the purposes of the this evaluation.

2.4 Limitations of the Evaluation

The Evaluation faced two challenges. First, according to the conditions of the CGA, the Evaluation by DFAIT was to take place three years from the signature of the CGA. This meant the Evaluation was to cover the period of 2005 to present. However, the funds under the CGA were not transferred until late December 2005--meaning that activities only began in January 2006. In addition, with the shifts in focus in 2007, the actual implementation time of the current approaches and programming has been only one and a half years. Given this, it was anticipated that, at best, preliminary results would be seen on the approaches being taken. Consequently, the Evaluation focused more on whether the new programming approaches have a potential for longer-term results and represent logical shifts in the APF's programming.

The overall Evaluation had a three month timeframe, stretching over the Christmas period. Despite this, the Team was able to contact and interview all primary individuals targeted under the methodology.


3.0 Key Findings

3.1 Relevance

Finding #1: Foundation's Niche

APF continues to fill a unique role within Canada on Asia Pacific relations.

The original Act provides the Foundation with a broad-based mandate on Canada-Asia Pacific relations. The Foundation has gone through a series of evolutions in fulfilling this mandate, with the approach of the Foundation shifting to reflect the changing character of the relationships.

Initially, in the 1980s, the focus was on basic awareness building and exposure of Canadians to Asia, including areas such as cultural "dos and don'ts." This reflected the basic nature of the understanding in Canada of Asia Pacific. The next phase saw the Foundation taking a more sophisticated approach to the business agenda--reflecting the increasingly sophisticated nature of the market. A heavy emphasis was placed on business development, market analyses and who was working where. These roles were also increasingly taken over by other groups. Large corporations now have branches in Asia and extensive networks of contacts and they use consultants. The small and medium enterprises active in Asia are operating through business contacts and family members. The Foundation's niche here decreased over time.

In more recent years, the Foundation has been moving into a different role. The Strategic Plan of 2005-08 started redefining this role, and the recent shifts have continued this process. In global terms, according to the Foundation's website the focus is now on the following roles.

"The Foundation functions as a knowledge broker, bringing together people and knowledge to provide current and comprehensive research, analysis and information on Canada's transpacific relations. It promotes dialogue on economic, security, political and social issues, fostering informed decision-making in the Canadian public, private and non-governmental sectors. The Foundation also provides grants to support policy research and informed discussion on Canada's relations with Asia."

The consensus from the interviews indicated that this redefined role for APF was not only appropriate but also needed. Without the APF, there would be a distinct gap in the knowledge base to support Canada's relations with Asia.

The advantages of this role were recognized in spite of the growing number of institutions in Canada and globally, dealing directly or indirectly with Asian issues. For example, the increasing number of universities in Canada focusing on Asia Pacific research is filling a need for academic work and long-term analysis. This is seen to be different from the policy support and analysis role of the Foundation. A wide range of new groups have emerged globally that are providing information on Asia and Asian issues. The APF, however, is the only one that continues to focus on the link between these issues and Canada.

Finding #2: Relationship with the GoC

The shift to an endowment and the establishment of the Conditional Grant Agreement has allowed the Foundation more flexibility to fulfill its mandate.

Prior to the endowment and CGA, government funding was received through contribution agreements with DFAIT and the Canadian International Development Agency (CIDA). This raised three issues in terms of the independent functioning of the Foundation.

First, with contribution agreements, federal departments are directly accountable for the spending and outcomes achieved. This meant that both DFAIT and CIDA needed to have an accounting and tracking record of the financial and programming decisions made with the government funding--which at the time was the vast majority of the Foundation's revenue. This arrangement made it more difficult for the Foundation's Board to have clear accountability for program and financial decisions and placed the two Departments in a position of having to account for an independent group.

Second, the manner in which the allocations were made was problematic for the APF's selection of strategic interventions to fulfill the mandate. For example, starting in FY 2002/03, the contribution from DFAIT was provided through two routes--approximately half as core funding and half as project funding. The latter was funding for specific projects that were agreed between the Foundation and DFAIT. The DFAIT Evaluation of the Foundation undertaken prior to the endowment indicated that "while the current funding formula can make the APF relevant and responsive to [DFAIT] priorities, the funding arrangement for the APF is at variance with its broader statutory mandate." Note 4 The relationship with DFAIT was dictating much of the research agenda of the organization, making the agenda too broad based and diverse.

Third, given the yearly uncertainty of funding levels, the Foundation was not able to undertake longer-term projects or make commitments to initiatives over a period of time. This further limited the options for the Foundation in terms of the type of research, linkages, and approaches it could take to fulfilling its mandate.

The endowment was intended to make the relationship between the Foundation and Government more arms-length and this has been accomplished. There is now recognition by the GoC and DFAIT that the Foundation is, and should be, fully arms-length. While the Foundation continues to be responsive to GoC requests and priorities, it now weighs a series of factors in developing its programs. Instead of DFAIT having to report on APF outcomes, annual reports are now submitted directly by the Foundation through DFAIT to Parliament.

Finally, the shift to the CGA has also allowed a rethinking of the type of programming that can be undertaken. Multi-year initiatives, such as the Gateway Initiative and Canadians Abroad are now possible--allowing the Foundation to have a more strategic and focused approach. This has allowed new models of influence to develop.

Finding #3: Consistency with GoC Priorities

The Foundation's activities and programs continue to be consistent with the priorities of the Government of Canada and DFAIT.

The original Act makes no reference to the Foundation specifically supporting Government of Canada priorities. This was introduced in the CGA in clause 5.8 which states "The Foundation's Strategic Plans must be consistent with the Government of Canada's priorities with regard to the Asia Pacific region." The GoC priorities have evolved from the time of the CGA, as have the ways in which the Foundation contributes to these.

The Foundation continues to be consistent with the priorities of the GoC as shown by the following examples.

  • A greater emphasis is now being placed by the GoC on "Whole-of-Government" approaches. This calls for greater alignment of programming and priorities across Departments in order to increase their effectiveness. The new initiatives by the Foundation are effectively demonstrating models for this. The Gateway Initiative and Canadians Abroad are both engaging a range of government stakeholders at the federal and provincial levels to bring greater coherence and coordination of efforts.
  • The international priorities of the GoC in Asia are now focused on Afghanistan, China and India. While the Foundation has looked at ways to support the Afghanistan work, it remains unclear how this could be done, as well as what the specific areas of support needed or required by the Government might be. The Foundation, however, continues to effectively engage in issues on China and India.
  • The Foundation has continued to provide support to key Canadian commitments in Asia. Some of these were embedded in the CGA such as the ABAC, APEC Study Centre, and PECC. As will be discussed below, the Foundation is providing support for Canada to engage in these regional initiatives. Others areas, such as the support to the Canada-Korea Forum, have also continued. The Forum was created in 1996 as a mechanism for high-level, unofficial and non-partisan discussion, with the most recent meeting being held in September 2008.
  • The Foundation continues to maintain a flow of information on Canada-Asia Pacific relations which feeds into the broader information needs of the GoC and DFAIT. This consistent supply of information provides unique Canadian perspectives on many issues.

Finding #4: Strategic Focus

The recent narrowing of focus by the APF is seen by stakeholders as important for improving its effectiveness in the future and strengthening the ability to support its mandate.

In the original Act and the subsequent CGA, the mandate of the Foundation was broad based. With the endowment, the Foundation has now begun to make strategic decisions about its focus and approaches. Each of the programming areas has shifted in recent years to narrow the audience and focus of the work. For example, the audience of the Foundation is no longer a broad cross-section of Canadians for general awareness building (including the business community, academia, and general public) as indicated in the CGA. The focus on Asia practitioners for its work has narrowed the audience to more specialized subsets within these groups. The "Knowledge that Matters" component has shifted to specific initiatives such as the Gateway aimed at supporting public policy formulation around specific topics. The Grants Program is now closely linked to the Foundation's priorities and not focused on broad-based proposal calls.

In the interviews for the Evaluation, this narrowing of focus was discussed with a broad cross-section of stakeholders within and outside the organization. There was agreement across the board that a narrowing was necessary in order to improve the effectiveness of the Foundation. While not all the respondents were aware of the specifics of the new approaches by the Foundation, they were in agreement that there was a need for APF to be much more strategic and focused in its approach.

Broad-based programming is no longer seen to be effective. The rationales for this were based on a series of factors that have changed in recent years. First, information now is sourced globally. Gone are the days when someone relied only on one or two sources for information on an on-going basis. For example, a wide range of groups now provide information and immediate analyses of Asian issues, including new think tanks emerging in Asia. This means that a group needs to clearly have a niche within this broader information/knowledge picture and a clear target group for its information, as well as to recognize that it will be one of many groups providing a piece of the picture.

Second, influencing policies and policy formulation has now become a much more crowded field in Canada. A wide range of groups have emerged in recent years and there is an increasing competition among them to have and show influence. This is happening at the same time as the shift in government, which is changing the basis for policy formulation and influence--making it more complex and narrow in terms of entry points. Picking specific issues to tackle (such as the Gateway), is a more effective approach for input into policy formulation given this competition.

Third, the way in which groups exercise influence and build awareness on issues has also begun to change. Despite the competition, networking among groups has become more important. New communications strategies are being found that create and advance knowledge and public policy dialogues. This again means that an organization needs to be focused and strategic in its approaches and work with others to leverage greater results.

3.2 Effectiveness

Finding #5: Producing Knowledge that Matters

Some of the new initiatives under the research themes have been highly effective in influencing policies and issues around Canada-Asia Pacific relations.

The shift to focusing on three research themes appears to be effective in achieving some important results to date. For each of the three topics, the Foundation has been taking different approaches and focusing on different target groups.

  • For Global Asia issues, APF did public opinion polls, studies, and articles on various issues to build general awareness. A great deal of time was spent with Parliamentarians at the federal and provincial levels using this research to raise the profile of Asia and trying to inform decision making on Asia Pacific.
  • For the Gateway Initiative, the emphasis has been on using research and dialogue to broaden the issues being considered by the major stakeholders beyond simply infrastructure concerns. Areas such as security, environment, and value-added services were highlighted as being important considerations. The Foundation has been working with groups such as Transport Canada, the British Columbia government, and Western Diversification to broaden the agenda to include these issues and facilitate greater coordination.
  • The Canadians Abroad Project has generated interest using targeted research and fora to highlight the importance of the issue and the lack of a coherent policy in the area. The objective is to build awareness among policy makers on why they need to develop a policy, and build a balanced understanding of the benefits and risks of various policy options.

Of the three areas, the Gateway Initiative has been the longest term and is the one seeing initial results. The Foundation is credited with a number of important results: broadening the understanding of the key stakeholders in the Gateway of the complexity of the issues; facilitating greater coordination of efforts across agencies; and shifting the policy approaches to look beyond infrastructure. This latter area is particularly important since issues such as security and environment had little audience when the Foundation began its work, but these same issues are now broadly accepted as being vital to the future approaches toward the Gateway. The Foundation is also credited with keeping the Gateway effort on the table with the change in Government. It undertook consultations across the country to highlight the importance of the work and the need for consistent commitment.

What were the critical elements in the success of the Gateway? The stakeholders consistently mentioned four things the Foundation did:

  • Effectively identified the key stakeholders that needed to be at the table and were able to attract the stakeholders to the discussion.
  • Identified critical issues that cut across interests (horizontal) and were not part of the current dialogue.
  • Provided a platform for discussion fed by policy research so decision making could become more informed and consistent across agencies.
  • Facilitated a change in thinking and adoption of new approaches.

Key Stakeholders

Similar interest is now being generated by the Canadians Abroad initiative. A specific website was launched in January 2009 to foster the debate (Canada-China Human Capital Dialogue) and new studies are now coming on stream. This presents a potential for APF to have the anticipated influence on policy. Many of the issues around Canadians Abroad are gaining in profile based on events such as the evacuation in Lebanon and the recent changes to passport and citizenship requirements by the Government. This will provide an opportunity for the Foundation to feed the dialogues and foster a more coherent response to a complex issue.

The least success to date has been seen with the Global Asia. While the elements of the work such as the opinion polls are seen to be valuable, the task of fostering greater general awareness of issues is difficult. A shift is being made toward finding more specific niches within this area such as fostering Canada-India relations. Finding the right opportunity here has proven difficult as well. As a result, the Global Asia theme was dropped in the most recent Work Plan.

While the Foundation has a plethora of ideas, the challenge will be to find initiatives that have similar potential for influence, funding, and collaboration with outside groups. The Foundation will only be able to undertake a limited number of these, and a strategic selection will be important.

Finding #6: Strengthening Networks in Canada and Across the Pacific

A balance has been struck in the efforts to support the various Secretariats, with participants positive about APF contributions. The restructuring of the grant arrangements is providing more strategic support for the Foundation's work.

In the Strategic Plan 2005-2008, a series of short-term results were anticipated from the work with networks as shown in the box to the right. The networks that are the focus of the Annual Work Plans and Progress Reports are areas of commitment under the CGA--namely the support to Canada's commitments in various groups such as ABAC and the grants program.

APF Strategic Plan 2005-2008

Short/Medium Term Results for Strengthening Networks in Canada and Across the Pacific Component

  • Increased cooperation between the Foundation and Asia Pacific business/research networks in Canada
  • Strengthened Canadian networks on Asia Pacific research and business issues.

The two primary academic networks the Foundation is supporting are: the APEC Study Centre which is a commitment of the Government of Canada; and the CAPRN which is a virtual network. For the business networks, the Foundation provides Secretariat services to the ABAC and the PECC.

Support for the APEC Study Centre, ABAC, and PECC is embedded as a condition in the CGA (section 2.4.2). There was some controversy in recent years about how the Foundation should meet its obligations. All three of these groups are commitments that the Government of Canada made in international settings. Each of the groups is regional with some countries providing extensive support to further their participation. For example, while Canada's budget contribution to ABAC is basically equal to that of the US, the US provides funding for six full time staff to work on ABAC issues. In Canada, three representatives appointed by the Government volunteer their services and are provided with the necessary research and support by the Foundation. In general, the Foundation is Canada's primary mechanism that provides the support for ensuring Canadian visibility in these fora.

In 2007, DFAIT questioned the extent to which APF was providing support to these networks. With the staff changes at the Foundation, the level of support had decreased in some cases. This was also raised at the June 20th 2007 Board meeting where DFAIT emphasized the critical importance of groups such as APEC and ABAC to Canada's interests in the Asia Pacific region.

Since that time, the APF has refocused its efforts to bring its support more in line with Foundation priorities while ensuring it meets its obligations. The results appear to be a good balance. The feedback from the ABAC representative on the Foundation's support was extremely positive--indicating that Canada's influence in the network was clearly enhanced due to the Foundation's support. Similar comments were seen with the APEC Study Centre.

CAPRN is the Foundation's virtual network on contemporary Asian affairs and Canada-Asia relations. It provides links to other Canadian organizations active in Asia Pacific and operates an online database of experts. The CAPRN expert database currently has 364 listings. To qualify, candidates have to meet three criteria: currently conduct research or consult on Asian affairs and Canada-Asia relations, or serve in a relevant policy-level position; have a PhD or appropriate professional experience; and have published books or journal articles on policy-relevant topics relating to Asian affairs within the last ten years. Limited feedback was able to be obtained from the outside interviews on the CAPRN since few people were familiar with it.

The Foundation deals with a range of other networks including organizations such as the Canada China Business Council (CCBC) or Shastri Indo-Canadian Institute. It is difficult to judge the extent to which networks are being fostered or strengthened as targeted in the Foundation's results statement. Positive feedback was provided on specific events done by different groups with the Foundation, but few comments were received beyond this. In fact, a few people interviewed indicated that the Foundation's profile was higher in Asia than Canada.

As mentioned in Section 1.2.2, the Grants Program was a condition of the CGA, with increasing levels of funding intended to go towards this. The original focus was on funding for a wide variety of research products. These produced limited results since in many cases the topics were fairly esoteric and had limited policy relevance. Few of the papers received attention after they were published.

The shift in the Grants Program toward building a closer link to the research themes, or other topics of priority to the Foundation appears to be more effective in terms of complementing the Foundation's approaches. Some of the background papers for the Gateway and Canadians Abroad programming have been or are being funded under the grants element and have received widespread coverage.

Finding #7: Increasing Public Awareness

Specific activities such as conferences, roundtables, and speeches are seen to be highly effective. Overall, however, there is limited understanding and visibility of APF's programs. While a narrowing of APF's focus to Asia practitioners is appropriate, there is a need to better define what that means and how best to reach the market.

The awareness programs of the Foundation have centred around a series of different types of activities including redesigning the website, opinion polls on attitudes toward Asia, media relations, speeches and roundtables. The audience is now Asia practitioners, not the general public as previously targeted. Within this are a diversity of interests and priorities. The approach used by the Foundation is to develop various products that can provide value to the audience.

APF Strategic Plan 2005-2008

Longer Term Results for Increasing Public Awareness component

  • Greater awareness of the Foundation and its work across Canada.
  • Greater knowledge of Asia and Canada-Asia relations among Canadians.

The feedback on the specific activities was extremely positive. Individuals who had contacts with the Foundation through events such as joint organization of conferences and roundtables felt the APF did an excellent job and had strong expertise in Canada-Asia relations. Those who were aware of the new website thought it was a vast improvement over the previous effort and providing information in a more accessible way. Previous events such as the Asia Summits were seen to be a good forum for raising the knowledge levels on Asia Pacific.

A similar pattern was reflected in the survey undertaken in 2008 by the Foundation of its primary users. The Foundation plans to undertake similar surveys each year to obtain feedback on their products, services, and activities Note 5. The following Table shows responses to questions about the usefulness and rating of the APF products, services and activities. In most cases, 50% or more rated them 4 or 5 on a 5 point scale.

Table 1: APF Survey Results
On a 5-point scale, where 1 = "Not very well", and 5 = "Very well", how well do you rate APF Canada for the following?123452008 Score
How well do the products, services and activities of APF Canada cover the sectors or subjects of interest to you?n=4053%7%24%44%20%3.71
How well do you think APF Canada adapts to changes in Asia Pacific and in Canada's relationship with the region?n=4054%5%33%41%16%3.61
On a 5-point scale, where 1 = "Not at all useful" and 5 = "Very useful", how useful do you find:123452008 Score
APF products in your business planning, operations or research activities?n=4025%12%34%32%17%3.43
APF products, services and activities in comparison to those provided by other institutions in Canada?n=4022%10%27%37%23%3.68

Source: Findings - 2008 Internal Evaluation - Asia Pacific Foundation of Canada

However, what the Evaluation interviews also revealed was that while individuals had contact with the Foundation in one context, they had limited or no knowledge of the other work the Foundation was doing. While the goal clearly cannot be to have everybody read everything or participate in all activities, to maximize the value of the Foundation to its audience they need to have a general awareness of how they could benefit. For example, someone may have been on a panel with an APF representative but were not aware there was a new website with information that might be useful for them.

The visibility of the Foundation was very activity based, and was not maximizing the potential impact of the APF as a broader knowledge-based organization or resource. The interactions did not trigger greater general awareness of the Foundation and its programming and priorities, but tended to represent more isolated events or contacts.

This same pattern was seen in the survey conducted by the Foundation. The following table shows one question related to the benefits from the contact with APF. While 52% saw benefits, a surprising 18% did not know, or were not sure whether the APF had an impact on their networking, information exchange or cooperation. One would expect yes or no answers, rather than "do not know."

Table 2: APF Survey Results
Has APF Canada helped increase your networking, information exchange or cooperation with others in your field involved with the Asia Pacific?
Yes52%
No30%
Don't know/Not sure18%
No response1%

Source: Findings - 2008 Internal Evaluation - Asia Pacific Foundation of Canada

So why does the Foundation seem to have lower visibility overall? This seems to relate to a series of factors.

Some of the Foundation staff feel that the Foundation has more limited visibility due to its funding levels. For example, the inability to have Asia Summits (due to funding restrictions) decreases the overall profile of the Foundation. This may be true in part, but there are also other issues besides budgets. They deal with how a group can lever its funding to raise visibility.

A study is done each year of groups and organizations designated as "think tanks" across the globe--using a very broad definition. Note 6 This year, the study took the 5,465 think tanks globally and undertook a ranking of the groups according to specific criteria by a panel of experts. An interesting result from the study was the fact that some of the top Canadian groups were very small operations--with budgets less than that of the APF. Despite their budget size, however, they were able to leverage their visibility and influence, and ranked higher than groups such as the Conference Board. A logical conclusion is that money or budgets cannot be the only reason for low visibility.

Part of the reason for lower visibility may be the shifting focus of the Foundation over time and the time required to build a new image and place in the marketplace. It has only been a short period of time that the Foundation has undertaken its current programming approaches. This was evident to a certain extent from the interviews since some people were no longer clear on the role that the Foundation was pursuing.

Another partial reason for why the Foundation is not generating greater awareness of its programs is seen in the manner in which it undertakes some of the work. The following graphic is a stylistic depiction of the two streams--building networks and increasing public awareness. What are the possible issues?

Program Overview

  • Few linkages between the network building component and awareness building - While the common link between the network and the awareness building activities is individuals within the APF, there are a few strategic links made between the two streams of work. The activities with the grants and the groups such as ABAC are coordinated with the Foundation's research themes and other priorities. However, awareness activities are not seen as being informal network builders for practitioners, but are focused mainly on individuals at the APF. How networks can become a critical part of the awareness building appears not to have been systematically developed. This includes the nature of the collaboration with various groups in Canada and Asia, which is mainly event-focused.
  • Limited synergy among elements to build greater visibility of the Foundation and awareness of the Foundation's work -The interventions to build awareness have been undertaken often in separate streams on more of a hub and spoke basis. These are often handled by different individuals within the Foundation. The marketing approach is not comprehensive. Some individuals asked, for example, why the website could not have five minute podcasts of visiting dignitaries or a summary of key trends emerging from closed roundtables. The website is also seen as being too one-directional, not interactive. Limited synergy is being built among the elements.
  • Lack of clarity on how speeches and roundtables fit - A large effort is put into sponsoring roundtables, speeches and visiting delegations. These are particularly important activities for building an awareness of the Foundation and Asia Pacific issues. However, besides extensive listings of the meetings, these events are not presented in the Annual Work Plan or Progress reports as part of a strategic focus. Criteria are used, for example, on what speaking engagements to take, but greater visibility of staff at events does not seem to result into increased awareness of the Foundation and its work.

The narrowing of the focus to Asia practitioners is appropriate given the potential impact that can be seen from working with this group. The difficulty is that, by definition, this group already has an "awareness" of Canada-Asia relations. If the Foundation is to fulfill the role of a knowledge-based organization, there is a need to better define what that audience needs and how best to reach the market.

This will include having a better understanding of the groups and individuals being targeted, developing insights into issues and policies that are or could be of interest, and determining how the Foundation can facilitate the interactions through a variety of routes. In some cases, this could be specific products, such as the news service. In others, it will be a more dynamic process that can facilitate interactions around issues and build networks of ideas. Better tapping this audience may not require new products but new processes and methods of interaction.

There is also a need to reassess the methods of collaboration with other organizations. The current climate demands greater coordination and the building of consortia around issues to have a greater influence. The separation of "building networks" and "building awareness" functions needs to be reviewed in order to leverage the new opportunities and approaches possible with other groups.

3.3 Governance

Finding #8: Board of Directors

The expansion of the Board, and its new composition, has improved the overall governance structure. However, some issues remain, in terms of how to better engage the Board members in decision making and APF activities.

At the time of signing the CGA, the number of Board members was at a low point (less than 8). The period was one of uncertainty. The endowment had been approved by the previous government only two days before it fell, and it was unclear whether it would go forward with the new government. As the terms of Board members expired, they were not replaced immediately until the future of the Foundation became clearer. In addition, there was some concern about possible liability of the Board for salaries and other expenses of the Foundation if the funds did not come through.

Since that time, there has been a gradual increase in the Board complement with both Order in Council (OIC) and Board appointed members joining. The current size is 14 plus the CEO. It represents a broader geographic mix (i.e., members from outside British Columbia) and political interests, as well as gender composition. With 3 seats coming up for renewal shortly, there are opportunities to continue to strategically select candidates that can maximize the benefits to the Foundation in terms of expertise or networks.

The Board has also implemented the Committee structure outlined in the CGA. There are currently five Committees in place--Executive, Investment, Audit, Governance, and Personnel and Compensation.

The current challenges seen with the Board are a function of this transition period. These are seen in three particular areas. First, with only two meetings a year due to budget constraints, Board members are not all actively participating or even clear about their roles. With limited opportunities to discuss the strategic direction of the Foundation and its work, Board members tend to be on different wavelengths in terms of their views of the current and future focus for the Foundation's work. In addition, some do not feel fully engaged in the decision-making process. The tendency is to have Senior Management and the Chair develop ideas and strategies, and bringing them to the Board for discussion. While the discussions at the meetings may be extensive, few opportunities emerge for broader engagement strategic issues outside the Board meetings.

Second, the functioning of the Committees continues to be in a state of transition. Shortly after the CGA was put in place, decisions were made by a core group of people. These included decisions, such as the termination of a number of staff members as a cost-cutting measure. This decision was taken by the Executive Committee and the full Board was informed afterwards. Since that time, the Board functioning has become more inclusive, however, the decisions of the individual Committees can be overridden by the Chair, without a full Board consultation.

Third, the Board members are appointed and are not compensated for their participation--a condition of the original Act. Their contributions then, are in fact only their participation in the Board meetings and on Committees as needed. Traditionally, there has not been an expectation of Board members that they would provide support in other areas, such as providing contacts or assisting in fund raising for projects. The Board represents a wide range of influential people who might be willing to provide more extensive support in networking or financial leverage. This is currently a lost opportunity.

Finding #9: Management Structures

The new management structure, including greater use of outside resources, makes sense in terms of ensuring flexibility for the organization. However, it is important to maintain a critical mass of in-house resources to effectively fulfill the vision of a knowledge-based organization.

After the major cut in staff in early 2007, a new model for delivering programming was developed. A shift began from operating primarily with in-house resources to using a mixed model, involving in-house complemented by out-sourced. Staff were shifted from being country-specific analysts to being Project Managers. Senior Fellows were appointed to work on specific areas on a part-time basis. The Grants Program focused more on commissioned research which complemented the themes and priorities being pursued.

The Foundation is now tapping people to do specific assignments, playing a greater brokerage role. APF decided on the topics and reaches experts to undertake the work. This is a more effective model in terms of flexibility and mobilizing high quality assistance. Doing everything in-house requires much more extensive staffing in order to cover a spectrum of issues.

However, this model still does not resolve all challenges. First, the profile of the organization is still based on its in-house resources. The ability to broker works for specific issues, does not offset the need for a critical mass of expertise within the organization in order to fulfill the vision of a knowledge-based organization. A wide range of interviewees from outside the Foundation indicated that the in-house capabilities were critical to the Foundation's credibility.

Second, getting the right mix of in-house expertise is also difficult with a more limited staff. As the ideas and approaches shift, so do the skill sets required. For example, with a shift towards more web-based services, expertise on how this can be maximized is required.

Third, the Senior Fellows model can work well if the Foundation gets the right people. This requires not only finding the right expertise, but also being able to get a greater impact from these contacts. Managing the process is important, but time consuming for staff.

3.4 Economy and Efficiency

Finding #10: Endowment

The endowment has not provided the expected predictability of funding due to the current financial crisis. However, the Board and management are dealing well with the issues and streamlining programs and activities to match the short term financial reality.

One of the original rationales for the endowment was that it would provide a predictable source of funding for the Foundation. Instead of the on-going negotiations and approvals by two Government agencies, the endowment would allow financial flows based on the investments.

Until recently, this was the case. However, like most organizations, the Foundation has been hard hit by the current financial crisis. By October 2008, the market value of the endowment had fallen below $50 million--although the 24-month rolling value was still at $55 million. Note 7 This was further exacerbated by the fact that some of the funding committed by groups outside the endowment was also put on hold.

Management and the Board moved quickly to deal with the issues. Steps were taken to reduce spending for the year by approximately 11% of the approved budget. The Investment Committee members reviewed the current investment portfolio and performance of the investment advisors and made some recommendations for change. The Board and staff are reviewing the situation on an on-going basis including revising forecasts.

Options for more substantial cuts in the budget are constrained by a number of factors, however. First, approximately 40% of the revised budget is for employee compensation. The staffing is at a low level currently with limited flexibility to decrease this further. Second, according to the CGA, the Foundation needs to maintain a minimum of 10% of last year's (not this year's) endowment income for the grants program. Note 8 Third, the Foundation has limited room to cut the resources committed to support groups such as ABAC and PECC. This again is a commitment within the CGA. These latter two items are approximately 15% of the revised budget forecast.

With the current financial crisis, there is clearly more pressure to generate funding outside of the endowment income. The following table shows the type of revenue generated in FY 2007/08 outside of the endowment. The projected estimate for 2008/09 was originally approximately $450,000. This has now been decreased in the revised budget by 50%.

Table 3: Summary of Funds Raised 2007/08
Funding SourceAmount
Total$571,017
Government of Ontario$10,000
Atlantic Canada Opportunities Agency$10,000
Government of British Columbia$460,000
Western Economic Diversification$68,729
Private sector$13,789
Miscellaneous$8,499

Finding #11: Outside Funding Models

The Foundation has developed an interesting funding model for its research themes. The next few years should determine its broader feasibility.

For the Gateway Initiative and the Canadians Abroad, a new type of funding model was developed. Building on initial research and dialogues with key stakeholders, the Foundation developed proposals for funding of longer term initiatives. The Foundation's funds were used as a base to get contributions from outside sources for the specific research themes. This leveraging was successful because APF found ideas that not only engaged stakeholders but represented issues which cut across stakeholders and could bring forward new ideas.

The revenue generated from the different sources allows specialists to be hired to provide support in areas such as research or the coordination/facilitation function. For example, the research commissioned in both initiatives has been critical in identifying ideas and concepts for further development with partners. Senior Fellows have now been assigned to both, the Gateway and the Canadians Abroad initiatives to move the initiatives forward. Websites have been developed to highlight both initiatives.

This type of strategic approach to fundraising allows more effective development of the research themes by providing a volume of support to tackle issues in a more comprehensive manner. Given the current financial problems, it would be tempting to chase funding opportunities. This has the danger of diffusing the Foundation's efforts and not allowing a critical mass of activities to develop to have an influence. Instead of chasing funding, this new approach allows a more strategic targeting of funding opportunities around specific issues.

Some of those interviewed, however, feel the model needs to be further tested to see its broader applicability. Others feel that with the right ideas, financial resources can be mobilized even in the current market. Over the next few years, it should become clearer the extent to which this provides a sound base to raise both the financial resources and the profile of the Foundation.

Finding #12: Measuring Performance

The Foundation has not placed enough emphasis on measuring performance and feeding this back into decision making.

The Strategic Plan 2005-08 set out a series of results that were targeted for the three priorities of the Foundation. It also included specific performance indicators that were to be tracked. This was in keeping with the requirement under the CGA that the Foundation develop an internal audit and evaluation capacity.

In a Board meeting in March 2006, the Board expressed concerns about the extensive work required in reporting under the CGA. Subsequently, in 2007, the Board decided that the set of quantitative indicators in the Strategic Plan would not be tracked but instead, a new system of monitoring was to be implemented based on stakeholder feedback via yearly surveys.

In 2008, the Foundation undertook its first evaluation survey--with some of the results reported above. The intention was to conduct the survey on an annual basis to assess the value of the Foundation's overall efforts. This was to become the primary feedback mechanism.

In addition, a few quantitative indicators are continuing to be tracked for inclusion in the Annual Report to DFAIT. These focus on the volume of activity undertaken during the year by the Foundation in a number of areas. The following is the results table from the 2007-08 Annual Report tabled with DFAIT. The Foundation is also now tracking references in articles, broadcasts, web-based references, and references in books and journals internationally.

Table 4: APF Results Report 2007-08
 2005-20062006-20072007-08
Funding Leveraged
1 The Foundation began to outsource its web maintenance in April 2007 and accordingly lost its in-house capacity to track web usage. In September 2008, the Foundation moved to a new system of tracking web usage as part of a major redesign of the website. With the completion of the new site, information is now starting to become available as of late 2008. The new tracking system, unfortunately, cannot be compared with previous years.
2 A comparable question was not included in the 2008 national opinion poll of Canadian views on Asia.
3 The Foundation did not organize an Asia Pacific Summit in 2007-08.
Additional Funds Leveraged$201,664$579,651$571,017
Funding Partners for APF Programs7227
Increasing Public Awareness
Media Articles referencing APF447677444
Opinion-Editorials in Canadian media62510
Public Speeches Delivered344751
Number of Visitors to APF Website586885472034n/a1
Documents Downloaded from APF Website158177231586n/a
Overall Awareness Level of Asia's Importance to Canada (based on National Opinion Poll results)51%70%n/a2
Strengthening Canada-Asia Networks
Key stakeholders participating in APF2573511538
Delegates attending the Asia Pacific Summitn/a515n/a3

The survey and the tracking of these indicators certainly provide insights into the overall activities of the Foundation, thus meeting the requirements of the CGA. What they do not provide, however, is enough information that could be used for decision making on priorities and programming. For example, the survey provides extensive information on use and usefulness of products and services. Do the products meet the needs of the Asia practitioners? Do they use the information? However, what the survey does not provide is other information important for decision making, such as the potential niches that could be filled. The Foundation is not in the product business but in the knowledge business. The Foundation's products are a tool--the question is how to facilitate knowledge using tools? The survey does not cover these issues.

Other indicators being tracked are also serving some functions, but not others. Every group that is trying to have policy influence tracks media coverage and web site hits. Media coverage, in particular, is seen to be an important indicator of visibility and should be tracked. However, it only provides one dimension of the picture: the amount of media coverage shows that an idea is in the public realm but does not tell whether the right people are seeing it, so that it can have an influence on debates. A number of studies have shown now that, while counting media coverage can provide insights into visibility, this is not necessarily a reliable indicator of influence. Note 9 Other feedback mechanisms need to be developed and used.

Measuring performance from web based activities is becoming more complex as well. Web hits tell one part of the picture in terms of popularity of particular documents or web pages. They do not tell whether this information is used or has an influence on how issues are discussed or formulated. Some new tools are now starting to emerge that try to measure other forms of visibility beyond simple web hits. For example, some researchers from the University of Regina are using a software package called Issues Crawler to look at the extent to which various groups are linked across the web. Note 10 The assumption is that groups with established reputations for solid research and information will develop more linkages with other groups and therefore greater visibility and influence.

Currently, performance measurement is seen by the Foundation primarily as a requirement for reporting to DFAIT and the GoC. In the current competitive environment, more emphasis needs to be placed on assessing performance and making strategic choices based on that performance. A more sophisticated approach needs to be developed by the APF that feeds back into the marketing and programming decisions. This includes revisiting the results being targeted by various activities, the annual survey, and the approaches to capturing results. In addition, the Results Framework from the Strategic Plan no longer fits in many ways--there is a lack of clarity of what results are being targeted among staff and the Board. This needs to be clarified so there is a basis to measure success and adjust programming to maximize performance.


4.0 Key Conclusions and Recommendations

4.1 Conclusions

This Evaluation has provided an opportunity to review the work of the Foundation and the changes it has begun to implement. APF has taken major steps in its organizational and programming transformation over the last several years. While the transition is still a "work in progress," the initial indications are positive for the future. With some adjustments, the new approach has potential for increasing the Foundation's influence and funding opportunities.

The Asia Pacific region remains important for Canada and the Foundation continues to play a unique role in the relationship. While the role has become more targeted, it is no less important overall. This role has also been fostered by a maturing of the relationship between the Foundation and the GoC. The endowment has allowed the Foundation to clearly operate as an arms-length independent organization--a positive move from both the Foundation's and the Government's perspective.

The new programming approaches have seen some clear successes and some areas which need adjustment.

  • The new initiatives in the first component "knowledge that matters" aim at supporting policy dialogue and changes. The initial work here has produced some positive and concrete results in terms of policy influence and coordination.
  • For the work with building networks, the Foundation is meeting its obligations to support key initiatives and commitments of the GoC in the region such as the ABAC, PECC, and APEC Study Centre. However, the broader work on building networks is less clear. Even though some relationships between various organizations and the Foundation are being built, the spinoffs from these relationships are difficult to assess.
  • The efforts on building awareness have shifted focus to the more specialized audience of Asia practitioners. The potential for greater impact here--versus the general public--is high. Some of the products and services are appreciated by users. Despite this, the Foundation continues to have limited visibility. The new market is proving difficult to tap and the current methods for building awareness need to be reassessed.

The governance structures have improved in recent years with an expansion of the Board and better articulation of Committee structures. Some continued adjustments are needed to allow fuller participation of the Board members in supporting the strategic direction of the Foundation, as well as its networking and fundraising activities.

Important changes have also been made in management structures. After an initial downsizing several years ago, the Foundation has further shifted its approach to a mixed model of in-house and outsourced resources. This has provided more flexibility to tackle issues and opportunities as they arise. Maintaining a strong in-house capacity and profile is still critical, since this is what defines the Foundation in the market.

The funding picture is one that is facing more difficulties. The predictability of the flow of funds from the endowment revenue has been disrupted by the current financial crisis. This is also true of other funding commitments from outside groups which have been put on hold in recent months. While management and Board have moved quickly to deal with these issues, they will continue to be challenging over the next year. A topic-based funding model which has been successfully implemented for the research themes has potential to counter this trend and provide new opportunities for impact and funding.

Finally, with all of the changes in programming and approaches over the last few years, there is a need to re-examine the results currently being targeted by the Foundation and how success will be judged over the coming years. The current approach to measuring performance is compliance based--a requirement under the CGA. It is important, however, that the Foundation develops better measures of success, monitors progress, and integrates these into decision making.

4.2 Lessons

The Foundation represents an organization that has gone through a number of transformations since its inception in 1984. The approaches have shifted to reflect changing conditions in Canada-Asia Pacific relations and the priorities of various stakeholders. The period when the endowment was approved was a difficult one for the Foundation, representing great uncertainty. The shifts that have taken place since that time have brought the organization in new directions, with new opportunities. Two lessons emerge from this experience.

  • Transitions are difficult periods for organizations. The Foundation has been challenged to find new methods of delivering programming, new audiences, and new approaches to policy influence. These transitions have also required flexibility from the funding groups to allow the shifts to take place. A strict adherence to the exact wording of the CGA would have limited the effectiveness of the Foundation by forcing it to be too diffused. The compromises that have been reached have allowed a more strategic approach to moving forward.
  • Transitions also require reassessments as they unfold. Once ideas are tested, the Foundation needs to learn from both its successful and less successful initiatives. One area that is becoming more apparent is the need to rethink the original Strategic Plan from 2005 and develop ways to better link and build a synergy among the three components--building knowledge, networks, and awareness.

4.3 Recommendations

Recommendation #1:

The Foundation should undertake a review of its approach to the Asia practitioner "market" and develop a new strategy for effectively building networks and awareness. To implement a strategy may involve changing elements, such as the Website, to make it more interactive and developing new approaches to networking.

Recommendation #2:

Enhanced collaboration with other groups in Canada and Asia Pacific should increasingly be considered as a method to increase influence and outreach. Greater collaboration will allow leverage of not just funding but ideas and networks that should contribute to raising the profile of the Foundation.

Recommendation #3:

The Foundation should continue improving the composition of the Board with this next round of appointments. This includes increasing the number of members who are specialists in areas where the Foundations works. The Board also needs to develop new methods for engaging the members in decision making and supporting the work of the Foundation in terms of networking or financial leverage.

Recommendation #4:

A new approach to assessing performance needs to be developed that is more closely linked to decision-making processes within the Foundation. Monitoring performance is an important aspect for testing whether the approaches being taken are reaching the right audiences and having the type of influence anticipated. Performance measurement can be done in a way that continues to meet the requirements of the CGA, while providing more useful information for decisions.


Annex A: Management Response and Action Plan

Table 5: Management Response and Action Plan
RecommendationsAPFC Management Response and Action PlanResponsibility CentreTime Frame
Recommendation 1:
The Foundation should undertake a review of its approach to the Asia practitioner "market" and develop a new strategy for effectively building networks and awareness. To implement a strategy may involve changing elements, such as the Website, to make it more interactive and developing new approaches to networking.Networking and awareness building are part of the mission of the Foundation. The new website focuses on this priority. It includes new products such as Asian Business Cases and research programs such as Canadians Abroad. These efforts are all supported by sponsorship and consortium partnerships. Further expansion of this mandate will be dependent on successful financial support. To increase the interactivity of the AsiaPacific.ca site, we have also expanded it to include feedback features on our commentary work. A new approach to networking is the launch of Asia Pacific Video, which features experts speaking on topical Canada Asia issuesAsia Pacific Foundation of CanadaThe Site was Launched this year and will be expanding this spring to include the Canadians Abroad, Asian Business case and other new approaches such as Asia Pacific Video are planned to launch in late spring.
Recommendation 2:
Enhanced collaboration with other groups in Canada and Asia Pacific should be increasingly considered as a method to enhance influence and outreach. Greater collaboration will allow leverage of not just funding but ideas and networks that should contribute to raising the profile of the Foundation.The business model developed two years ago was in fact premised on the enhancement of our collaborative efforts. The current themes are supported by a consortium group which will be expanded this year. It is agreed that wider inclusion will serve to raise the profile of the Foundation and to raise the awareness of Asia to the general Canadian public. Recent examples of our collaborations include the Canadians Abroad research project which partners with the CIC, BC Government, Gordon Foundation, DFAIT and Stats Can. Strategic partners for other projects include Health Canada, CCBC and the Canadian International Council.Asia Pacific Foundation of CanadaOngoing
Recommendation 3:
The Foundation should continue improving the composition of the Board with this next round of appointments. This includes increasing the number of members who are specialists in areas where the Foundations works. The Board also needs to develop new methods for engaging the members in decision making and supporting the work of the Foundation in terms of networking or financial leverage.The profundity and commitment of the Board is a tremendous asset of the Foundation. It is paramount that the Foundation ensures that new appointees provide national representation and meet the high level of professionalism the current Board provides. The composition of the Board depends both on OIC and Board appointments. In the last board meeting (Feb 2009) the directors committed to seeking new members who were from areas not currently represented. The Board committees were also renewed with an effort to ensure that members were more involved in decision-making. A new Executive Director was hired to work with the Board to ensure that more focus is given to fundraising. Given the mandate of increasing financial support and leveraging, the Board's involvement will be key to the Foundation's success.Executive teamOngoing
Recommendation 4:
A new approach to assessing performance needs to be developed that is more closely linked to decision making processes within the Foundation. Monitoring performance is an important aspect for testing whether the approaches being taken are reaching the right audiences and having the type of influence anticipated. Performance measurement can be done in a way that continues to meet the requirements of the CGA, while providing more useful information for decisions.This is a very key assessment of the Foundation and one that exposes the need for a greater proportion of resources commitment. It is important that performance monitoring is used to ensure that we are meeting our mandates in a timely manner. The first survey in regards to the Foundation's new product line was completed last year and will serve as a baseline for this year's enhanced and improved effort. The new survey results will be more closely linked to the decision-making process within the foundation.Executive EditorSpring 09


Footnotes:

Note 1 Annex B: Conditional Grant Agreement between Her Majesty the Queen in Right of Canada and the Asia Pacific Foundation of Canada. March 31, 2005.

Note 2 This was originally called Transnational migration/Canadians.

Note 3 Note that the literature on knowledge based organizations focuses almost exclusively on businesses and, therefore, was not relevant.

Note 4 Foreign Affairs Canada, Evaluation Division. May 2005. "Summative Evaluation of the Asia Pacific Foundation of Canada: Final Report," page 25.

Note 5 These same questions were mirrored in the questionnaire by the Evaluation Team to Ambassadors with similar results.

Note 6 James G. McGann. 2008. The Global Go-To Think Tanks 2008: The Leading Public Policy Research Organizations in the World.

Note 7 The compliance measure to meet the conditions of the CGA is the 24 month rolling average.

Note 8 The endowment revenue for FY 2007/08 was $3.246 million.

Note 9 See for example, Donald Abelson, 2002. Do Think Tanks Matter? McGill-Queen's University Press.

Note 10 Kathleen McNutt Gregory Marchildon. Forthcoming. "Think Tanks and the Web: Measuring Visibility and Influence."

Office of the Inspector General


* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Footer

Date Modified:
2012-11-19