Evaluation of Diplomacy, Trade and International Assistance Coherence in the Asia-Pacific Branch, 2015-16 to 2020-21

Evaluation Report
Prepared by the Evaluation Division (PRA)
Global Affairs Canada (GAC)
December 17, 2021

Initialisms and acronyms

ADM

Assistant Deputy Minister

BFM

International Business Development, Investment and Innovation Branch

CBS

Canada-based staff

CFLI

Canada Fund for Local Initiatives

CIDA

Canadian International Development Agency

CSO

Civil society organization

DFAIT

Department of Foreign Affairs and International Trade

DG

Director general

EDM

Expert Deployment Mechanism

FIAP

Feminist International Assistance Policy

FPDS

Foreign Policy and Diplomatic Services

FTE

Full-time equivalent

HOM

Heads of mission

HQ

Headquarters

IFM

International Security

LES

Locally engaged staff

MFM

Global Issues and Development Branch

MSR

Management summary report

NGM

Americas Branch

O&M

Operations and maintenance

ODAA

Official Development Assistance Act

OGD

Other government department

OGM

Asia-Pacific Branch

PFM

Strategic Policy Branch

PRD

Evaluation and Results Bureau

PSOPs

Peace and Stabilization Operations Program

SDGs

Sustainable Development Goals

TFM

Trade Policy and Negotiations Branch

WGM

Sub-Saharan Africa Branch

Executive summary

This thematic evaluation of the Asia-Pacific Branch (OGM) examined coherence across business lines, that is, diplomacy, trade and international assistance, from 2015-16 to 2020-21. It is the third in a series of 4 geographical coherence evaluations. Its main objectives were to provide an assessment of the extent to which OGM operated in a coordinated and coherent manner and to examine the factors that either fostered or hindered the ability of streams to collaborate when mandates and outcomes were shared.

The evaluation found that streams frequently collaborated on joint files, with several examples of cross-stream collaboration identified. Diplomacy was a natural partner for most streams when collaboration was expected or desired. Staff in the Asia-Pacific Branch valued coherence but also underestimated the willingness of others to collaborate on joint initiatives. Staff also reported that joint initiatives led to the achievement of outcomes that could not have been achieved without the contribution of other streams, thus confirming the added value of collaboration. Three main types of impacts resulted from cross-stream initiatives: 1) the emergence of a culture of coherence and collaboration across the department; 2) increased efficiencies; and 3) the achievement of departmental and stream-specific impacts.

“Leadership,” “capacity and expertise” and “organizational structure” were the most important factors that determined the level of collaboration within the Asia-Pacific Branch. Some of main lesson learned for these 3 factors included:

• Leadership: Coherence thinking among management was observed primarily through the strategic annual planning cycle. Unless specifically mandated, potential joint initiatives strategically identified by senior management tended to not result in tangible joint initiatives. In large part, the success of joint initiatives was found to be determined by the direction and guidance senior management provided to staff.
• Capacity and expertise: Staff confidence in their skillsets for collaboration were high, but there may have been gaps in practical knowledge and competencies that hindered efforts for effective collaboration. Limited knowledge of other streams, lack of incentives, and capacity issues across the department were found to be important hurdles to collaboration.
• Organizational structure: OGM’s organizational structure was not a significant barrier to coherence. The reorganization that occurred within some bureaus was found to have had a positive impact on coherence. The 3 business lines had sufficient flexibility to engage in cross-stream collaboration, although they were limited by features inherent to their relative streams.

Program background

Background: Coherence

Key definition of coherence

Introduction to evaluating coherence

In the context of an evolving global landscape, there is a growing need for integrated policy advice and coordinated programming to deliver on departmental priorities effectively and efficiently. Coherence has been identified as an enabler to facilitate Global Affairs Canada’s ability to deliver in this evolving environment. As a result, there is increasing interest in assessing the extent to which the department has been able to break silos and create the necessary conditions for increased coherence across its main business lines, namely diplomacy, trade and international assistance.

To support senior management in understanding the state of coherence in the department and specifically across the 3 business lines, the evaluation team has committed to a suite of evaluations related to coherence within the geographic branches. This evaluation focuses on the Asia-Pacific Branch (OGM). The Sub-Saharan Africa (WGM) and the Americas (NGM) branches were evaluated in the last 2 years, and the Europe, Arctic, Middle East and Maghreb Branch (EGM) will be evaluated next year. The 4 geographic bureaus will be evaluated successively, culminating in a meta evaluation in the final year.

Definition of coherence

The concept of coherence has gained prominence due to a recognition of the growing complexity and interconnectedness of global challenges. As the concept has been used to describe various policy and programming approaches, there is no one standard definition of what coherence entails. To provide consistency for the eventual meta evaluation, the following definition was established:

“Coherence is an enabler for identifying and leveraging synergies across diplomacy, trade and international assistance, ultimately contributing to increased efficiency and better results in Canada’s international engagement.”

Background

The Asia-Pacific Region

Figure: Map of the Asia-Pacific region.

Text version

The Asia-Pacific is home to

60% of the world’s population and

46% of the global economy, and accounts for over

10% of Canada’s global trade.

The Asia-Pacific is a dynamic region of diverse actors

The Asia-Pacific region is a dynamic space of diverse actors where trade, international assistance, diplomacy and security interests converge. Key regional security, economic, environmental and geopolitical challenges, compounded by China’s increasingly assertive posturing in the region, present opportunities for Canada to diversify its partnerships and recalibrate relations and cooperation with other partners.

In the coming years, several evolving factors may affect Canada’s engagement in the region. Growing authoritarian capitalism may constrain foreign competition, the rule of law and human rights. The strategic tensions between the United States and China may lead to further uncertainty and instability, with repercussions throughout the region. Canada-China bilateral tensions may have negative ramifications for Canadian interests with other countries in the region. Regional tensions, sub-national conflict and environmental disasters have created precarious conditions for millions of people. Over and above existing factors, the impact of COVID-19 is yet to be determined. These trends and issues cross trade, diplomacy and international assistance in a variety of ways. As such, generating insights into the extent of policy and programming coherence in the Asia-Pacific region will aim to support decision-makers in advancing strategies for Canadian engagement in the region.

Branch profile

Human and financial resources

2020-21 Human resources

Figure 1. Number of FTEs by bureau, 2020-21

Text version

OAD

HQ: 79

Mission: 199

OPD

HQ: 57

Mission: 409

OSD

HQ: 42

Mission: 173

Figure 2. Proportion of FTEs at mission and headquarters, 2020-21

Text version

LES: 560

CBS: 232

Headquarters: 183

Figure 3. Breakdown of FTEs by bureau and streams at mission, 2020-21

Text version

OAD

HOM support:43

Trade: 66

Diplomacy: 41

Consular: 13

Development: 36

OPD

HOM support: 46

Trade: 204

Diplomacy: 96

Consular: 61

Development: 2

OSD

HOM support: 50

Trade: 85

Diplomacy: 15

Consular: 9

Development: 14

Human resources, 2016-17 to 2019-20

Between 2016-17 and 2019-20, the number of full-time equivalents (FTEs) increased by 9%, from 893 to 940. Figure 4 reflects actual FTEs reported at year end.

Human resources, 2020-21

OGM information, available as of October 2020, listed a total of 975 FTEs. OPD is the largest bureau in the Branch, with 466 FTEs, including 409 employees in 17 missions (Figure 1).

When considering the allocation of human resources at missions by business line, trade officers represent 45% of all resources; Foreign Policy and Diplomacy Service (FPDS) 19%; consular officers 11%; and development officers 7% (Figure 3). This suggests a significant focus on trade activities, particularly in North Asia and Oceania, the sub-region overseen by OPD.

Figure 4. Asia-Pacific Branch FTEs, 2016-17 to 2019-20

Text version

2016-17

LES: 516

CBS: 377

2017-18

LES: 522

CBS: 377

2018-19

LES: 519

CBS: 394

2019-20

LES: 537

CBS: 403

Financial resources

The Branch’s financial resources were variable in the period between 2016-17 and 2020-21. Notably, there was a significant budget cut in 2017-18 for salary, operating and maintenance (O&M) expenses (Figure 5) as well as in overall funding for grants and contributions (G&C) (Figure 6). However, salary and O&M have more than doubled since then, whereas G&C funding increased significantly in 2018-19 but has since trended downward.

Figure 5. Salary and operation & maintenance,2016-17 to 2020-21

Text version

2016-17

$67,410,645

2017-18

$46,441,099

2018-19

$84,993,248

2019-20

$92,762,098

2021 (initial allocation)

$96,134,624

Figure 6. Grants and contributions, 2016-2017 to 2020-2021

Text version

2016-17

$224,963,537

2017-18

$207,789,697

2018-19

$307,259,887

2019-20

$283,517,971

2021 (initial allocation)

$276,297,182

Evaluation scope and methodology

Strategic considerations for coherence

Conduct of the evaluation

The evaluation was conducted internally by the Evaluation Division (PRA), with the support of 2 independent consultants. One of the consultants worked on developing and conducting a survey of OGM staff. The other consultant supported the drafting of 1 of the 4 cases studies realized as part of the evaluation.

Methodological approach

The evaluation used a mixed-method approach, which involved triangulation across lines of evidence to provide more robust findings. The report has 2 levels of analysis: general insight into coherence within the Branch and in-depth analysis of tangible examples of coherence using case studies.

Seeing that there are multiple interpretations of coherence, that it is a dynamic and evolving concept, and that coherence is not an end goal but rather an enabler of greater outcomes, the evaluation took an exploratory approach in its assessment. It integrated elements of appreciative inquiry and goal-free evaluation to generate findings.

To provide rigour and comparability, the evaluation included an organizational factor framework that built on the work conducted in the previous 2 evaluations. The organizational factors helped generate insights into the conditions that enabled coherence within the Branch, with specific emphasis on the elements that contributed to successful coordination of efforts in the case studies.

Evaluation scope and objectives

Evaluation scope

The evaluation focused on policy and programming coherence among the 3 business lines (“streams”) of diplomacy, trade and international assistance in the Asia-Pacific Branch. More specifically, it explored how expertise was leveraged across these business lines, both at headquarters and in the Canadian missions in the region.

Although the emphasis of the evaluation was on OGM activities, it also considered the broader scope of programming in the Asia-Pacific region. Where relevant, the scope included activities conducted by other branches, including Trade Policy and Negotiations (TFM), International Business Investment and Innovation (BFM), Global Issues and Development (MFM), and International Security and Political Affairs (IFM). The evaluation covered the period from 2015-16 to 2020-21.

The following elements were excluded from the evaluation scope to calibrate the level of efforts with the resources available to conduct the evaluation:

• Afghanistan, which was the object of a country evaluation and approved in 2019-20
• coherence with other government departments (OGDs) located at missions
• coherence between OGM and Canada’s involvement in multilateral organizations (although some aspects were covered in the case studies)

Evaluation objectives

• identify the conditions that have enabled policy and programming coherence
• identify examples where expertise and perspectives have been integrated across business lines to achieve greater outcomes
• identify lessons learned related to coherence that can be applied to other branches in the department

Emergence of a stable coherence measurement framework

The measurement of coherence has evolved based on lessons learned from 2 previous evaluations.

1. Sub-Saharan Africa Branch

This first coherence evaluation was a learning opportunity and denoted the challenge of defining coherence and measuring its contribution to improved results across diplomacy, trade and international assistance. The evaluation assessed the degree to which key elements of organizational coherence (policy alignment, organizational structure, branch leadership, corporate systems, roles and responsibilities, and communications) were in place within the branch to enable effective coordination and collaboration across business lines.

The evaluation identified 4 key coherence areas across business lines: diplomacy-trade, diplomacy-international assistance, trade-international assistance and diplomacy-trade-international assistance.

The identification of cross-stream initiatives was not the focus of the evaluation, so the exercise provided only limited examples of joint initiatives.

2. Americas Branch

Building on the work of the first evaluation, this evaluation focused on coherence across diplomacy, trade and international assistance, and identified 4 coherence areas: diplomacy-trade, diplomacy-international assistance, trade-international assistance and diplomacy-trade-international assistance. This evaluation had a particular focus on 3 issues: coherence in programming and results, organizational coherence, and delivery models to strengthen coherence.

The evaluation developed a scorecard to assess factors of organizational coherence (policy alignment, organizational structure, branch leadership, corporate systems, and communications). The definition of coherence was updated.

Efforts were made to document cross-stream initiatives and therefore assess whether coherence was being achieved.

3. Asia-Pacific Branch

The third coherence evaluation has refocused on the key organizational factors and conditions that enabled policy and programming coherence (policy alignment, organizational structure, branch leadership, corporate systems, and capacity and expertise). Coherence factors have been streamlined by removing 2 factors to address lessons learned from the Americas coherence evaluation. One new factor was added: capacity and expertise. The measurement framework for the organizational factors has been revised and updated to include new indicators to measure each factor. This new framework features an updated concept of coherence which now includes 2 principal dimensions: collaboration and coherence thinking. Coherence with other branches was also explored.

Cross-stream initiatives have been systematically identified and new indicators were used to assess whether coherence was being achieved.

Methodology

To maximize the possibility of generating useful, valid and meaningful findings, the evaluation used a mixed-method approach, wherein both qualitative and quantitative data were collected. Extensive use of triangulation was undertaken as an analytical method, in which data from multiple lines of evidence were examined to help corroborate findings. The methods listed below were deemed to be the most appropriate ones to answer the evaluation questions, based on data availability and project imperatives.

Evaluation limitations and mitigation measures

Findings – Profile of collaboration within the Asia-Pacific Branch

Overview of coherence in the Asia-Pacific Branch

Figure 7: The 4 key coherence areas covered by the evaluation, excluding the Triple Nexus (development, humanitarian assistance, and peace and security)

1. Diplomacy and development
2. Diplomacy and trade
3. Development and trade
4. All streams

Figure 8: Visual representation of the coherence measurement framework based on empirical evidence collected as part of the evaluation

Text version

Figure 8: Visual representation of the coherence measurement framework based on empirical evidence collected as part of the evaluation that features leadership, organizational structure, capacity and expertise, and alignment of priorities, all pointing towards OGM coherence.

A culture of collaboration is emerging within OGM. Several examples of cross-stream initiatives were identified.

One of the main objectives of the evaluation was to provide an assessment of the extent to which OGM operated in a coordinated and coherent manner. The assessment was performed by 1) measuring the level and type of collaboration across streams, 2) asking OGM staff to share their perception of the value of working with other streams, and 3) examining the extent to which OGM messaging to external partners in the region was coherent.

Level of collaboration across the streams

The evaluation found that streams frequently collaborated on joint files. More specifically, OGM staff reported to have worked on an average of 2 joints files over the last 3 years. Moreover, a large proportion of survey participants—43% of OGM employees—reported to have worked on at least 4 or more cross-stream initiatives. The diplomacy stream (56%) and executives (77%) reported having worked on a least 4 joint files, the highest level of collaboration within the branch. Several examples of cross-stream collaborations were also identified as part of the 4 case studies, the survey open-ended questions and the key informant interviews. These findings suggest that a culture of coherence is emerging at OGM and across the department.

Findings from other lines of evidence complemented the survey results by providing several concrete examples of cross-stream collaboration. In fact, almost all interviewees were able to identify 1 or 2 examples of meaningful collaboration across the streams. Additionally, the case studies provided robust evidence that successful cross-stream initiatives have been delivered in the branch over the last 5 years. It must be noted, however, that although successful initiatives were identified, many evaluation participants also noted untapped coherence areas, where increased collaboration could lead to enhanced impacts. In light of this information, it appears that the efforts deployed by OGM senior management over the last 5 years to foster collaboration and coherence have yielded results.

Collaboration network across the streams

Figure 9. Diplomacy frequently works with all streams and programs, whereas limited collaboration is occurring between trade and development

Text version

Figure 9:

Trade

All: 66%

Current stream International assistance: 40%

Current stream Diplomacy: 66%

Current stream Trade: 79%

Diplomacy

All: 77%

Current stream International assistance: 84%

Current stream Diplomacy: 82%

Current stream Trade: 73%

Development

All: 31%

Current stream International assistance: 60%

Current stream Diplomacy: 39%

Current stream Trade: 12%

Humanitarian

All: 12%

Current stream International assistance: 24%

Current stream Diplomacy: 14%

Current stream Trade: 4%

Peace & Security

All: 14%

Current stream International assistance: 16%

Current stream Diplomacy: 23%

Current stream Trade: 7%

Key collaboration networks within and outside the Asia-Pacific Branch

This section relies primarily on the results of a regression analysis, which provided robust evidence that helped to measure and qualify the relationships across the different streams. This analysis was complemented by triangulating these results with data from other lines of evidence. While the results stemming from the regression analysis were expected given the insight provided by previous coherence evaluations, the robustness of the evidence it generated could not have been achieved by relying solely on qualitative data analysis. The analysis showed the following results:

• International assistance: The evidence showed that development did not frequently collaborate with trade as there was a negative and significant correlation between these streams. Similarly, very few examples of joint collaborations between these 2 streams were identified in other lines of evidence.^{Footnote 1} However, the advancement of a Canada-ASEAN FTA case study provided a more nuanced perspective on these general conclusions, as this initiative was found to be a stellar example of how collaboration across these 2 streams, although challenging to deliver, could yield unmatched results when compared with a siloed approach.
• Trade: Trade reported a significant and negative correlation with both development as well as peace and security programs, thus indicating that little collaboration was taking place between these streams. Although the regression analysis did not find any significant relationships with diplomacy, results from other lines of evidence suggest that diplomacy and trade frequently collaborated on joint initiatives or coordinated their efforts, when relevant. In addition, positive examples of collaboration between these streams were often accompanied with examples of missed opportunities for increased collaboration in areas such as climate change and clean technologies, human rights, security, and commercial diplomacy.
• Diplomacy: There was a positive and significant relationship between diplomacy and peace and security programs, which confirms that these streams frequently collaborated on joint initiatives. Although the regression analysis did not find any significant relationships with either trade or development, there was strong evidence that suggested that diplomacy played a pivotal role in supporting all other streams when opportunities for joint initiatives emerged. In fact, findings from the qualitative data analysis suggest that diplomacy appeared to be a natural partner for most streams or branches when collaboration was expected or desired. This could be explained, in part, by the flexibility that diplomacy appears to have when compared with other streams, but particularly when compared with the development stream, which is perceived by other streams as more rigid. Despite this perception of rigidity, other evidence showed that the development stream can be flexible.

OGM staff’s perception of coherence

Figure 10. Staff in all 3 streams reporting that they personally value working on joint files

Text version

Figure 10:

Trade

Agree: 85%

Do not agree: 15%

Diplomacy

Agree: 89%

Do not agree: 11%

Development

Agree: 88%

Do not agree: 13%

Figure 11. Perception that other staff within their own stream value working on joint files

Text version

Figure 11:

Trade

Agree: 71%

Do not agree: 29%

Diplomacy

Agree: 79%

Do not agree: 21%

Development

Agree: 71%

Do not agree: 29%

Figure 12. Perception that other staff outside of their own stream value working on joint files

Text version

Figure 12

Trade

Agree: 55%

Do not agree: 45%

Diplomacy

Agree: 63%

Do not agree: 37%

Development

Agree: 56%

Do not agree: 44%

OGM staff from all 3 streams underestimated the willingness of others to collaborate on joint initiatives. Coherence of communication with external partners could be improved to ensure that Canada speaks with one voice.

OGM staff views on the value of coherence

Overall, 87% of OGM staff stated that they personally value joint initiatives. This proportion significantly drops when survey participants are assessing staff in their own stream or outside their stream (see Figures 11 and 12). These results suggest that staff underestimate the appetite of colleagues outside of their streams to work on joint files.

Coherence of Canada’s external messaging in the Asia-Pacific region

The consistency of Global Affairs Canada’s messaging to outside partners was also used as a proxy metric to assess the level of coordination and coherence within the region. Mixed results were noted across the streams:

• Only 42% of survey participants reported that OGM provides consistent messaging to outside partners as it pertains to Canada’s priorities. Staff in executive positions had a more positive perception (58%) of Canada’s external messaging in the region. These results suggest that real opportunities for improvement exist on this front within the region.
• 63% of survey participants from the North Asia and Oceania bureau (OPD) felt that Canada’s external messaging in the region was not coherent. The higher level of disagreement expressed by OPD staff could potentially be explained by the fact that this bureau, which is leading the coordination of Canada’s whole of government approach for China, was more aware of potential communication challenges given that it is responsible for addressing such issues. It must be noted that other lines of evidence also pointed to opportunities for improvement with regards to Canada’s formal positioning toward various China-related issues. OGM is currently spearheading whole-of-government efforts to clarify Canada’s posture in the region, including improving messaging around China-related issues.
• The 4 case studies provided a considerably more positive perspective on this topic. These showed that a strong level of integration across streams can help improve the communication of Canada’s priorities in the region. Conversely, allowing silos can have a negative impact on external messaging (see Appendix C).

Findings – Impacts of coherence

Typology of joint initiatives

OGM undertook 4 types of joint initiatives. The provision of expertise and the delivery of Global Affairs Canada flagship initiatives have yielded the strongest impacts.

The evaluation closely examined the impacts stemming from cross-stream collaboration with the purpose of assessing the added value of working in a coherent manner.

Staff’s perception of the contribution of other streams on successful impacts

OGM staff recognized the contribution of other streams in the success of joint projects, as a large number of OGM staff (82%) agreed that the contribution of other streams was instrumental in achieving project results. Interestingly, disaggregated data shows there was little variation across the different groups regarding the contribution of other streams. This finding suggests that the same level of impacts could not have been achieved in the absence of cross-stream collaboration.

Typology of joint initiatives

Overall, a mosaic of joint initiatives was identified as part of the evaluation. These initiatives and activities varied depending on a series of factors such as 1) the level of integration of the initiative, 2) the presence (or absence) of key drivers for coherence, and 3) their complexity and their duration in time. The qualitative analysis identified the following types of initiatives, from the lowest level of integration across the streams to the highest:

1. Senior management briefings and preparation of Cabinet documentation
   • integrated and coordinated approach for Minister of Foreign Affairs briefings, Cabinet documents, and any briefing that involves multiple streams
2. Joint ad hoc events
   • joint advocacy events and/or policy dialogue with common partners on topics of shared interest, anniversaries of bilateral relationships, senior official visits and trade shows
3. Provision of stream-specific expertise, knowledge and network to support decision-making
   • COVID response, trade advice provided to development on the Canadian Trade and investment Facility for Development (CTIF) and an infrastructure program, leveraging of networks of contacts, engaging in joint advocacy outreach, and SEED scholarship and EduCanada
4. Delivery of Global Affairs Canada priorities or flagship initiatives for the Asia-Pacific region
   • Advancement of the Canada-ASEAN FTA, Rohingya response, United Nations Social Campaign, opening of mission in Fiji and Kolkata trade office, coordination of Canada’s whole-of-government approach on China, and the Indo-Pacific strategy

Impacts stemming from joint initiatives

Figure 13. All streams reporting on whether cross-stream collaboration leads to increased efficiencies

Text version

Figure 13:

Trade

Agree: 72%

Do not agree: 28%

Diplomacy

Agree: 70%

Do not agree: 30%

Development

Agree: 80%

Do not agree: 20%

Cross-stream initiatives have resulted in strong impacts and increased efficiencies that could not have been achieved through a siloed approach.

Key impacts - The following 3 categories of impacts were identified:

1) Emergence of a culture of coherence thinking and collaboration across the department

• strengthened relationships, knowledge and understanding of other streams by OGM and Global Affairs Canada staff
• increased ability to identify opportunities for collaboration across the streams (i.e. bottom-up coherence thinking)
• new skills and expertise among staff to champion and foster coherence capacity in the department
• establishment of new partnerships and approaches to collaboration (e.g., Expert Deployment Mechanism for the Canada-ASEAN FTA case study, SEED common platform, use of the Crisis Pool Quick Release Mechanism for the Rohingya response)

2) Increased efficiencies

• 72% of survey participants felt that the joint files in which they were involved led to increased efficiencies
• Trade facilitated access to sectoral expertise and ideas about the Infrastructure sector to support the creation of a new development program in Vietnam. As a result of this collaboration, the program was established 1 year earlier than expected
• OGM used a new coordinated approach across the sector to develop the Indo-Pacific strategy as well as in the preparation of key briefing material. More specifically, ministerial briefings prepared for the Indo-Pacific strategy, response to the coup in Myanmar, and human rights in Xinjiang province, China, were reported to have successfully integrated information and advice from multiple business lines. This approach is expected to result in a highly coordinated approach to the Government of Canada’s current and future presence in the region.
• Clear roles and responsibilities were established to prevent porous communication lines or duplication of efforts, thus reducing the risk that streams would work at cross purposes

3) Enhanced impacts (DRF, mandate letters)

The evidence suggests that cross-stream collaboration contributed to furthering the achievement of Departmental Results Framework (DRF) expected outcomes as well as ministerial mandate letter commitments. Additionally, there is strong evidence that cross-stream collaboration was a key mechanism that strengthened the ability of each stream to achieve their stream-specific objectives. While a large diversity of impacts was identified, the following 4 general types of impacts were the most frequently reported by evaluation participants:

• improved bilateral and multilateral (ASEAN) relationships
• increased access to funding instruments to further Canada’s priority initiatives (EDM, SEED, Crisis Pool)
• expanded and strengthened network of influence
• enhanced ability to understand and address complex issues that required a diversity of expertise

Impacts – Trade and development

Trade and development coherence area

The evaluation found the following trade and development nexus points where trade and development share synergies:

• Cleantech and climate finance
• Gender equality (e.g. women’s equal access to capital markets, technology, business development services)
• Free trade agreements (e.g. progressive trade chapters, capacity building to negotiate and/or implement agreements)
• Development programming in agriculture and infrastructure for middle-income countries (e.g. capacity building in public-private partnerships)
• Education scholarships (e.g. SEED)

Text version

[Figure: Intersection of Development and Trade Focal Points]

The approach to furthering the Canada-ASEAN FTA illustrated that the department made significant progress in breaking down silos across streams and branches. It also illustrated the strong added value that cross-stream collaboration can generate.

A review of the documentation covering the trade and development nexus provided ample evidence of the rationale for and advantages of the 2 streams working collaboratively to achieve departmental outcomes, the Sustainable Development Goals (SDGs) and other international commitments. However, almost all the examples of trade and development initiatives identified in the literature focused on multilateral programming (e.g. WTO) or bilateral development programs with very limited integration between the streams. To provide insights about how the development and trade nexus can be operationalized in the Canadian context and to better understand OGM’s key achievements on this front, a case study of the most integrated approach to cross-stream collaboration—a Canada-ASEAN FTA—was undertaken (see Appendix C for full list of achievements).

Advancement of a Canada-ASEAN FTA – key achievements

The Canada-ASEAN FTA initiative has provided a focus point for all 3 business lines to leverage and advance a bilateral relationship with the ASEAN Secretariat and its member states, and to demonstrate Canada’s interests in the region. The evidence suggests that a Canada-ASEAN FTA would not have advanced to the point it has had it not been for cross-stream and cross-branch collaboration and considerations. Adapting trade negotiations with least-developed countries (LDCs) to new circumstances has led to key achievements in the trade and development nexus, including the following:

• As a result of Canada’s on-going engagement for a potential Canada-ASEAN FTA, ASEAN member states agreed to discuss launching negotiations as a priority economic deliverable for 2021
• Streams leveraged each other’s expertise and networks in joint initiatives to further negotiations efforts. Development was leveraged by trade, in particular through technical assistance and through the good standing and reputation of previous aid programming; development was leveraged by diplomacy and provided advantages to bilateral relationships and advocacy campaigns. Diplomacy was leveraged by trade and development through advocacy, bilateral relationships, and the provision of political context and networks
• The new Expert Deployment Mechanism (EDM) demonstrated the possibilities of mutually beneficial collaboration to all streams involved. Without the EDM, trade policy experts and negotiators would be less able to respond to technical assistance requests from ASEAN’s member states and Secretariat
• The SEED scholarship was established to strengthen relationships with ASEAN member states. This program demonstrated Canada’s commitment to the region, which supports trade’s broader efforts to continue its on-going negotiations for a Canada-ASEAN FTA

Impacts – Diplomacy and the other streams

Several missed opportunities for collaboration between trade and diplomacy were identified as part of the evaluation, particularly in large missions.

Diplomacy and trade

Findings from the key informant interviews showed that trade and diplomacy usually coordinated their key decisions, policies and initiatives at the strategic level. However, very few examples of cross-stream initiatives were reported by evaluation participants at the operational level. The main examples of collaboration in this nexus area included:

• Commercial diplomacy: These types of initiatives involve the mobilization of diplomacy’s advocacy efforts and network of contacts to influence trade policy issues or the use of commerce as an advocacy tool for diplomacy. Some of the most successful initiatives reported include the collaborative work done by the streams to address an export ban on pharmaceutical and personal protective equipment (PPE) imposed by India at the beginning of the COVID pandemic. The work performed by trade and diplomacy helped remove major barriers that would have had an impact on Canada’s response to the pandemic. More importantly, a few interviewees noted that in certain markets such as China and India, politics and business are closely related, and commercial diplomacy is essential to opening markets for Canadian industries. This situation with China also requires a coordinated approach, as political issues are closely linked to trade-related issues and vice versa.
• Market intelligence analysis: In a few missions, diplomacy provided analysis on trade/industrial policies and market access (macro economy). This information was used by trade to better inform Canadian companies interested in key markets in the region.
• Public affairs: Public affairs were also a point of convergence across these streams. Many interviewees noted that improvement could be made at mission on this front to improve advocacy and promotion of Canada, in general.
• Education: Trade promoted Canadian universities to students, and diplomacy engaged and expanded networks of alumni and relationship between foreign and Canadian universities.

Diplomacy and development

The evaluation documented several examples of joint initiatives between diplomacy and development. More specifically, the case studies on the Rohingya response, the Canada-ASEAN FTA and the Mongolia extractive sector clearly outlined the benefits and challenges of collaboration between these 2 streams (see Appendix C for details on impacts).

Findings – Leadership

Leadership

Mechanisms to support coherence thinking

Leadership was found to be the primary factor in terms of its impact on coherence, and superseded all other organizational factors.

Strong evidence indicated that the “leadership” organizational factor was key in fostering cross-stream coordination, collaboration and communication at all levels. This factor was also found to be pivotal in creating and allowing space for staff to think strategically and creatively across streams to generate new and innovative approaches to joint initiatives.

The coherence evaluation framework defined 5 key dimensions to measure the leadership factor:

• mechanisms established by management to foster coherence thinking
• perceived prioritization of coherence by senior management
• identification of tangible joint initiatives by management
• provision of guidance to staff
• decision-making process

A sixth factor emerged from the data analysis: the role of middle-management as a factor of success to achieved coherence.

Integrated annual strategic planning

One of the main goals of the evaluation was to better understand the mechanisms used by senior management at missions to define shared priorities and identify tangible joint initiatives. In this regard, the interview data showed that the annual planning process was the main mechanism through which coherence thinking materialized at mission. However, the interview data showed that each stream’s level of participation in the annual planning process varied across missions and that there was no standardized approach for the integration of streams in missions. These observations were echoed by survey findings which indicated that 69% of executives reported that the planning process included other streams. These results suggest that room for improvement exist in terms of fostering coherence thinking as part of the annual planning process.

Evidence showed that the participation of multiple streams in the integrated annual planning cycle did not guarantee that coherence thinking on the part of senior management would lead to tangible joint initiatives at the working level. When leadership did not identify a joint initiative as a priority, other organizational factors determined if an opportunity for cross-stream collaboration would materialize. Other factors included:

• staff capacity (skills and workload)
• knowledge of other streams
• presence/absence of incentives (PMA objectives, career promotion, prizes, etc.)
• personalities
• adaptability of streams to engage on a meaningful joint initiative

Leadership

Prioritization and guidance

Provision of the right guidance

Figure 14. Only one third of OGM staff from the diplomacy stream state that senior management provides guidance to staff regarding coherence

Text version

Figure 14:

Trade

Agree: 41%

Do not agree: 59%

Diplomacy

Agree: 34%

Do not agree: 66%

Development

Agree: 43%

Do not agree: 57%

The success of joint initiatives was in large part determined by the direction and guidance provided by senior management to staff.

Identification of joint initiatives by senior management

As noted in the previous sections, the evaluation identified several instances of successful joint initiatives as well as several missed opportunities in key coherence areas. There was also evidence that in the case of mandated joint initiatives, management provided clear direction to staff by proposing tangible joint initiatives. However, only 51% of OGM staff believed that senior management (i.e., ADM, HOM, DG levels) tried to identify joint priorities across the 3 streams. These mixed results converged with other lines of evidence, which noted that coherence may be centralized in a few clusters of collaboration within the branch, thus explaining the split views on this element of the coherence framework.

Prioritization of coherence by senior management

The evaluation also sought to better understand how OGM staff perceived how coherence ranked in terms of OGM senior management priorities. While there is strong evidence to suggest that senior management valued coherence and were making efforts to foster coherence within OGM, only a small proportion—39%—of OGM staff felt that coherence ranked high among senior management priorities. This could indicate that coherence thinking occurs primarily at the strategic level. Results from the multivariate regression analysis showed that OGM prioritization of coherence was perceived more positively by 1) the development and trade streams, 2) staff at mission rather than at HQ, and 3) smaller-sized missions that had fewer than 50 staff. Surprisingly, diplomacy staff felt that senior management did not prioritize coherence despite their strong involvement in joint initiatives.

Provision of guidance to staff

The evaluation found that senior management did not provide sufficient guidance on what coherence meant in practical terms. Within OGM, only 36% of staff believed that senior management provided the necessary guidance to engage in cross-stream collaboration. Interestingly, the diplomacy stream reported the lowest level of agreement despite its frequent collaboration with other streams. The data presented in Figure 14, for its part, indicates that both trade and development did not feel that they were provided with the right guidance to deliver joint initiatives. There is therefore evidence to suggest that the lack of guidance and the lack of tangible propositions by senior management were hurdles to the achievement of coherence. Case study findings suggested, however, that when there was a clear signal and direction to prioritize joint initiatives, they tended to materialize. The strategic identification of joint initiatives via effective integrated planning was primarily observed in the case of mandated joint initiatives such as the advancement of a Canada-ASEAN FTA, the Rohingya response and the SEED scholarships initiatives. Strong leadership in this context provided clarity and purpose, and effectively enabled coherence across complex organizational structures that at times included up to 3 branches, HQ and missions, and development program delivery models (i.e. centralized, decentralized).

Leadership

Other factors

Staff perception regarding decision-making process

Figure 15. The three streams share a similar view of the extent to which the decision-making and approval processes foster joint initiatives

Text version

Figure 15:

Trade

Agree: 38%

Do not agree: 62%

Diplomacy

Agree: 40%

Do not agree: 60%

Development

Agree: 31%

Do not agree: 69%

Figure 16. There is a strong gap in perception between non-executives and executives regarding the extent to which the decision-making and approval processes foster joint initiatives

Text version

Figure 16:

Non-executive

Agree: 28%

Do not agree: 72%

Executive

Agree: 71%

Do not agree: 29%

Middle management’s role in fostering coherence

The role of middle management (i.e. below HOM/DHOM and directors general) was found to play a central role in the success of joint initiatives. Non-mandated initiatives and staff proposals for joint initiatives (i.e. bottom-up ideas for collaboration) were reported to sometimes be inadequately considered by middle management. Simultaneously, employee workload, the lack of guidance from the department around the delivery of joint initiatives, the absence of incentives for collaboration, and cumbersome decision-making and approval processes deterred middle management from prioritizing coherence initiatives involving multiple streams. On this last point, only 33% of OGM staff indicated that current decision-making and approval processes supported coherence.

Conversely, the case studies showed that when there was an established middle-management-level coordination structure across branches and bureaus, processes were not barriers to collaboration. This was observed in the case study on the advancement of a Canada-ASEAN FTA, where a joint approval process was developed at the deputy director level to coordinate simultaneous approvals across a multi-divisional and multi-branch organizational structure in an effective manner.

Findings – Capacity and expertise

Expertise

Skills and training

Figure 17. All streams report that incentives for working on joint files are not currently available

Text version

Figure 17:

Trade

Agree: 24%

Do not agree: 76%

Diplomacy

Agree: 21%

Do not agree: 79%

Development

Agree: 25%

Do not agree: 75%

Staff confidence in their skill sets for collaboration were high, but there may have been gaps in practical knowledge and competencies that hindered efforts for effective collaboration.  

The analysis framework established to measure coherence includes 2 new elements that pertain to capacity and expertise. The first element examines how OGM staff perceives their own ability to engage with other streams; the second covers all other potential barriers to coherence as it pertains to this factor.

Coherence-related skills

Overall, more than 70% of OGM staff reported having the skills required to engage in cross-stream collaboration. This was particularly the case for employees in the diplomacy stream and for executives. However, interview findings also pointed to 2 main critical gaps in OGM staff’s current skill set:

• Limited knowledge of other streams: Most evaluation participants indicated that it was important to have a basic understanding of how the different streams operate and what their priorities are; they also noted that this issue goes beyond the branch itself. Suggested approaches to filling this knowledge gap included formal training, team-building activities, coaching or cross-stream appointments.

• Coherence thinking: Further capacity and expertise building in coherence areas was identified as a need that could foster more knowledge sharing and effective programming, especially in the trade and development nexus and the Triple Nexus. Aggregate evidence suggested that coherence thinking mostly occurred in the context of a mandated joint initiative and when strategic joint planning or programming was incentivized. An interviewee highlighted the importance of creativity as a key competency to enhance coherence thinking and expanded engagement in joint initiatives: “Not skill or capacity [issue], there is creativity involved. Officers put their head down. People get very comfortable in their space of work. That [i.e. cross-stream initiatives] pulls them out. It is about larger vision. Bigger picture thing, creative thinking.”

In terms of the preferred format for learning, most interviewees felt that in-class training had value but expressed preferences for on-the-job training activities such as cross-stream team building, coaching or cross-stream appointments.

Incentives

The perception of several interviewees was that stream-specific experience and individual achievements were rewarded and contributed to career progression—a perception that could impede the impulse to collaborate. This perception was shared across the streams but particularly within the diplomacy stream. Interviewees suggested a few ways to incentivize staff, including:

• encouraging cross-stream appointments or job-shadowing
• identifying cross-stream objectives in employee’s PMA in non-executive positions
• offering rewards/prizes (i.e. recognition) for participation in joint initiatives or for behaviours that foster coherence

Capacity and incentives

Financial resources in support of cross-stream initiatives

A review of the key sources of funding available to OGM highlighted the absence of specific resources dedicated to supporting joint initiatives. Most of the cross-stream initiatives studied as part of the evaluation were supported with stream-specific resources. The absence of funding was also found to limit, in some instances, staff’s willingness to engage in these types of initiatives.

The evaluation also found that some sources of funding that are not dedicated specifically to supporting coherence had a significant impact on the ability to deliver joint initiatives and bring streams together. The 2 examples identified are the EDM and the Crisis Pool (see Appendix C). The evidence shows that these 2 funding mechanisms allowed for flexibility, planning and program delivery. Overall, furthermore, they fostered more coherent and effective strategic approaches for new, quick and purposeful programming that enabled collaboration and coherence thinking. In this regard, the case studies found that in the absence of these 2 funding mechanisms, cross-stream initiatives would not have achieved the same level of success.

Capacity and workload

There was consensus among interviewees, including senior management, that the capacity issues that OGM experienced over the last 5 years was one of the main hindrances affecting coherence and willingness to engage with other streams. These views align with the results of the 2019-20 Risk at Mission report prepared by Corporate Planning Performance and Risk (SRD). For OGM, the risk analysis found the following:

• 27% of missions located in the South, Southeast Asia and Oceania regions identified “people management,” which includes capacity, as a top-3 risk. For comparison, only 1 region out of 9 (i.e. SRD regional classification) reported a higher percentage of missions that identified people management as a top-3 risk.
• 8% of missions located in the North Asia region identified people management as a top-3 risk, the lowest proportion of the 9 regions
• The results of the 2020 Public Service Employee Survey also highlighted the challenges faced by staff as it pertained to capacity (see questions 17, 18[c], 70[e] and 70[f]). In fact, of the 4 survey questions related to workload and its impact on stress levels, OGM, as well as the other 3 geographic branches, ranked in the lower tier when compared with other branches in the department. Only the MIN, MFM and WGM branches reported heavier workloads and increased stress levels associated with workload.

Findings – Organizational structure

Organizational structure

The OGM’s organizational structure was not a significant barrier to coherence.

Survey participants had mixed opinions regarding the extent to which the current OGM structure fostered coherence within their streams. While an almost even split was observed at the aggregate level, disaggregated data by position and bureau provided a more nuanced picture. All executives felt their structure fostered coherence, while only half of staff found the same. Even fewer staff within the South-Asia bureau (OSD)—the only bureau not to undergo a restructuring over the last years—felt that their structure supported coherence, suggesting that the amalgamation that took place in other bureaus had a direct impact on perceptions of coherence. Interestingly, the case studies found that in the matter of mandated initiatives, the OGM’s structure was rarely a significant barrier to the successful delivery of joint initiatives.

In addition to the survey results, findings from the interviews and case studies highlighted the following:

• Employees from the South Asia bureau (OAD) had a significant and positive view of the structure of their bureau when compared with other bureaus
• Being in a small mission improved the likelihood of operating in a coherent manner
• Colocation of employees improved coherence by increasing the knowledge and understanding of other streams
• The division of Myanmar and Bangladesh into separate bureaus created coordination challenges, which were highlighted by the refugee crisis and the coup d’état in Myanmar
• Built-in coordination structures that provided policy advice and support to Cabinet and GAC senior management were found to help strengthen coordination and coherence across the Government of Canada (China committee)
• The ASEAN mission played a central role in coordinating initiatives in support of ASEAN-related priorities

Interviewees and key documents identified department-level opportunities for enhanced coordination across the various bureaus that share roles and responsibilities around the trade and development nexus. In this regards, strategic planning between the Trade Policy and Negotiations and Chief Trade Negotiator NAFTA branch and regional branches (i.e. OGM) were found to play a key role in enabling operational coherence at a regional programming level. In addition, it should be noted that the trade branch plays a convening role in terms of multilateral trade and negotiations (i.e. APEC, G7, G20, OECD, WTO), which can provide additional strategic insights to regional branches. The Strategic Planning, Policy and Operations division (OAZ) of the Asia-Pacific Branch has shown that it plays an important and successful coordination role in this nexus area.

At the departmental level, senior managers interviewed also stressed the inefficiencies and level of effort associated with briefing different ministers on a same topic. These interviewees explained that to address this issue, a framework was established for ministerial briefings on the Indo-Pacific Strategy and the coup in Myanmar. In interviewees’ views, these coordinated briefings were well received by MIN.

Organizational structure

Adaptability

Flexibility

Figure 18. Development perceives its flexibility to collaborate on joint initiatives by other streams less positively

Text version

Figure 18:

Trade

Agree: 75%

Do not agree: 25%

Diplomacy

Agree: 73%

Do not agree: 27%

Development

Agree: 58%

Do not agree: 42%

The 3 streams had sufficient flexibility to engage in cross-stream collaboration, although they were limited by features inherent to their relative streams.

Overall, 71% of OGM staff felt that their stream had the flexibility to collaborate on joint initiatives. While no major differences were observed by position and bureau, Figure 18 suggests that development has less flexibility than other streams.

Diplomacy

Diplomacy was by far the stream having the most flexibility. As shown by evidence laid out in previous sections, the diplomacy stream was one of the main nodes of Global Affairs Canada’s collaboration system. In fact, almost all joint initiatives identified as part of the evaluation involved the contribution of the diplomacy stream. The multivariate analysis and the 4 case studies provide robust evidence of the flexibility and added value of engaging diplomacy in joint files. The fast-paced nature of the work performed by diplomacy, its broader roles and responsibilities, and the basic skills required of a diplomat all contributed to diplomacy’s inherent flexibility.

Trade

Evaluation participants shared mixed views on the level of the trade stream’s flexibility. For most interviewees, the imperative for trade commissioners to achieve TRIO2 key performance indicators (KPIs) sometime negatively affected their willingness to engage in joint initiatives. Nonetheless, many interviewees also noted that trade was very flexible when leadership supported initiatives.

Development

The long-term objectives pursued by development combined with the rigid financial and accountability management systems that govern its operations limited this stream’s ability to rapidly adapt. Some interviewees from this stream felt that some OGM staff were not fully aware of limitations in the flexibility of development programming. As stated by 1 interviewee from the development stream, “We can’t use development funds for political whims. Development funds can’t respond to an immediate need and answer to a politics issue—but they can be used in other ways.”

Despite this, strong evidence suggested that development had the flexibility to engage and align with other business streams at the programming design stage rather than at the stage that a project was approved, and funds committed. The case study on the advancement of a Canada-ASEAN FTA provided convincing evidence of the added value that development can provide to other streams. Case study interviewees reported that the development stream was open, supportive and flexible when collaborating with the trade stream, although interviewees mentioned that it took many years to bring development on board. The establishment of the EDM was a stellar example of trade and development pooling their resources to deliver on departmental priorities that cut across stream-specific objectives.

Findings – Other factors

Alignment of priorities and programs

Perception of priorities and programming alignment

Figure 19. Trade agrees less than diplomacy and development with the statement that initiatives delivered at mission support the priorities of the OGM branch

Text version

Figure 19:

Trade

Agree: 68%

Do not agree: 32%

Diplomacy

Agree: 80%

Do not agree: 20%

Development

Agree: 82%

Do not agree: 18%

Figure 20. A sizeable number of interviewees at HQ do not seem to be aware of the priorities of their stream

Text version

Figure 20:

Mission

Agree: 67%

Do not agree: 33%

Headquarters

Agree: 52%

Do not agree: 48%

 “No system is going to be perfect. Trying to change a system can be insurmountable. If it’s not broken, I would not fix it. I am weary of trying to convert into 1 converged system. […] Fundamentally, the corporate systems are not a barrier to coherence as they are now.”

The factors policy & priority alignment and corporate systems & processes had a limited influence on coherence.

Alignment of priorities and programming

OGM’s main priorities, especially those established at headquarters, were clearly laid out in the Asia-Pacific Branch Strategic Plan, updated annually. At the mission level, the annual planning process for trade (i.e. BFM Branch) and diplomacy (supported by Strategia) captured mission priorities. Other lines of evidence and survey findings suggest that although priorities were available for staff to consult, many employees indicated that the priorities of their streams were not clearly communicated to them by management. In fact, only 64% of OGM staff noted that that senior management had clearly communicated the priorities of their stream to them. This was found to be particularly problematic at headquarters, as illustrated in Figure 20. Interestingly, the lack of knowledge of priorities did not appear to be a major barrier to collaboration, as 73% of OGM staff noted that initiatives delivered at mission supported the branch’s priorities. For its part, the multivariate analysis concluded that staff in executive positions and staff located at headquarters had a significant and positive view on the level of alignment of priorities at mission.

An overview of some of the main evidence of collaboration with other branches is available in Appendix D.

Corporate planning and reporting systems

With regards to this factor, few evaluation participants raised coherence-specific issues. Most of the discussions about systems revolved around the inherent advantages and disadvantages of each system (e.g. TRIO2 and Strategia) and therefore provided limited new information relevant to the issue of coherence. While interviewees recognized that the systems were not perfect, very few people suggested that major improvements were needed. The main issue raised pertained to the fact that TRIO2 did not capture any contribution that did not align with KPIs. One interviewee noted that qualitative contribution to initiatives in key priorities areas of the Feminist Foreign Policy could be captured and contributed to the trade commissioner reward system.

Conclusions

The Asia Pacific Branch (OGM) evaluation was the third in a series of 4 coherence evaluations. As part of the learning process, this third evaluation innovated by further developing the coherence measurement framework by including new indicators. This allowed for more targeted survey questions and facilitated a multivariate data analysis. Overall, the evaluation found that OGM is operating, for the most part, in a coordinated and coherent manner across diplomacy, trade and international assistance. While OGM has made progress in the recent years on this front, the evidence also suggests that opportunities for improvement exist to enhance the branch’s impact and efficiency. Principle findings include:

Recommendations

1. OGM should provide clear guidance to branch management and staff on how to identify and operationalize cross-stream initiatives.
2. OGM should establish mechanisms to ensure that staff in each OGM stream (trade, diplomacy, development) has better knowledge of each other’s streams.
3. OGM should establish incentives to further motivate staff to engage in cross-stream initiatives.

Appendix A – OGM background

OGM profile

Organizational structure

Asia-Pacific Branch organizational change

The Asia-Pacific Branch is composed of 3 bureaus: South Asia, Southeast Asia, and North Asia and Oceania. Beginning in 2018, the branch undertook change management activities that affected the organizational structure of some bureaus.

In the South Asia Bureau, 4 country divisions were transformed into 2 amalgamated country divisions. This change brought the point of integration between the 3 business lines to the director level, in the aim of facilitating the provision of integrated policy advice. In addition, the former corporate planning division was rebranded as Strategic Planning, Policy and Operations.

In the same year, the North Asia and Oceania Division created 2 distinct divisions for China: Greater China Trade and Investment, and Greater China Political and Coordination. Northeast Asia and Oceania includes all 3 business lines.

Southeast Asia has not undergone organizational change in recent years.

The Asia-Pacific Branch covers 40 countries with the support of 42 missions, and is organized into 3 bureaus: OAD, OSD and OPD.

*In addition to the listed missions, 10 Canadian trade offices in China fall under OPC: Chengdu, Hangzhou, Nanjing, Qingdao, Shenyang, Shenzhen, Tianjin, Wuhan, Xiamen, Xi’an.

Figure: Asia-Pacific sector table (OGM)

Appendix B – Notes on methodology

Annex I: Organizational factors

Measurement framework

The evaluation established a measurement framework, consisting of 5 organizational factors, that was used as component of the Evaluation of Diplomacy, Trade and International Assistance Coherence in the Asia-Pacific Branch, 2015-16 to 2020-21.

Figures:

Annex II: Scorecard results for the Asia-Pacific Branch

In parallel to previous coherence evaluations (WGM and NGM), scorecards were designed to assess each of 4 organizational factors, taking into account their respective indicators. Scores given to each organizational factor were determined based on document reviews, survey responses and key informant interview analysis.

Each scorecard includes the definition of each factor, along with the scale and definition of the 3 potential scores. Lastly, the score for each element is indicated in blue.

Note that 1) the factor corporate systems and tools was not scored, as it was found to neither enable nor impede coherence in the OGM branch; and 2) case-study findings were not aggregated into the final scores, as the factors mainly scored “fully enables coherence” when a mandated joint initiative was present—skewing OGM’s general results, where mandated joint initiatives are not always present.

Sample scorecard: Leadership

Appendix C – Case studies

Annex III: Canada-ASEAN FTA

Trade and development nexus

The advancement of a Canada-ASEAN FTA demonstrated coherence across trade, diplomacy and development. This case study demonstrated strong leadership and capacity and expertise, both of which allowed key stakeholders to navigate a complex organizational structure.

As a bloc, ASEAN ranks as Canada’s sixth-largest trading partner, and advancing a Canada-ASEAN FTA is a key priority for Global Affairs Canada. This priority is aligned with GAC’s strategy to strengthen and diversify Canada’s trade partnerships in the Asia-Pacific region, which the Ministry of Foreign Affairs is mandated to support. ASEAN member states have made the launching of negotiations towards a Canada-ASEAN FTA a priority economic deliverable for 2021.

The Canada-ASEAN FTA would not have advanced to the point where it is now had it not been for cross-stream and cross-branch collaboration and considerations. The advancement of a potential Canada-ASEAN FTA has been led by OGM’s trade stream and has involved the participation of the diplomacy and development streams. ASEAN is a complex partner that requires the consensus of all member countries to launch FTA negotiations and, therefore, bilateral relations and advocacy with the ASEAN Secretariat and individual ASEAN member states play a key role. Lastly, although development does not directly participate in FTA negotiations, it has played a key role in supporting the advancement of a possible Canada-ASEAN FTA by demonstrating Canada’s commitment to the region through development programs, in particular through the EDM.

Advancement of the Canada-ASEAN FTA – key achievements

• completed exploratory discussions on a possible Canada-ASEAN FTA in 2019
• completed a Canada-ASEAN Joint Feasibility Study in 2018
• organized and participated in 4 trade policy dialogue sessions on potential areas of divergence in an ASEAN-Canada FTA
• coordinated a regional FTA advocacy strategy
• established ASEAN as a priority region under the EDM in 2020, and completion of a country needs assessment plan (CNAP) for the ASEAN region, also in 2020, which aims to provide a view of the trade and investment context and outlines a series of proposed development programming areas; the CNAP identifies a grouping of potential technical assistance requests aligned with the objectives of the EDM project
• Canada co-hosted the annual ASEAN-Canada Senior Economic Official Meeting (SEOM) consultations, the most recent of which was on June 14, 2021
• as head of delegation, Canada’s Minister of International Trade participated in the annual ASEAN Economic Ministers’ Meeting (AEM), the most recent of which was held on August 28, 2021

Annex IV: The Rohingya response (Triple Nexus)

The Rohingya response demonstrated OGM’s ability to work collaboratively across a complex structure that included diplomacy, development, humanitarian assistance, and peace and security; the latter 3 are also known as the “Triple Nexus.”

Informed by the recommendations of Canada’s Special Envoy to Myanmar, Canada announced on May 23, 2018, a comprehensive strategy to respond to the Rohingya crisis in Myanmar and Bangladesh, committing $300 million of international assistance over 3 years to address development, humanitarian assistance, and peace and security (stabilization) needs.

The evaluation found that, overall, the Rohingya response demonstrated coherence across streams in terms of leadership and capacity & expertise, which helped to address some of the challenges found in a complex organizational structure. More importantly, the Rohingya response would not have been as successful had all 4 streams (diplomacy, development, humanitarian assistance and PSOPs) worked independently. In addition, Canada’s bilateral relationships with Myanmar and Bangladesh, and its advocacy initiatives, were seen to be central to delivering the Triple Nexus strategy, making the diplomacy stream essential to the Rohingya response.

The Rohingya response – key achievements

• Canadian leadership:Overall, the Rohingya response has provided a focal point for both diplomacy, development and humanitarian assistance in Bangladesh and diplomacy, development and PSOPs in Myanmar. With coordination, collaboration and considerations across streams, GAC demonstrated effective, strong, capable and global leadership in response to the Rohingya crisis through, notably:
  • the special envoy’s role with regards to collaboration with the UN and the international community, which reflected political weight and gravitas
  • the convening and coordinating role of the Canadian high commissioner and the head of cooperation in Dhaka, who sit on various committees with the international community
  • its advocacy in supporting continued calls for the UN Security Council to refer the situation to the International Criminal Court (ICC)
  • its support of The Gambia’s case against Myanmar at the International Court of Justice (ICJ) alleging violations of the Convention on the Prevention and Punishment of the Crime of Genocide against the Rohingya (the “Rohingya genocide case”).
• Goodwill between Canada and the Government of Bangladesh: Bangladesh has expressed appreciation for Canada’s strong advocacy on accountability, which itself was a result of collaboration between diplomacy, development and humanitarian streams. GAC demonstrated a commitment to supporting the Bangladeshi host community and continuing independent bilateral programming, and has worked with Bangladesh to allow for advancements in development and humanitarian assistance as part of the Rohingya response. The latter includes Bangladeshi policy updates that will allow for a longer-term approach to durable refugee solutions, for example, education.

The Rohingya response (cont’d)

Lessons learned

The factors of leadership and capacity & expertise played a significant role in facilitating coherence across streams for the purpose of the Rohingya response. First, leadership with respect to high-level political engagement established a clear vision of and objectives for a unified strategic approach and facilitated coherence by galvanizing cross-stream collaboration. Leadership at various management levels (special envoy, ministers, senior management, HOMs, director general, directors and deputy directors) at HQ and at mission (both Dhaka and Yangon) also played a constructive role in providing guidance on priorities. Second, the level of capacity and expertise, both in terms of previous experience and cross-stream exposure, contributed to efficiency and effectiveness by providing a foundation for communication and networks. In this case study, leadership and capacity & expertise superseded the organizational structure factor and helped to mitigate challenges experienced in the latter.

Coherence thinking was reflected at both the strategic planning and programming levels, but was limited to mandated priorities. Evidence suggests that collaboration is highly valued by all streams in the Rohingya response, while coherence thinking is valued by individual streams mostly when it directly impacts their own programming. This mandated joint initiative supported, incentivized and enabled a general anticipation and consideration of other streams’ work, in great part due to the special envoy’s strategic report. Outside of specific joint directives, coherence thinking was not found to be prioritized, leaving streams focused on their respective silos. Triple Nexus literature reflects coherence thinking, but the link to its operational application is not yet apparent insofar as successful programming for the Rohingya response was found to be guided by departmental mandates and policies (FIAP) and was aligned with the special envoy’s comprehensive report; programming was not formally guided by Triple Nexus theory. Limited capacity (time and resources) of staff working on the Rohingya response appears to be one factor limiting the potential extent of coherence thinking and its application.

The Crisis Pool Quick Release Mechanism marked a crucial element in facilitating coherence, without which Canada’s quick and purposeful Rohingya response would not have been possible, considering the development programming mode—especially in Bangladesh. This funding mechanism allowed for flexibility in the creation of new programming and initiatives across the development, humanitarian assistance and PSOP streams, as well as timely program delivery in association with multilateral organizations that could specifically address the crisis at hand. Although the purpose of the Crisis Pool is not explicitly to foster coherence, having access to the Crisis Pool and adequate resources helped coherence in terms of 1) Canada sustaining a leadership role in protracted crisis, and 2) permitting flexibility when dealing with unknowns that can occur in the context of a political crisis.

Overall, the Rohingya response would not have been as successful had all 4 streams (diplomacy, development, humanitarian assistance and PSOPs) worked independently of each other. This funding mechanism enabled cross-stream collaboration that allowed for a more effective and coherent strategic approach. Lastly, the evaluation found that FPDS used the Post Initiative Fund (PIF) as an advocacy mechanism to support awareness of the Triple Nexus’s objectives and initiatives.

The Rohingya response (cont’d)

Figure 20. Diplomacy and development play a central role in the Triple Nexus

Text version

[Graph illustrating the central role played by Diplomacy and Development in the Triple Nexus.]

Complex organizational structure and dual strategy

The evaluation found that organizational structure impacts approval, consultation and briefing processes. For the most part, it was found that OGM managed well the Rohingya response’s complex organizational structure—involving 2 ministers, 2 bureaus and 2 countries—owing to strong leadership and capacity & expertise. The response did, however, pose the following challenges: 1) two ministers (MINA and MINE) expected different briefing styles or products to be delivered for the same content, which led to duplication and impacted time management and effective use of resources; and 2) approval and consultation processes were based more on an ad hoc or common-sense basis. Although overall consultation processes were said to be inclusive over the past 3 years, 30% of key informant interviewees believe that, in 2021, there was an inadequate consultation process regarding more recent foundational policy documents and inadequate communication of strategic planning with regards to next steps. It is important to note that in the early years of the Rohingya response, Special Envoy Bob Rae straddled both bureaus and is said to have played a constructive role in coordination between the 2 bureaus and missions.

The evaluation found that in the Rohingya response, both diplomacy and development were central players in connecting the inter-relationships between other streams and were found to be critical pillars for enabling coherence (see Figure 20). Document reviews and key informant interviews found that diplomacy, in terms of advocacy and bilateral relationships, was particularly key in advancing strategic Triple Nexus programming and initiatives. In addition, diplomacy played the role of lead policy coordinator and implementer of the Rohingya strategy renewal. Development was also a central pillar in cross-stream collaboration: it was found to be in constant communication with humanitarian assistance (Bangladesh) to identify and coordinate programming to address immediate humanitarian needs, along with longer-term development needs, and to ensure there was no programming duplication; it also worked closely with diplomacy (at both missions) and PSOPs (Myanmar).

It is important to note that diplomacy is not recognized in Triple Nexus literature, which only refers to 3 sub-streams: development, humanitarian assistance and PSOPs. While PSOPs and humanitarian assistance play an important role in addressing immediate issues, they are not central to enabling coherence across streams. This is also reflected in their respective centralized models, in which both do not have staff at the relevant missions and are thus supported by diplomacy and development streams at the DHAKA and YNGON missions.

Annex V: The Mongolia extractive sector

The Mongolia extractive sector case study provides insight into the incremental learning that has happened at Global Affairs Canada with respect to coherence across streams, prior to and following the amalgamation of the Department of Foreign Affairs (DFAIT) and the Canadian International Development Agency (CIDA).

A new development program for Mongolia was created in 2011, centred around Canadian mining interests, which provided an opportunity for trade and development to work together in the period just before and after the amalgamation of DFAIT and CIDA. At this time, based upon an analysis of “Canadian national interest” and a formal request from the president of Mongolia, the Canadian prime minister directed that Canada should launch a mid-sized development program for Mongolia, based on 2 Canadian national interests: investment (mining) and democracy. Accordingly, Government ministers determined that the aid program should focus on improving governance with an emphasis on the extractive sector. Following these decisions, a substantial support program (i.e. $25M over 5 years) was put in place to strengthen extractive-sector governance, with a secondary thread of advancing democracy. Although the development program was announced in 2011, significant bilateral spending only began flowing in FY 2015/2016, notably through the mining governance projects MERIT, SESMIM and Mongolia Program Support Facility.

Overall, the evidence shows that potential coherence across streams was not achieved between trade and development as was originally intended. When compared to other more recent case study analyses, lessons learned from this case study demonstrate that the department has since learned how to increase coherence across streams.

Mongolia extractive sector - key achievements

• The development program (SESMIM and MERIT initiatives) was found to have improved governance, predictability and transparency in the management of mineral resources. In addition, development created a donor coordination group for the extractive sector that brought several donors and, occasionally, extractive companies together to share information and to plan. Trade would sometimes participate in those meetings
• The development program provided a solid basis for Canada, especially the Mongolia HOM, to capture and retain the attention of senior Mongolian officials and to maintain a diversity of relationships at national and sub-national levels
• Diplomacy and trade helped build relationships, opened doors and established Canadian standing that enhanced Canadian political advocacy and the investment environment. Diplomacy advocacy initiatives regarding transparency and good governance enhanced the investment environment for Canadian mining companies
• A foreign investment promotion and protection agreement (FIPA) was approved and a generally increased interest of Canadian mining investors interests.

Annex VI: Canada-ASEAN Scholarships and Educational Exchanges for Development (SEED)

The SEED program demonstrated coherence through leadership, expertise, and policy alignment.

On August 6, 2017, the new SEED program was announced to mark both ASEAN’s 50th anniversary and the 40th anniversary of ASEAN-Canada dialogue partner relations. The SEED program is a $10-million, 5-year (2018-2022) scholarship program aimed at providing post-secondary students and mid-career professionals from Southeast Asia with access to education in Canada, and is managed by GAC’s ASEAN Regional Development Program (OSF).

The evaluation found that the administration of SEED is an excellent example of cross-stream coordination and collaboration and that it advances development, trade and diplomacy objectives. SEED aims to reduce poverty in ASEAN developing countries and contributes to raising Canada’s profile as an international education destination by elevating Canada’s overall inbound scholarship portfolio. The program also demonstrates Canada’s commitment to the region, which supports trade’s broader efforts to launch negotiations for a Canada-ASEAN FTA. Interviewees noted that the SEED program has been well-received across the region, and is frequently referenced in ASEAN-related speaking points, especially at missions. Overall, interviewees stated, SEED contributes to strengthening Canada’s bilateral and regional relations with ASEAN member states.

SEED program - key achievements

SEED-related coherence impacts include increased operational efficiencies for OGM; consistent messaging across business lines; and a greater global recognition of Canada as a leader in international education.

• EduCanada brand: The EduCanada website has increasingly become a one-stop portal for all GoC scholarships. This has contributed to a more cohesive narrative surrounding Canada’s role as a leader in international education and has increased visibility of Canada’s investment on scholarships as part of a broader story (e.g. commitment to education). GAC continues to seek the support of provinces/territories who co-own the brand to further increase coherence in this area.
• Intradepartmental Scholarship Coherence Committee and Working Group: Through the DG Scholarship Coherence Committee meeting (DGSCC) and the Scholarship Coherence Committee and Working Group (SCWG), EduCanada has been proposed as the umbrella scholarship brand, pending feedback and approval from the Council of Ministers of Education, Canada (CMEC), which co-owns the brand. According to interviewees, the proposed approach would maintain the identity of individual scholarship programs, recognizing that they have disparate objectives, terms and conditions, beneficiaries and results metrics. The DGSCC and SCWG have also reduced duplication of efforts, improved external communications, and enhanced efficiencies through the sharing of best practices and coordination.
• Repatriation efforts for SEED recipients stranded in Canada due to COVID 19: As of October 31, 2020, a total of 22 SEED students were supported through OSF’s COVID-19 emergency fund. In addition to coordinating efforts to ensure coherent approaches across scholarship programs, OSF leveraged skills, expertise and networks of trade colleagues (BIE).

Appendix D – Collaboration within the department

Annex VII: Alignment of priorities and programs

Perception of priority and program alignment

Figure 21. A sizable number of employees from the trade and diplomacy streams feel that initiatives delivered by other branches are not aligned with OGM’s priorities

Text version

Figure 21:

Trade

Agree: 41%

Do not agree: 59%

Diplomacy

Agree: 48%

Do not agree: 52%

Development

Agree: 63%

Do not agree: 38%

While the evaluation did not conduct a thorough review of all the “touch points” between OGM and the activities and programming delivered by other branches in the Asia-Pacific region, some findings provide information on the level of coordination and collaboration in the region. Only 44% of OGM employees stated that the programming delivered in the region by branches other than OGM aligned with OGM priorities. There is therefore evidence that opportunities for improvement exist in terms of coordination of activities in the Asia-Pacific region.

Case study findings and interviews depicted a positive picture in terms of collaboration across the branches active in the Asia-Pacific region. Collaboration and regular coordination with IFM, TFM, BFM and MFM were reported. Examples of positive collaboration between OGM and other branches include:

• Trade and diplomacy: With regard to the advancement of the Canada-ASEAN FTA, the Southeast Asia Division (OSC) works closely with the Trade Policy and Negotiations Asia Division (TCA) and the Trade Negotiations Division (TCW), who play a central role in the T&D nexus on briefing and advocacy material. Interactions between the Trade Policy and Negotiations and Chief Trade Negotiator NAFTA branch and OGM also include collaboration on strategic documents for the Asia-Pacific region at large.
• Trade and development: Collaboration between the Trade branch, the development stream in the Global Issues and Development branch, the regional Southeast Asia desk and the ASEAN mission in OGM takes place in order to coordinate and support the advancement of a potential Canada-ASEAN FTA. In this case, development plays an important role in consulting with the ASEAN mission as part of the country needs assessment plan (CNAP) for the EDM. Additionally, this nexus area was also reflected in the SEED program case study, where OGM collaborated with BIE for program administration and operations.
• Triple Nexus: Diplomacy, development, humanitarian assistance, and peace & security coordinate and collaborate well on the Rohingya response. This includes both OGM bureaus at HQ (South Asia Bureau ([OSD] and Southeast Asia Bureau [OAD]), both missions (Embassy of Canada to Myanmar [YNGON] and High Commission of Canada in Bangladesh [DHAKA]), the PSOPs Programs Division (IRG), the Conflict Prevention, Stabilization & Peacebuilding Division (IRZ) and the International Humanitarian Assistance - Americas and Asia and Natural Disaster Response Division (MHA).
• Diplomacy, peace & security: A few interviewees noted a great level of collaboration with the IFM branch when it comes to strategic planning and cooperation for programs and initiatives focused on the Asia-Pacific region, as well as for foundational policy or strategic documents. Interviewees from both OGM and IFM agreed that the collaboration between the branches is positive, which has in part enabled Canada to develop a strong partnership with Asia-Pacific countries to combat chemical, biological, radiological and nuclear (CBRN) proliferation.

Appendix E – Lessons learned from coherence evaluations

Annex VIII: Lessons from coherence evaluations

What do we know about coherence so far?

Findings from the 3 coherence evaluations conducted over the last few years were reviewed to identify recurring themes and key differences. The following observations depict the key lessons learned from this review:

• Coherence leads to enhanced impacts and efficiencies that could not be achieved by working in silos
• While Senior Management demonstrated that coherence is a priority, a significant proportion of staff is not aware of this
• Coherence thinking, which refers to the ability of all OGM staff to identify opportunities for collaboration in nexus areas, needs to be further developed in all branches
• Development has the ability and willingness to work with other streams
• Diplomacy was a natural partner for most streams when collaboration was expected or desired
• Incentives that encourage staff to collaborate with other streams are currently lacking in all 3 branches evaluated so far
• There are currently no dedicated funds to support cross-stream initiatives. Availability of resources can sometimes be a significant barrier to collaboration
• While integrated annual planning exercises involving the 3 main streams are a best practice, this is currently not a widespread practice across the branches
• From an organizational structure perspective, the integration of the 3 main streams at the director level increases the likelihood of coordination and coherence
• The lack of regional strategies can hinder coherence
• Many joint initiatives are taking place, but the best practices and lessons learned are not converted into explicit knowledge that can be easily shared across the organization
• The coordination of Canada’s external messaging in all three branches (i.e., WGM, NGM, OGM) could be improved