Foreign Affairs, Trade and Development Canada
Symbol of the Government of Canada

Foreign Affairs, Trade and Development Canada

international.gc.ca

EVALUATION OF THE POLITICAL ECONOMIC REPORTING AND PUBLIC AFFAIRS RENEWAL INITIATIVE

(May 2010)

(PDF Version, 528 KB) *


ABBREVIATIONS, ACRONYMS AND SYMBOLS

ADMAssistant Deputy Minister
ASAdministrative Staffing Classification
CBSCanada Based Staff
CPEOChief Political Economic Officer
DECDepartmental Evaluation Committee, DFAIT
DFAITDepartment of Foreign Affairs and International Trade Canada
DMDeputy Minister
EACEvaluation Advisory Committee
EBPEmployee Benefits Package
EXExecutive Staffing Classification
FSForeign Service Staffing Classification
FSDPForeign Service Development Program
FTEFull Time Equivalent Employee
GLBStrategy and Services Division, DFAIT
HOMHead of Mission
HQDFAIT Head Quarters (125 Sussex Drive and 111 Sussex Drive, Ottawa)
LESLocally Engaged Staff
MCOManagement Consular Officers
NGMNorth America Branch, DFAIT
NGONon-Governmental Organization
OGDOther Government Department
PERPolitical/Economic Relations
PERPAPolitical Economic Reporting and Public Affairs
PERSUPolitical Economic Renewal Support Unit
TBSTreasury Board Secretariat
ZIDOffice of the Inspector General, DFAIT
ZIEEvaluation Division, DFAIT
ZIVInspection Division, DFAIT

 


EXECUTIVE SUMMARY

Introduction

The Political Economic Renewal Initiative was designed to brand the Department's Political Economic Reporting and Public Affairs (PERPA) program as one designated to provide seven core or essential services. The Renewal is aligned with the transformation agenda of the Department. Its strategic aim is to renew PERPA's model of service delivery which contributes to an international agenda shaped to Canada's benefit and advantage in accordance with Canadian interests and values. PERPA Renewal is meant to influence the PERPA program, which by definition is cross-cutting or horizontal in nature, and positively impacts on achieving the Department's key priorities on Afghanistan, United States and the Hemisphere and the pursuit of greater economic opportunities for Canada.

The goal of the PERPA Renewal evaluation was to provide DFAIT's Departmental Evaluation Committee with a neutral and an evidence-based assessment of relevance and performance of the initiative, and progress made to date.

The specific objectives of the evaluation are as follows:

  1. To evaluate the relevance of PERPA Renewal Initiative by assessing the extent to which it addresses the needs of the PERPA community, and is aligned with federal government priorities and with DFAIT's strategic outcomes;
  2. To evaluate the performance of the Initiative in achieving its objectives efficiently and economically with its allocated resources.

Key Findings

The Renewal was a positive start in advancing the relevance of PERPA activities to the priorities of the Department, and to the needs of the PERPA community. However, the evaluation found that gaps remain. Missions in non-priority countries and regions seek more clarity in their work in an effort for their reporting to be made more relevant. Also, the Department needs to identify a vision to guide the activities of PERPA.

PERPA Renewal has been successful in providing direction and guidance to Missions, especially regarding visits management. Although staff at missions abroad are able to streamline visits management by referring some aspects to alternative service providers, the future role of PERPA political economic reporting and other core services such as advocacy and policy development, needs to be determined and communicated. The PERPA label of "General Relations" seems to continue to undermine the focus and effectiveness of PERPA.

The Renewal also has been effective in communicating PERPA's core business, and in defining standards to manage clients' expectations for services. There has been considerable success in provision of practical tools and guidance that help PERPA staff in their daily work. Clarifying roles and responsibilities of PERPA is a significant step forward. However, the Renewal has not resulted in better management of priorities at Missions. There continues to be an inability to link PERPA activities to DFAIT's priorities.

In addition, there is a need for the Department to have a dialogue with the DMs and ADMs on the role of the Chief Political Economic Officer, and the leadership of the DMs and ADMs in supporting the Renewal within the Department and with external partners and clients. There was an expressed desire for opportunities to hear from the senior management about their vision of PERPA, how the function contributes to the mandate of DFAIT, and how future challenges may be met.

In order to assess PERPA renewal's on-going progress, there is a critical need to develop performance measurement and reporting tools to demonstrate the contribution of PERPA activities to the mandate and priorities of the Department and the Government of Canada. Results-based PERPA performance information is essential for better priorities setting and their effective management.

The evaluation concluded that the full potential of PERPA Renewal has not been realized because of its limited profile, attention, and resources. Resourcing of the PERPA Renewal has been ad hoc for the Initiative to have had a transformational impact. The community believes that the collective leadership of the Renewal with senior management should be maximized to reach its potential for change.

Recommendations

Recommendation 1:It is recommended that the vision and focus of PERPA be better defined and communicated in the context of DFAIT's Transformation Agenda and of the Department becoming a modern foreign ministry.

 

The evaluation of the Renewal revealed a need, in the context of the Transformation Agenda and modernizing of the Department, to better define and communicate to staff the vision of PERPA to guide the community. The vision statement will be important particularly to missions in non-priority countries in contributing to the mandate and priorities of the department.

The work of PERPA staff would be better focused with more clear articulation of priorities and strategic objectives. Corporate efforts currently underway to improve business processes at missions should also help in operational effectiveness of the core services.

Recommendation 2:It is recommended that HQ, in consultation with Missions, define a framework to facilitate more effective use of PERPA core services for achieving the priorities of the department and of the Government of Canada.

 

One of the outcomes of the Renewal that needs more attention is improved PERPA priorities setting and management. Adoption and consistent implementation of a results-based framework would help in improving the relevance of PERPA activities, the achievement of priorities, and in demonstrating the contribution of the PERPA function.

Recommendation 3:It is recommended that PERPA leadership from the Chief Political Economic Officer, the DM and ADM levels be enhanced to sustain the Renewal process.

 

Most interviewees at missions abroad expressed a need for opportunities to have face-to-face dialogue with the CPEO, the DM and ADMs on issues of interest to the PERPA community. Given that PERPA outcomes are by nature less concrete than for example those of Trade, a dialogue with the senior management is important in discussing the opportunities and the challenges facing PERPA staff.

Recommendation 4:It is recommended that the resources allocated to the PERPA Renewal Initiative be commensurate with the Initiative's stated objectives and expected outcomes.

 

As discussed earlier in this report, the PERPA Renewal was undertaken with minimal and ad hoc resources which to some extent slowed down its progress. The resourcing issue also includes senior management's time in providing leadership to the Renewal process. There may also be possibilities to creatively use existing Trade Renewal resources and infrastructure to assist with the implementation of the PERPA Renewal.

Recommendation 5:It is recommended that a cost-effective performance measurement and reporting system be developed to manage and credibly demonstrate the contribution of PERPA.

 

Measuring and reporting on the contribution of PERPA and the PERPA Renewal continues to be a challenge, perhaps more so than other program areas at the Department. Effective performance measurement and reporting of PERPA results are however critical the success of the function both in terms of better meeting the needs of the clients, and for accountabilities purposes.


1.0 INTRODUCTION

The Evaluation Division (ZIE) at Foreign Affairs and International Trade Canada (DFAIT) is housed within the Office of the Inspector General (ZID) and is mandated by Treasury Board Secretariat (TBS) through its new Evaluation Policy (effective 1 April, 2009) to conduct evaluations of all Direct Program Spending of the Department (including Grants & Contributions).

The Evaluation Division tables its reports quarterly to the Departmental Evaluation Committee (DEC) which is chaired by the Deputy Ministers of the Department. The evaluation of the PERPA Renewal Initiative was conducted in accordance to the Department's five-year evaluation plan. It is supported by the Department's Chief Political Economic Officer and the Political Economic Renewal Support Secretariat. The target audience for this evaluation is primarily the Department as it pertains to its effort to support Canada's key political and economic representatives such as the Prime Minister of Canada and its Ministers. The results of the evaluation will also benefit Canadians including private industry and public sector as clients of the Department.

1.1 Background and Context

Over the past two decades, the Department has been proactive in its study of the effect of globalization on international affairs; more specifically, on how this trend has affected the delivery of its political economic program and emphasized the need to articulate its direction (Hunter and Halstead, 1991; Office of the Inspector General, 1996 and 2000, Duxbury and Higgins, 2004). The objectives of those studies were to describe and define the Department's political and economic activities as a basis for performance measurement, to identify the strengths and weaknesses of its culture and, to address management issues that affect programming at the Department.

The studies' recommendations were instrumental to formally establish the need for the PERPA Renewal Initiative in 2005 and its operational arm, the Secretariat, in 2008. The purpose of the renewal was to transform the PERPA program by identifying partners as well as defining and focusing on the delivery of seven core, or essential, services: Information and Analysis, Policy Integration, Policy Development, Advocacy, Access, Program Delivery and, Visit guidance. Key in this transformation was the definition of the central, or core, role of the Foreign Service Officer.

A second reason for this transformation was to provide a centralized and standard structure to determine the legitimacy of service requests to clients for whom the Department has a professional obligation to provide services as opposed to those contacts for whom the Department can provide services based on an expectation of reciprocity (e3 Committee on Political/Economic Relations (PER) Clients and Services, 2002; Peck, 2003). This delineation of service delivery was considered necessary to align the work of the Foreign Service on core services that advance Canada's interests, support the priorities and principles of Government and the Department as well as enhance their personal sense of value as designated within the Foreign Service. The ability to streamline service delivery according to the principle of either "serve or refer" was considered to be fundamental for the Foreign Service officer in the face of often competing and contrasting challenges.

1.2 Program Objectives

There were two main drivers for this Initiative. One driver was external. It was to meet the demand for political-economic renewal to provide a modernized foreign service to deal with the present-day challenges of foreign policy. The second driver which was internal, was to balance the Department's efforts to streamline and renew service delivery on political-economic renewal to that of the trade commissioner service at the Department.

The Political Economic Renewal Support Unit (PERSU), established in 2008 is part of the Secretariat which implements PERPA Renewal. The aim of the Secretariat is to drive the implementation of PERPA Renewal's mandate under the following objectives:

  • to interpret the newly defined seven core services of the Political Economic Relations and Public Affairs Program (PERPA);
  • to assist PERPA officers in reviewing difficult requests from clients;
  • to recommend appropriate responses to ensure consistent service delivery world-wide;
  • to achieve economies of scale and promote best practices within the PERPA community; and,
  • to support the newly-created position of Chief Political Economic Officer (CPEO).

1.3 Governance

The Political Economic Renewal Secretariat reports directly to an Assistant Deputy Minister who fulfills the role of Chief Political Economic Officer (CPEO). At the Department, the governance structure for PERPA Renewal is currently anchored by a Steering Committee and an Advisory Committee with support that was provided from a Communications Ginger group during the pre-implementation stage.

Membership of the Steering Committee is considered representative. It includes senior managers from functional, geographic and personnel Departmental branches, as well as representatives from small, medium-sized and large missions. It is chaired by the CPEO. Its mandate is to provide overall strategic direction to PE renewal, discuss specific initiatives and suggest implementation approaches, periodically assess progress on renewal, advise on communications plans, assist in aligning with other initiatives, and assist in disseminating key messages.

Membership of the Advisory Committee is also considered representative and includes directors and other managers from functional and geographic branches at the Department. It is chaired by a director-level member of the committee who is appointed by the CPEO. Its mandate is to render the provision of our core services more effective and hence more manageable through the implementation of targeted projects that aim to establish standard operating procedures as well as provide tools and resources to facilitate the implementation of political, economic and public diplomacy responsibilities.

The Ginger Communications Group was primarily active in the several months prior to the implementation of PERPA Renewal when communications efforts were critical for a successful launch of the initiative. This group was chaired by the Director General for Communications and was represented by officers and managers from the Communications Bureau as well as PERPA representatives from the Department.

The PERPA Renewal Secretariat has two main activities. They are to educate the PERPA community on how best to serve its clientele as well as to act as the central liaison between Headquarters and the Missions on the updated model of service delivery for PERPA. Please refer to the Logic Model for more detail (see Annex A).

1.3.1 Partners

Although PERPA Renewal pre-dates the Department's formal launch and approach of its Transformation Agenda, its mandate and objectives are integral with transformation. PERPA Renewal is closely tied to the work and clients of the PERPA community. The Secretariat does not operate on formal partnerships with its partners/clients established through such mechanisms as Memorandums of Understanding. It works indirectly with external PERPA clients, while directly with its main partners such as internal divisions at the Department, PERPA colleagues involved through consultation structures, Steering committees, Advisory committees and PERPA renewal envoys. The Secretariat also has partnered with the Canadian Foreign Service Institute for learning and training as well as the IT Support Division for the development and dissemination of on-line training. The Secretariat works with ZIV (Inspection division) to review PERPA during its inspections of Missions.

1.3.2 Clients

PERPA Renewal's principal client is the Department but, it also supports PERPA program officers and managers (CBS and LES), other Government Departments (OGDs), managers (for implementation) and non-governmental organization (NGOs) or associations. Given the nature of the work, invariably the decisions taken for service delivery will affect all who come to rely on or contact PERPA.

1.4 Program Resources

PERPA Renewal was a Departmental initiative funded solely through departmental allocations. Initial funding of $336K, in 2005-06, was obtained through the Departmental Contingency Reserve, with supplemental funding provided through the North America Branch (NGM). This branch cash-managed the initiative until September 2008. At that time, the Department created a reference level for PERPA Renewal with an annual budget of $557K up to 2010-11. The budget included four FTEs (with EBP) at an annual cost of $460K and an annual operating budget of $97K for 2008-09, 2009-10 and 2010-11. This initiative was not subject to a Treasury Board Submission.

The Secretariat has operated with four FTEs since its inception, with an average annual operating budget of $97K. Operating funds have been expended mainly towards IT support for on-line training, communication publications, translation and some modest travel to communicate PERPA renewal objectives since implementation at selected missions.

The staffing profile (FS-04, FS-03, FS-02 and AS-02) of the Secretariat has remained relatively unchanged with few exceptions. In 2008-09, the AS position was reclassified from AS-03 to AS-02 to reflect the duties needed for the Secretariat and one director-level (EX-01) was included mid-term in 2009-10 (overfilling the FS04 position).


2.0 EVALUATION SCOPE & OBJECTIVES

The goal of the PERPA Renewal evaluation is to provide DFAIT's Departmental Evaluation Committee with a neutral and an evidence-based assessment of relevance and performance of the initiative, and progress made to date.

The specific objectives of the evaluation are as follows:

  1. To evaluate the relevance of PERPA Renewal Initiative by assessing the extent to which it addresses the needs of the PERPA community, and is aligned with federal government priorities and with DFAIT's strategic outcomes;
  2. To evaluate the performance of the Initiative in achieving its objectives efficiently and economically with its allocated resources.

3.0 EVALUATION COMPLEXITY & STRATEGIC LINKAGES

The Political Economic Renewal Initiative was designed to brand the Department's PERPA program as one designated to provide seven core or essential services. It relates to the transformation agenda of the Department and, its strategic goal is a renewed PERPA model of service delivery, contributing to an international agenda shaped to Canada's benefit and advantage in accordance with Canadian interests and values. PERPA RENEWAL is meant to influence the PERPA program, which by definition is cross-cutting or horizontal in nature and positively impacts on achieving the Department's key priorities on Afghanistan, United States and the Hemisphere and the pursuit of greater economic opportunities for Canada.


4.0 KEY CONSIDERATIONS

There were two key considerations in conducting the evaluation: a) availability of reliable performance information on the activities, outputs and results of the Renewal, and b) tight timelines within which the evaluation needed to be completed in order to be timely.

As evaluations are now used as primary evidence on the relevance and performance of programs and initiatives in the Strategic Review process, the PERPA Renewal Evaluation needs to provide evidence-based findings on the continued relevance, and achievement of immediate and intermediate outcomes. The attribution of outcome results to PERPA activities depended on the availability of quality performance information. The absence of such information presented a challenge to the work of evaluators.

The evaluation was completed within considerably tight timelines. A high number (~100) of interviews were scheduled and carried out via telephone and in-person, as per the evaluation workplan.


5.0 EVALUATION APPROACH & METHODOLOGY

The evaluation of the impact of PERPA Renewal on the delivery of the PERPA program was timely to understand:

  • The needs of the Mission and how those needs are affected by the Mission's geographic location, size and scope of PERPA services it delivers;
  • The strengths and weaknesses of the PERPA Renewal;
  • The role of the Foreign Service Officer in PERPA Renewal and the effect of the renewal effort on their personal sense of value-added service delivery;
  • The success of PERPA Renewal in providing services to the Department's clients and partners; and,
  • The effect of the principle of "serve or refer" in the delivery of PERPA services.

5.1 Evaluation Design

The evaluation was designed to collect information from multiple lines of evidence to assess the relevance and performance of PERPA Renewal. These lines of evidence included documents(1) and interviews with PERPA representatives at Missions and at the HQ. The reference period for data collection was November 2009 to March 2010.

5.2 Sampling

Missions for inclusion in the evaluation were selected based on two main criteria. The criteria was size (as defined by the FTE-base to be considered small, medium or large missions) and work composition (bi-lateral, multi-lateral). EAC members provided suggestions on which missions could also be considered as appropriate candidates. For instance, the selection took into account missions that had been included in the January 2008 pilot study on PERPA Renewal(2) and were subject of in-person site visits to gain hands-on training on the Renewal. A total of 16 Missions were selected.

5.3 Lines of Evidence

Document Review

PERPA Renewal documents from the Department's Steering, Advisory and Ginger Communication Group Committees were reviewed for decisions taken on the direction of the initiative. PERPA activity reports were reviewed to note the nature and extent of planning and performance monitoring which include the key milestones achieved and planned activities. Planning and activity reports for the Arctic Foreign Policy Centre were also reviewed as this Centre operates on dedicated PERPA resources with a policy-program focus.

Key Informant Interviews

One hundred (100) key informant interviews were held with Missions, at HQ and with other representatives of other Foreign Ministries. A total of 78 interviews were held with Missions, 15 interviews were held at the HQ and with PERPA stakeholders and, 7 interviews with representatives from other Foreign Ministries.

The interviews included PERPA and Trade representatives at the Department regarding their experiences with the renewed trade model on streamlining services along with OGDs, NGOs and other relevant associations. Interviews with key interlocutors were held in-person or by telephone. An unstructured open-ended interview questionnaire was developed to solicit information on activities and management of evaluation issues.

Field Visits

Field visits took place in four (4) missions that represented various experiences related to the Renewal, level of implementation and or uptake, and geographic representation. Field visits were determined according to the volume of PE visits at missions, the PERPA work composition and their availability for a field visit.

The 17 missions that participated in January 2008 PERPA Renewal pilot were included in the population from which the sample for field visits was selected. Field visits took place in Budapest, Vienna, Washington DC, and Oslo.

5.4 Data Analysis

The evaluation collected interview data which was synthesized and triangulated where possible with document data. The results of which led to the identification of themes or patterns. These patterns were contextualized in accordance to the evaluation issues from which the findings were delineated.


6.0 LIMITATIONS TO METHODOLOGY

The evaluation's primary data source was interviews, and as a result, this type of opinion-based data varied by respondent and subject to individual bias which reduces its ability to be generalized to reflect the views of the PERPA community. There was also little performance information available on PERPA or PERPA Renewal thereby limiting the ability to analyze PERPA Renewal efforts objectively because of no set standard for comparison. Given the variation in political economic programming across Missions, it was also difficult to conduct comparative analyses or benchmarking exercises.


7.0 EVALUATION FINDINGS

In evaluating the PERPA Renewal, the evaluation, in accordance with Treasury Board Policy on Evaluation, examined a number of issues regarding the relevance and the performance of the Initiative.

RELEVANCE

7.1 Relevance Issue 1: Continued Need for the PERPA Renewal Initiative

Finding # 1:Overall, PERPA Renewal is relevant to the need for focusing the work of PERPA, and setting limits to services offered. However, the relevance of PERPA Renewal at Missions varies depending on their size, geographical location and service delivery.

 

The PERPA Renewal was fuelled by the overall need to modernize foreign service to meet the present-day challenges of foreign policy. The renewal was to transform the PERPA program by identifying partners as well as defining and focusing the delivery of seven core or essential services: Information and Analysis, Policy Integration, Policy Development, Advocacy, Access, Program Delivery and, Visit Guidance. Key in this transformation was the definition of the central, or core, role of the Foreign Service Officer. The Renewal was also driven by the need to balance departmental efforts on streamlining and renewing service delivery on political-economic renewal to that of the trade commissioner service at the Department.

Overall, the evaluation found that streamlining of service delivery according to the principle of either "serve or refer" has been and continues to be relevant to the needs of PERPA function at missions abroad. Most interviewees indicated that definition of the seven core services has provided clarity in terms of what service may by expected from PERPA officers. This clarification is essential to better delineation and management of priorities at missions abroad.

One of the tools the Renewal provided for streamlining PERPA services was visit management services standards. These standards are seen a valuable tool and used by most missions. However some HOMs practice discretion in exceeding standards in providing services such as airport pick-up of officials. These missions believe that a degree of flexibility in application of the standards is necessary given that missions operate in significantly different contexts and hence have different needs. For example, smaller mission in non-priority areas may not receive a high volume of visits by senior officials. When such visits do occur, the HOM may wish to provide airport pick up and use the occasion to discuss issues of strategic importance to the Department and the Government.

It is important to note that while PERPA Renewal has been relevant to the needs of most missions; its impact has been limited on multilateral missions. Interviewees at these missions pointed out that their work focuses to a large extent on advocacy activities - seizing opportunities for Canada in a multilateral forum and environment. At multilateral missions, the demands of providing visit guidance service are less imposing than those of bilateral PERPA staff. Some multilateral officials partner with bilateral colleagues when assisting with major Canadian visits.

Lastly, the evaluation received mixed feedback regarding the need to narrow the definition of PERPA's service delivery focus. Although not a dominant view, some interviewees, at bigger missions such as Washington, believe that they have had sufficient clarity on the parameters of the services they are to provide, and on the expectations of the clients. The Renewal Initiative seemed to have had a lesser impact on these missions.

Finding # 2:Although the Renewal has been effective in providing PERPA officers and managers the necessary tools to manage expectations efficiently regarding visits, DFAIT stakeholders seem to be unclear as to the level of service expected from PERPA at Missions as a result of the Renewal. There remains a need for the community to consider the implications of inconsistent service delivery in visits management.

 

The evaluation found that the use of discretion to exceed service standard has resulted in inconsistent service delivery to stakeholders across missions. For example, senior officials visiting multiple missions may be referred to alternate service provider by one mission, while at another mission they would receive the service from the mission staff. This leads to confusion and a sense of dissatisfaction on the part of clients and stakeholders.

Another factor that seems to have contributed to this lack of clarity regarding service levels was that not all OGD and external partners are informed of the new service delivery standards, and the core services. The OGDs' lack of awareness has raised questions about expectations. The interviews revealed a need to better communicate the goals and implications of PERPA Renewal with Government and external partners.

7.2 Relevance Issue 2: Alignment with Government Priorities

To investigate the alignment of the Initiative with government priorities, the following Evaluation Question was posed: To what extent is the mandate of PERPA Renewal aligned with Federal government priorities and with DFAIT's strategic outcomes?

Finding # 3:PERPA Renewal's definition of seven core services has helped align the work of PERPA with Federal government priorities and DFAIT strategic priorities, but more work is needed to align PERPA with the Government of Canada's policy agenda.

 

The evaluation found that overall, the Renewal's definition of seven core services has resulted in better alignment of the PERPA with Federal Government's priorities. In fact, most interviewees noted that the Renewal has significantly contributed to clarifying of roles and responsibilities of PERPA staff, and there has been considerable progress in this area. The improved understanding of PERPA roles and responsibilities has also helped with defining day-to-day operations of PERPA and service delivery to clients. PERPA activities seem to be more focused as a result of the Renewal Initiative.

One of the goals of the Renewal was to improve priority setting by PERPA staff. During the course of the interviews, there was general agreement that priority setting remains a challenge in the context of PERPA work. This is particularly so in the case of countries where current Government priorities may not be prevailing issues. For PERPA work to become more strategic, there needs to be more guidance and direction, enabling missions to better link their activities to the priorities and objectives of the Department.

The evaluation found that mission reporting agreements for political-economic reporting to HQ have provided a valuable framework for strategic definition of issues and work planning. The agreements are broad enough to allow missions the needed flexibility to respond to issues and operational priorities as they arise. However not all missions seem to have a reporting agreement to guide their work.

One example of the effective alignment of PERPA is the thematic policy integration effort at the Canadian International Centre for the Arctic Region (CICAR). The CICAR staff work in collaboration with missions, HQ & partner federal departments and agencies to advance the international dimension of the Northern Strategy through outreach activities with such organisations as the EU Commission.

On the whole, most interviewees are of the view that the Renewal remains a work in progress, and further effort is needed in aligning of PERPA with the Government's policy agenda. Sustaining the momentum of PERPA Renewal is even more crucial at this time given the New Business Model where the role and responsibilities of the HQ Geogroup vis-à-vis missions are transformed.

7.3 Relevance Issue 3: Consistency with Federal Roles & Responsibilities

To investigate the consistency with federal roles and responsibilities, the following Evaluation Question was posed: In delivering PERPA services and carrying out the activities, is the Renewal Initiative fulfilling its appropriate role?

Finding # 4:In the context of the Transformation Agenda and the New Business Model, it is now an opportune time to discuss and determine the optimal balance of PERPA core services in order to maximize their relevance, and better allocate resources.

 

As noted above, the Renewal and the definition of seven core services have made significant contribution to a better understanding of PERPA staff's roles and responsibilities. There is now a need for redefining the mix and the balance of the core services in order to maximize the relevance of the function to the needs of internal and external clients. Departmental factors such as fiscal pressures and DFAIT's Transformation Agenda have created an opportune environment for considering strategic changes to the core services to enable PERPA to contribute more effectively to Government priorities.

The interviews revealed a range of views within the department on how the core services may be redefined. For example, several interviewees expressed the view that political reporting is now less relevant given that there is ready access to information through the media including the Internet. Others believe that reporting continues to be of strategic value, one that provides a unique Canadian angle on a given issue or event, and its implications for Canada. Another example is advocacy, and the potential for more emphasis on it as an activity. The evaluation found that as visit guidance services are streamlined, more resources could be reallocated to advocacy work in advancing Canada's interest.

In the context of Transformation and the New Business Model, it important to consider the role of the Locally Engaged Staff (LES) in PERPA Renewal. Both locally-engaged and Canada-based officers are part of the team that delivers the PERPA program at missions abroad. One of the goals of the Transformation Agenda is for LES to play an increasing role, including in the renewal and delivery of PERPA.

PERFORMANCE

The evaluation addressed four (4) key questions. They were as follows:

  1. Does the PERPA Renewal Initiative, its design, processes and activities contribute to the achievement of immediate and intermediate outcomes?
  2. Are the most appropriate and effective means being used to achieve PERPA Renewal's outcomes? Is PERPA Renewal able to achieve its outcomes under its current complement?
  3. How is PERPA Renewal governed? And, is this governance model effective?
  4. How does DFAIT manage and monitor progress regarding PERPA Renewal?

7.4 Performance Issue 1: Achievement of Expected Outcomes

Finding # 5:PERPA Renewal has achieved considerable success in producing its key outputs with limited resources.

 

The evaluation found that PERPA Renewal, with a relatively limited annual operating budget of $97K for 2008-09, 2009-10 and 2010-11, has managed successfully to achieve several key outputs over this same time frame. Operational funds were expended on IT support for on-line training, communication publications, translation and some modest travel to communicate PERPA Renewal objectives since implementation at selected missions. These in turn contributed to establishing the seven (7) core services, the dissemination of the on-line training course for PERPA Renewal; face-to face training for FSDPs; and communications to heightened awareness of PERPA's core services (e.g., calendar, easy guide to service delivery accessible to all employees).

The staffing profile (FS-04, FS-03, FS-02 and AS-02) of the Secretariat has remained relatively unchanged with few exceptions over the same time frame. In 2008-09, the AS position was reclassified from AS-03 to AS-02 to reflect the duties needed for the Secretariat and one director-level (EX-01) was included mid-term in 2009-10 (overfilling the FS04 position). Essentially, the PERPA Renewal foot-print was established by relatively few staff at GEMX.

Finding # 6:PERPA Renewal has made progress toward its immediate outcomes including heightened awareness and knowledge of PERPA core services, some increased alignment of PERPA resources at Missions, increased implementation of PERPA core services guidelines and timely review of PERPA requests by the Secretariat.

 

Evaluation interviews at missions found that many PERPA officers were aware of the Renewal, their roles and responsibilities in terms of the services provided by PERPA, and the delineation of the seven core services helped PERPA officers in describing their work. However, there was less of a consensus over how the seven core services linked to the core business plans of the PERPA program and Departmental objectives. For example, some PERPA officers said that their reporting on country specific issues was not considered relevant when linked to the Departmental Strategic interests. Therefore, this suggested an under-reporting of PERPA-related activity. It was also suggested that the Department may wish to include a general reporting category which would capture issue-reporting on areas not directly linked to the macro-level strategic priorities but of importance at a micro-level, or ones that could be viewed as potentially emerging issues.

Very few PERPA officers interviewed consulted with the Secretariat. Many said there wasn't a need to contact the group for guidance and those who did consult, found them to be helpful and timely in their responses. When the Secretariat was consulted, it was for discussion on the nature and extent of the level of involvement of for example, senior PERPA officers (FS-03s) for planning non-Ministerial or senior level delegation visits. However, where the Secretariat and CPEO were active was in discussions on the role of Canadian diplomatic presence at meetings during Ministerial and Provincial Ministerial delegation visits.

Finding # 7:The Renewal has had mixed success in reaching intermediate outcomes but it has clarified the roles and responsibilities of PERPA and streamlined and enhanced PERPA service delivery on the basis of the "serve or refer" principle.

 

The evaluation found that PERPA Renewal would benefit from alignment with the Trade Commissioners Service Renewal to create synergies, to leverage existing resources and to reduce duplication of effort. In addition, this alignment would create more effective management of Departmental and Government of Canada priorities. Although PERPA Renewal has already adopted the principle of "serve or refer" from the Trade Service Commissioners for those activities where an alternate service provider can be called upon to deliver specific non-core activities, the evaluation did not find substantial evidence to support the hypothesis that the PERPA officer now had more time to be active on core or essential PERPA activities. In other words, the evaluation did not find strong evidence for a significant off-setting or time shift of PERPA officers to enable more time for issues-reporting, advocacy or policy-analyses. Nonetheless, there is evidence to suggest that those PERPA officers interviewed do now reflect upon the services they offer to identify which ones merit serve as opposed to referring them to an alternate service provider.

Finding # 8:There is a need to more efficiently use PERPA resources to maximize influences within the internal PERPA community.

 

The Secretariat's efforts on training, provision of education, and awareness tools (e.g., a directory of "Who is Who" in GeoConcerto) were seen as helpful and practical, but many interviewed believed that they were not used widely enough by the PERPA community. A relatively small percentage of the PERPA community at Missions and at HQ had taken the on-line course (504 employees registered; of which 50% took the course). This suggested that the on-line course to have limited impact, given the actual number of PERPA officers worldwide. Of those who had completed the on-line course since its inception, most were rotational PERPA FS stream (62%), 29% LES and the remaining 9% were non-rotational. The majority of course participants found the course easy to follow but overly targeted to issues on visits management.

Some interviewees believed that PERPA needs to be merged with a stronger platform such as Transformation which includes a mandate for organizational change, aligning missions and accountability. Although an often-stated belief was of the difficulty in measuring PERPA, many interviewed were of the opinion that PERPA needs to be quantified (e.g., # of contacts, amount of time spent by activity, overtime amount, influence at meetings, client satisfaction) given that the one of the goals of PERPA is to ensure efficient use of taxpayer funds.

7.5 Performance Issue 2: Demonstration of Efficiency & Economy

Finding # 9:Using modest resources, the Renewal Initiative has made significant progress in achieving its outputs.

 

Several Missions noted that the practicality of the Secretariat's tools (particularly the service standard guidelines in the flip-up calendar and the laminated placemat) helped them quickly in targeting service delivery to clients. The evaluation also found that PERPA Renewal accomplished service standards for visits management that are similar across the three main programs of the Department (PERPA, Trade, and MCO). This approach emphasizes co-ordination of services for visits in an effort to increase efficiency.

It was noted that PERPA renewal was also dealing with the effect of other ongoing programs and discussions at the Departmental level, including Transformation of the Geogroup, which may have impeded upon the Renewal's optimal level of success to date. Some interviewees suggested that there are too many transformation-types of exercises occurring at the same time at the Department which confounds results and reduces the ability to maximize upon.

Finding # 10:Resourcing has been ad-hoc for the Renewal, which has negatively impacted its sustainability.

 

The limited size of the Renewal's operating budget only allowed the Secretariat for one-time, targeted on-site visits to some Missions to inform and educate officers on PERPA Renewal. Although the need for the level of face-to face interaction was questioned by some missions given their relative level of satisfaction with PERPA, many interviewed believed that the face-to-face interactions were effective. Some expressed a need for more on-going, face-to-face interaction with the Secretariat and communication with the CPEO to gain a better understanding of their role. An example of the type of communication that was considered to be effective was the town meeting held by the CPEO in January 2010 and more interactive communication as such would encourage the sustainability of the renewal efforts of PERPA.

Finding # 11:The PERPA community feels less like a "catch-all" group.

 

The Renewal's effort in establishing core services has decreased the phenomenon of PERPA's "catch-all" identity. Many interviewed saw this as a marked accomplishment to know what services were offered by PERPA and which ones could be referred. There however were some missions who were neutral because in their view, PERPA was already defined. Others voiced the opinion that the work of PERPA Renewal was challenging given the varying levels of understanding and PERPA work performed across Missions but the renewal effort was necessary. PERPA Renewal also needed strong leadership from the CPEO to offer support and firmness of decisions. There are also other variables which affected the consistent application of PERPA service standards across the Mission network. They include such issues as the relationship of PERPA managers with their Heads of Mission. Many PERPA officers interviewed voiced the view that the tone of the relationship between PERPA and the Head of Mission can significantly influence PERPA work demands.

Finding # 12:CPEO doesn't seem to be as prominent or adequately resourced for the Renewal. There is also a need for active leadership by geographic and functional ADMs in support of the Renewal.

 

Many PERPA officers interviewed expressed a desire for the CPEO to be a more visible, active, and engaged champion for the community. While it was understood that the CPEO has a multitude of roles and responsibilities, which adds to competing time demands on its functions, many believed that the CPEO's role should be clarified and more prominent. Many interviewees also believed that geographic and functional ADMs need to support the CPEO in promoting Renewal at Missions by their increased presence, and actively discussing PERPA engagement when visiting Missions.

The evaluation also underlines the importance of Heads of Mission, and in larger missions, such as Washington, Beijing and London, the role and leadership of Deputy Heads of Mission as well as senior managers, in leading by example so to ensure that the implementation of the renewal and delivery of the PERPA program succeeds - not only in the PERPA Section of the mission itself but across the entire mission.

Finding # 13:The Secretariat is not widely understood by all interviewees.

 

There is a perception among many interviewed that the increased focus on the Transformation Agenda recently at the Department reduced the effectiveness of the Renewal effort and the role of the Secretariat. Interviewees noted that over the last several months, the visibility and presence of the Renewal had decreased and more attention was devoted to the Transformation Agenda (e.g., more broadcast messages on transformation, more publicity). It would appear then that this absence or reduction in presence contributed to confusion within the PERPA community concerning the link between PERPA Renewal and the Transformation Agenda. It seemed many thought PERPA Renewal was discontinued rather than still active.

Finding # 14:The Secretariat needs to be identified in the organizational structure of DFAIT and to be better positioned in terms of its function.

 

During the evaluation interviews, there were discussions over where best to situate the Secretariat to maximize its effectiveness and its potential to generate change. Few interviewed suggested that the Secretariat be placed in a Strategic Planning and Policy area while many more believed that it could be integrated within the trade renewal (to reflect the integrated model of the Department); still others believed that the Secretariat should be housed within the office of the CPEO. The latter of which would secure its alignment with the CPEO but that this should be an issue for CPEO to champion.

Finding # 15:Varied views were found on the relevance and effectiveness of the DESK Officer role.

 

The evaluation found that the perceived value-added of the DESK officer varies among Missions, and often depends on the relationship with the assigned officer. There was no consensus over where best to situate the "Desk Officer" in the context of Geogroup renewal and its impact on any future directions on political economic reporting. There was also a lack of consensus as to where the DESK Officer should physically be situated (i.e. at Missions, or tele-working from Ottawa). Among the missions interviewed, some believed the role of desk officer should remain unchanged, while others were open to further discussion over any possible reconfiguration of its role. For example, issues of capacity were noted when considering moving the responsibility of DESK from HQ to Missions in that small missions in highly volatile regions of the world would be less likely to assume the capacity of DESK officer responsibilities, in addition to their other mandated functions. Therefore, any reconfiguration would need to be examined strategically and in consultation with Missions.

Finding # 16:There is no systematic tracking of PERPA activities and outputs.

 

The evaluation found that no consistent adoption of reporting agreements across Missions. This contributes to missions' political economic reporting not being directly linked to Departmental priorities. There is also no systematic performance information on the impact of PERPA Renewal, including client feedback. This limits the capacity of the PERPA Renewal or the PERPA program to actively manage and monitor its activities. The Department (GLB) is currently in the process of developing standard mission planning and reporting tools to assist in the reporting of PERPA activities linking them to Departmental priorities. Once information is collected through these templates, then it could be aggregated up for reporting corporately on PERPA.

Finding # 17:There is a need to better define and communicate to staff the vision for PERPA, and the results to be achieved.

 

Without a clearly articulated vision for PERPA and its outcomes, renewal efforts have been less successful than expected. The absence of a vision statement and a results framework has particularly impeded the extent to which Missions in non-priority countries could potentially contribute to the Department's mandate and priorities.

The evaluation found considerable variance across Missions on the nature and extent of work performed by PERPA officers. Missions in the United States, for example, appeared to have a more strategic focus in the delivery of PERPA services, including an integration of PERPA and trade activities. These activities were based on clear and integrated priorities that seemed tangible and achievable to officers interviewed. Other Missions are also extending an effort to better define and report on PERPA results. At the Departmental level, the initiative to develop an on-line business planning and reporting tool would facilitate the task of consistent reporting on Political Economic activities.


8.0 CONCLUSIONS OF THE EVALUATION

Conclusion 1:The Renewal was a positive start in advancing the relevance of PERPA activities to the priorities of the Department, and to the needs of the PERPA community. However, gaps remain. Missions in non-priority countries and regions seek more clarity in their work in an effort for their reporting to be made more relevant. The vision of PERPA and the focus of its activities need to be further identified.

 

Conclusion 2:PERPA Renewal has provided direction and guidance to Missions, especially regarding visit management. Staff at missions are able to refer clients to alternate delivery service providers, and streamline service delivery in this area. The future role of PERPA, including Political Economic reporting vis-à-vis other core services such as advocacy and policy development, needs to be determined and communicated. The "General Relations" label seems to continue to undermine the focus and effectiveness of PERPA.

 

Conclusion 3:The Renewal has been effective in communicating PERPA's core business and defining standards to manage clients' expectations for services. There has been considerable success in provision of practical tools and guidance that help PERPA staff in their daily work. Clarifying roles and responsibilities of PERPA is a significant step forward. However, the Renewal has not resulted in better management of priorities at Missions. There continues to be an inability to link of PERPA activities to DFAIT's priorities.

 

Conclusion 4:There is a need to strengthen the role of the Chief Political Economic Officer, and the leadership of the DMs and ADMs in supporting the Renewal within the Department and with external partners and clients. There is an expressed desire for opportunities to hear from the senior management about their vision of PERPA, how the function contributes to the mandate of DFAIT, and how future challenges may be met.

 

Conclusion 5:It is critical to develop performance measurement and reporting tools to demonstrate the contribution of PERPA activities to the mandate and priorities of the Department and the Government of Canada. Results-based PERPA performance information is also essential for better priorities setting and their effective management.

 

Conclusion 6:The full potential of PERPA Renewal has not been realized because of its limited profile, attention, and resources. Resourcing of the PERPA Renewal has been too modest and ad hoc for the Initiative to have a transformational impact. The community believes that the collective leadership of the Renewal with senior management could be enhanced.

 


9.0 RECOMMENDATIONS

Recommendation 1:It is recommended that the vision and focus of PERPA be better defined and communicated in the context of DFAIT's Transformation Agenda and of the Department becoming a modern foreign ministry.

 

The evaluation of the Renewal revealed a need, in the context of the Transformation Agenda and modernizing of the Department, to better define and communicate to staff the vision of PERPA to guide the community. The vision statement will be important particularly to missions in non-priority countries in contributing to the mandate and priorities of the department.

The work of PERPA staff would be better focused with more clear articulation of priorities and strategic objectives. Corporate efforts currently underway to improve business processes at missions should also help in operational effectiveness of the core services.

Recommendation 2:It is recommended that HQ, in consultation with Missions, define a framework to facilitate more effective use of PERPA core services for achieving the priorities of the department and of the Government of Canada.

 

One of the outcomes of the Renewal that needs more attention is improved PERPA priorities setting and management. Adoption and consistent implementation of a results-based framework would help in improving the relevance of PERPA activities, the achievement of priorities, and in demonstrating the contribution of the PERPA function.

Recommendation 3:It is recommended that PERPA leadership from the Chief Political Economic Officer, the DM and ADM levels be enhanced to sustain the Renewal process.

 

Most interviewees at missions abroad expressed a need for opportunities to have face-to-face dialogue with the CPEO, the DM and ADMs on issues of interest to the PERPA community. Given that PERPA outcomes are by nature less concrete than for example those of Trade, a dialogue with the senior management is important in discussing the opportunities and the challenges facing PERPA staff.

Recommendation 4:It is recommended that the resources allocated to the PERPA Renewal Initiative be commensurate with the Initiative's stated objectives and expected outcomes.

 

As discussed earlier in this report, the PERPA Renewal was undertaken with minimal and ad hoc resources which to some extent slowed down its progress. The resourcing issue also includes senior management's time in providing leadership to the Renewal process. There may also be possibilities to creatively use existing Trade Renewal resources and infrastructure to assist with the implementation of the PERPA Renewal.

Recommendation 5:It is recommended that a cost-effective performance measurement and reporting system be developed to manage and credibly demonstrate the contribution of PERPA.

 

Measuring and reporting on the contribution of PERPA and the PERPA Renewal continues to be a challenge, perhaps more so than other program areas at the Department. Effective performance measurement and reporting of PERPA results are however critical the success of the function both in terms of better meeting the needs of the clients, and for accountabilities purposes.


10.0 MANAGEMENT RESPONSE AND ACTION PLAN

RECOMMENDATION # 1
It is recommended that the vision and focus of PERPA be better defined and communicated in the context of DFAIT's Transformation Agenda and of the Department becoming a modern foreign ministry.

Associated Findings: # 17, 3, 4, 11

Management Responses & Action PlanResponsibility CentreTime Frame
Agreed. The introduction of the New Business Model (NBM) and other Transformation initiatives will have real impacts on the daily work of PERPA staff throughout the department.GEMX1 year *

Contingent on the roll out of the NBM

When it was launched, in February 2008, the vision of the New Way Forward (NWF) initiative was focused on making the department's Political Economic program more strategic, effective and efficient. It aimed at building the most vibrant and effective Political and Economic (PE) team possible and establishing the highest quality services to achieve Canada's foreign policy priorities and goals. The NWF Secretariate (GEMX) continues to communicate this vision.GEMX 
We will review NWF objectives in the context of the NBM. We will assess the role that GEMX can play, either working independently or in concert with other divisions, to clarify and communicate the role of PERPA, in HQ and in Missions, in this context. Once this assessment is completed, these changes will be introduced and communicated through a communications plan.GEMX - BCF4 months *

Contingent on the roll out of the NBM

 

* The implementation of the recommendations will depend on the A-Base funding being established.

RECOMMENDATION # 2
It is recommended that HQ, in consultation with Missions, define a framework to facilitate more effective use of PERPA core services for achieving the priorities of the department and of the Government of Canada.

Associated Findings: # 1, 3, 4

Management Responses & Action PlanResponsibility CentreTime Frame
Agreed. GEMX continues to encourage the use of PERPA core services as currently defined. In the context of the NBM, core services may have to be reviewed and updated. Any revision to core services will proceed in consultation with the PE community throughout the network.GEMX - FXITOver the next year *

Consultations will need to take place once the NBM is implemented

Tools such as the new Mission Planning and Reporting tool, which is being introduced this fiscal year and which was developed in consultation with HQ and Missions, could be used as a starting point to measure political, economic and advocacy work.GEMX - GLD
It is planned that the mandate of the Operations Coordination Committee will include a review the current core services and discussion of ways to measure effectiveness of PE activities as they relate to Departmental and Government priorities.GEMX - GLD

 

RECOMMENDATION 3
It is recommended that PERPA leadership from the Chief Political Economic Officer, the DM and ADM levels be enhanced to sustain the Renewal process.

Associated Findings: # 11, 12, 1

Management Responses & Action PlanResponsibility CentreTime Frame
Agreed. The PE community has expressed, on a number of occasions, that it is keen to see more senior level engagement on this initiative.CPEO 
The Chief Political and Economic Officer (CPEO), an ADM level champion, has been the key driver for the NWF to date. In its mandate it is supported by the NWF Steering Committee, a group mostly made of ADMs and HoMs.GEMX/BCFOver the next 6 months*
To achieve greater success in reaching objectives, an action plan will be developed to deploy Senior Management to convey messages in support of NWF and NBM objectives. This plan will also tailor the role of the CPEO to ensure visibility of the initiative. A communication plan will be developed to support this campaign.DMS, ADMS 
A commitment to PE Renewal in the context of Transformation and the NBM with mandatory requirements will be reflected in PMAs for this fiscal year.PDSOver the next 2 months

 

RECOMMENDATION 4
It is recommended that the resources allocated to the PERPA Renewal Initiative be commensurate with the Initiative's stated objectives and expected outcomes.

Associated Findings: # 10, 5

Management Responses & Action PlanResponsibility CentreTime Frame
PE Renewal was officially launched in February 2008. There is no A-Base funding associated to this initiative. The current 5 FTEs and 90ks operating budget have been so far absorbed by the Geo Group as pressures.

Should management determine that the initiative should continue resources will need to be identified. An action plan would be developed over the next 2 years activities which would likely include the following:

  • Identification of synergies with the NBM and collaboration with other renewal initiatives
  • Update and increase penetration of core services
  • Training: Increased number of users of the PE Renewal online course and sessions with FSDPs
  • Development of a Communication plan
  • Engagement of Senior managers into PE Renewal
  • Outreach Phase II: Briefing of OGDs and provinces
  • Inclusion of PE activities in measurement (e.g., PMAs, PAA, Mission Planning and Reporting tool)
  • Development of a contact management tool reflective of PE community, partners and interlocutors
  • Support for a PE Community Esprit de corps

In any event, expectations have been generated with the PE community. This factor will need to be considered in the decision process.

GEMXOver the next 2 years, should 5 FTEs and an operational budget be allocated *

 

RECOMMENDATION 5
It is recommended that a cost-effective performance measurement and reporting system be developed to manage and credibly demonstrate the contribution of PERPA.

Associated Findings: # 16, 2

Management Responses & Action PlanResponsibility CentreTime Frame
We fully endorse the Evaluation's recommendations and we will begin their implementation.GEMXOver the next year
Measurement of PE activities could build on work undertaken by GLD on the mission planning and reporting tool. As well as the work of PDS to build similar systems for Headquarters divisions.

Collaboration with the department's unit responsible for performance measurement will be sought, to reflect and include PE activities in the department's performance.

Any performance review system should ensure NWF and the NBM are effectively being supported by Senior management, including HoMs.

GEMX, GLD, PDSOngoing

ANNEX A: PERPA RENEWAL LOGIC MODEL

PERPA Renewal Logic Model (PDF 214 KB)*


1. GEMX 2009 Pulse check, Arctic Foreign Policy Centre Planning Activities and Planning Reports, Departmental Steering Committee and Advisory Record of Decisions and Meeting Reports

2. Brussels, Canberra, Harare, Pretoria, Vienna (VIENN, VPERM, VOSCE), Warsaw, Yaoundé, Cairo, Dallas, Jakarta, Mexico, Minneapolis, Nairobi, Paris, Santiago.

Office of the Inspector General


* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Footer

Date Modified:
2013-01-04