Foreign Affairs and International Trade Canada

international.gc.ca

Common menu bar

• Français
• Home
• Contact Us
• Help
• Search
• canada.gc.ca

Breadcrumb

1. Home
2. >
3. About the Department
4. >
5. Accountability

Five-Year DFAIT Evaluation Plan

(PDF Version, 1.8 Mb) ^*

May 2009

Table of Contents

• List of Acronyms
• 1.0 Introduction
• 2.0 Evolution of DFAIT's Evaluation Function
• 3.0 New Treasury Board Secretariat Evaluation Policy
• 4.0 Key Drivers and Strategic Approach
  • 4.1 Coverage
  • 4.2 Materiality
  • 4.3 Priorities
  • 4.4 Corporate Risk Profile and Considerations
• 5.0 Methodology
  • 5.1 Document Review and Preliminary Consultations
  • 5.2 Branch Consultations with Assistant Deputy Ministers (ADMs)
  • 5.3 Preliminary Scoping Exercise
• 6.0 Current and Anticipated Resource Requirements
• 7.0 Five-Year Evaluation Work Plan
• 8.0 Appendix A: Evaluation Division Logic Model
• 9.0 Appendix B: Five-Year Evaluation Work Plan Aligned to the Program Activity Architecture
• 10.0 Appendix C: Gap Analysis based on Past Evaluation Projects (FY03/04-FY08/09) as per the Program Activity Architecture
• 11.0 Appendix D: Evaluation Planning Approach and Considerations Framework

List of Acronyms

ACM

- International Platform Branch

ADM

- Assistant Deputy Minister

AG

- Auditor General

ANA

- Chief Air Negotiator

BFM

- International Business Development, Investment Innovation

CFM

- Consular Affairs and Emergency Management

CIDA

- Canadian International Development Agency

CIFA

- Canada Investment Fund for Africa

CISP

- Community Investment Support Program

CTCB

- Counter-Terrorism Capacity Building

CWD

- Chemical Weapons Destruction

DEC

- Departmental Evaluation Committee

DFAIT

- Department of Foreign Affairs and International Trade

DFD

- Afghanistan Task Force

DFM

- Afghanistan Task Force

DFO

- Fisheries and Oceans Canada

DND

- Department of Defence

EU

- European Union

Fed AA

- Federal Accountability Act

FPT

- Federal/Provincial/Territorial

FTEs

- Full Time Equivalents

FXIT

- Office of Transformation

FY

- Fiscal Year

GC

- Grants and Contributions

GCS

- Global Commerce Strategy

GCSP

- Global Commerce Support Program

GEM

- Europe, Middle East, Maghreb Chief Political Economic Officer

GG

- Governor General

GHC

- Aboriginal and Circumpolar Affairs Division

GLM

- Latin America and Caribbean

GNM

- North America

GOC

- Government of Canada

GPM

- Asia and Africa and Chief Trade Commissioner

GPP

- Global Partnership Program

GPPC

- Global Partnership Program for Competitiveness

GPSF

- Global Peace and Security Fund

GSRP

- Global Security Reporting Program

HCM

- Human Resources

HQ

- Head Quarters

IBD

- International Business Development

ICT

- International Counter Terrorism Unit

ICX

- International Crime and Terrorism

IFM

- International Security Branch and Office of Political Director

IGX

- Global Partnership Program

IM / IT

- Information Management / Information Technology

IRD

- Stabilization and Reconstruction Task Force (START) Secretariat

ISTPP

- International Science and Technology Partnerships Program (ISTPP)

JFM

- Legal Advisor

MAF

- Management Accountability Framework

MDD

- Environment, Energy and Sustainable Development Bureau

MFM

- Global Issues

MHD

- Human Rights and Democracy Bureau

NAPP

- North American Platform Program

NRC

- Natural Resources Canada

NRS

- Nuclear Radiological Security

OGDs

- Other Government Departments

OPI

- Office of Primary Interest

OSI

- Office of Secondary Interest

PA

- Program Activity

PAA

- Program Activity Architecture

PCO

- Privy Council Office

PERPA

- Political Economic Program

PFM

- Strategic Policy Planning

PMO

- Prime Minister Office

PMP / PMA

- Performance Management Program / Performance Management Agreement

PPTC

- Passport Canada

PSAT

- Public Safety and Anti-Terrorism

RFWS

- Redirection of Former Weapon Scientists

SCM

- Corporate Finance and Operations

TB

- Treasury Board

TBS

- Treasury Board Secretariat

TCS

- Trade Commissioner Service

TOR

- Terms of Reference

TPC

- Trade Policy and Negotiations Branch

TPI

- Export and Import Controls

UN

- United Nations

XDD

- Chief of Protocol

ZID

- Office of the Inspector General

ZIE

- Evaluation Division

1.0 Introduction

This rolling five-year evaluation plan provides both the strategic approach and proposed list of evaluation projects that will be undertaken by the Evaluation Division (ZIE) over the next five years (FY 2009-2014).

This plan takes into consideration four criteria which underpin the evaluations selected:

1. 100% Coverage of all Direct Program Spending over a five-year cycle as per the Treasury Board Evaluation Policy
2. Federal Government and Departmental Priorities
3. Corporate Risk Factors and Considerations
4. Materiality in terms of budgets and FTEs related to programs/initiatives.

The first criteria of providing full coverage of all direct program spending within the Department over a five year cycle was given the highest priority for the selection of evaluation projects consistent with the new evaluation policy recently launched by Treasury Board. In addition, a risk-based approach to planning coverage was used as shown in the table in Appendix D: Evaluation Planning Approach and Considerations Framework where evaluation coverage was assessed against Government of Canada and DFAIT priorities, corporate risk profile ratings, past coverage, materiality, and Treasury Board Submission and Grants and Contributions requirements.

This five-year DFAIT Evaluation Plan includes a number of strategic evaluations that cut across a number of program and policy areas allowing for the conduct of evaluations within the broadest focus possible. Consultations with senior management greatly aided in the selection of these and other evaluations on this plan.

An Evaluation Division logic model has been introduced in the plan this fiscal year to demonstrate the Department's accountability for the evaluation function as well as the linkage between evaluation and departmental strategic outcomes. Further, follow-up to the recommendations in management responses to evaluations was instituted by the Evaluation Division last fiscal year using an approach approved by the Departmental Evaluation Committee (DEC).

While specific evaluations are identified in this plan, this is an evergreen document which will be reviewed and updated on an annual basis. It is also subject to an annual approval process by the Departmental Evaluation Committee (DEC) and the Treasury Board Secretariat.

2.0 Evolution of DFAIT's Evaluation Function

The evaluation function has evolved significantly over the last few years. With an increased focus on accountability through the introduction of the Federal Accountability Act (Fed AA) in December 2006, federal government departments have been expected to demonstrate achieved results. The Fed AA introduced measures to strengthen accountability and increase transparency within the Federal Government. This has resulted in more emphasis on the evaluation function.

In addition, exercises such as the Strategic Review and the renewal of the Expenditure Management System, and the Management Accountability Framework (MAF) have also focussed on the importance of having systems in place to measure and demonstrate results achieved. Evidence-based program assessments such as evaluations are increasingly being used by federal departments to clearly demonstrate the relevance, performance, and value for money of their programs, policies and initiatives and as a basis for the allocation and reallocation of direct program spending.

With a new evaluation policy in effect on April 1, 2009, Departments have until March 31, 2013 to fully implement the new policy and associated directives. The Evaluation Division (ZIE) of the Inspector General's Office at DFAIT has been working proactively to meet the requirements of the new policy as briefly described in the section below. Further, the Evaluation Division (ZIE) has undertaken a number of initiatives to effectively manage and resource the evaluation function which is described in the section on resource requirements.

3.0 New Treasury Board Secretariat Evaluation Policy

The new Evaluation Policy was launched by Treasury Board in April 2009 and replaced the 2001 policy. The objective of the policy is to ensure that evaluation information is available to Ministers, departments and central agencies to support evidence-based decision-making on policy, expenditure management and program improvements.

The new policy is in addition to the requirements of the Fed AA to evaluate all Grants Contributions programs over a period of five years.

The new Evaluation Policy:

• puts in place a regular and systematic cycle of program evaluation as a normal part of the business of departments;
• improves the information base for Strategic Reviews through expanded coverage and an increased focus on value for money;
• requires all direct program spending, excluding grants and contributions, to be evaluated every five years. Departments have until March 31, 2013 to fully meet this requirement;
• requires all Grants and Contributions to be evaluated at least once every five years;
• promotes the neutrality of the evaluation function and improves the credibility of evaluation results by enhancing standards for the conduct of evaluations and assessing value for money;
• ensures that the right capacities, people and systems are in place to support higher quality evaluations.

The Evaluation Division of the Inspector General's Office at DFAIT has been working proactively to meet the requirements of the new policy. In September 2007, DFAIT established the Departmental Evaluation Committee (DEC) as proposed in the new policy. In February 2008, a five-year Departmental Evaluation Plan was finalized and approved by the DEC. Both the 2008 and this new 2009 plan have moved the department toward providing 100% coverage of all direct program spending over a five-year cycle.

Full details of the new policy on Evaluation are available on the Treasury Board Secretariat's site.

4.0 Key Drivers and Strategic Approach

The Government of Canada has undergone a major change in evaluation policy which drives this evaluation plan by requiring federal government departments to build capacity to deliver on the commitment to provide 100% coverage over a five-year period.

As mentioned above, a risk-based approach to planning 100% coverage of all direct program spending was also employed in the development of this plan. In the table in Appendix D: Evaluation Planning Approach and Considerations Framework, each evaluation is mapped against the following factors: Government of Canada and DFAIT priorities, corporate risk profile ratings, past coverage, materiality, and Treasury Board Submission and Grants and Contributions requirements.

Below is a description of each of the four criteria which underpin the evaluations selected for the five-year plan:

4.1 Coverage

Determining 100% coverage over a five-year period has been based on the Program Activity Architecture (PAA) 2009/2010 version prepared on February 19, 2009. The Program Activity Level of the PAA, the highest tier of the framework, has been used for the selection of evaluation projects for this five-year evaluation plan.

The department's seven program activities are as follows:

1. PA 1: International Policy Advice and Integration: providing strategic direction, intelligence and advice, including integration and coordination of Canada's foreign and international economic policies.
2. PA 2: Diplomacy and Advocacy: engaging and influencing international players and delivering international programs and diplomacy.
3. PA 3: International Commerce: managing and delivering commerce services and advice to Canadian business.
4. PA 4: Consular Affairs: managing and delivering consular services and advice to Canadians.
5. PA 5: Passport Canada: managing and delivering passport services through the use of the Passport Canada Revolving Fund.
6. PA 6: Canada's International Platform: Support at Headquarters: managing and delivering services and infrastructure at headquarters to enable Canada's representation abroad.
7. PA 7: Canada's International Platform: Support at Missions Abroad: managing and delivering services and infrastructure at missions to enable Canada's representation abroad.

DFAIT's Program Activity Architecture (PAA) 2009/2010 document includes further information on "Ongoing Planning Highlights" related to each of the Program Activity areas. As well, three boards that guide decisions in these areas are identified, namely, The Policy and Programs Board, External Services Board, and the Missions Board. Internal Services including Corporate, Human Resources, Executive, Legal and Communications are listed at the bottom of the architecture since they support all strategic outcomes. Internal Services are guided by the Core Services Board. Note that this evaluation plan includes evaluations related to Internal Services as well given the linkage to all strategic outcomes and the direct relevance and performance of these areas in support of all departmental efforts.

4.2 Materiality

Materiality in terms of FTEs and budgets has influenced the choice of evaluation projects. Similar and smaller projects have been grouped so as to increase economies of scale and to review programs in a larger context, where possible.

4.3 Priorities

Government and departmental priorities guided the development of the five-year DFAIT Evaluation Plan, placing emphasis on conducting evaluation projects that examine the work being undertaken in each of these areas. The attached Appendix on "Evaluations Planning Approach and Considerations Framework" shows linkages of evaluations to the various priorities.

The three policy and program priorities and one management priority identified for fiscal year 2009-10 for DFAIT are as follows:

DFAIT 2009-2010 Priorities

Policy and Planning Priorities

1. Greater economic opportunity for Canada, with a focus on growing/emerging markets

   Contributing to economic recovery and opportunity by: Implementing Global Commerce Strategy and boosting Canadian commercial engagement in the world. Advancing bilateral negotiations on trade liberalization and market access with key states, with a new attention to Asia. Developing Canada-EU economic partnership. Finalizing implementation of Free Trade Agreements with European Free Trade Association, Peru, Colombia and Jordan. Focusing on air services as well as foreign investment promotion and protection agreement negotiations. Delivering programming for 2010 Winter Olympic Games. Advancing Canadian interests in the Doha Round. Preparing for the 2010 G8 Summit.

2. United States and the Hemisphere

   Implementing comprehensive strategy for relations with the US, including: competitiveness and global economy, peace and security, trade relations, border, energy and climate change, Afghanistan, Arctic. Advance relations with Mexico. Implement Americas Strategy, with a focus on specific initiative to promote greater economic. Prosperity, security and democratic governance through enhanced bilateral and multilateral engagement.

3. Afghanistan, including in the context of neighbouring countries

   Integrated approach to Afghanistan policy priorities for which the department is responsible: Afghanistan-Pakistan border management; Afghan National Police, justice and corrections, in Kandahar province; and supporting Afghan-led reconciliation. Implement all diplomatic aspects of the Afghanistan mission. Develop integrated regional approach, including with Pakistan. Develop integrated strategy for the transition of Canada's mission in Afghanistan until 2011.

4. Transforming the Department

   Aligning with government priorities; strengthening our international platform; improving services to Canadians; focusing on core business; strengthening our accountability; renewing our human resources. Focus on missions and regional offices. Strengthen policy capacity. Simplify business practices.

   As part of the planning and scoping of each evaluation project, both the Alignment Review and the Transformation Agenda (identified above as the fourth priority), will be considered, e.g., in developing Terms of Reference (TORs), interviews, etc. The Transformation Agenda is also identified as a specific area for evaluation on the evaluation plan for FY 2009-2010.

   The vision of the transformation agenda is a modern, integrated 21^st century foreign service and trade ministry that refocuses on its core business, transfers resources from headquarters to missions abroad and regional offices in Canada to improve service delivery, and better aligns activities with Government of Canada priorities.

The transformation agenda focusses on six key themes:

1. Aligning with government priorities
2. Strengthening our international platform
3. Improving services to Canadians
4. Focussing on our core business
5. Strengthening accountability
6. Renewing our human resources

4.4 Corporate Risk Profile and Considerations

As provided in Appendix D: Evaluation Planning Approach and Considerations Framework attached to this plan, the Corporate Risk Profile and strategic considerations were factored into the development of this evaluation plan. All planned evaluations were mapped against the Corporate Risk Profile in Appendix D: Evaluation Planning Approach and Considerations Framework.

The annual Corporate Risk Profile prepared by DFAIT identifies a number of factors that can affect the department's ability to succeed in its work. Risk factors such as unanticipated policy shifts, international instability, catastrophic disasters, etc. were examined in the development of the risk profile . Their potential impact and likelihood on program and policy areas were assessed for DFAIT's Corporate Risk Profile.

As mentioned throughout this document, the Evaluation Division (ZIE) carried out a number of consultations with Assistant Deputy Ministers to identify priority areas of evaluation. The planned evaluations and their scheduling reflect the outcome of these consultations including the ADMs' perception of risk.

5.0 Methodology

5.1 Document Review and Preliminary Consultations

Further to the development of a data base last fiscal year that mapped evaluations to the Program Activity Architecture and the evaluation criteria of coverage, priorities, corporate risk factors, and materiality as identified in this plan, the Evaluation Division conducted an extensive review of corporate information to update the data base. In addition, preliminary planning meetings were held with a few key Assistant Deputy Ministers and senior managers including the Office of Transformation (FXIT) to help identify any strategic considerations in developing the 2009-10 plan.

Included in document review were the following: Treasury Board documentation on the new Evaluation Policy, the Program Activity Architecture for DFAIT, the Report on Plans and Priorities for DFAIT, the Departmental Performance Report for DFAIT, and material related to the Alignment Review, Strategic Review, the Management Accountability Framework, and the draft Corporate Risk Profile. A Gap Analysis of evaluations produced over the last ten years was conducted to help identify evaluation coverage priorities.

The attached annexes: Appendix B: Five-Year Evaluation Work Plan Aligned to the Program Activity Architecture, and Appendix C: Gap Analysis based on Past Evaluation Projects (2002-2009) as per the Program Activity Architecture were updated from last fiscal year.

5.2 Branch Consultations with Assistant Deputy Ministers (ADMs)

Consultations were held with all ADMs in the Department and formed the basis for the selection of many of the evaluation projects listed in the plan. ADMs were provided with an analysis of evaluation coverage and gaps with suggestions for evaluations that also addressed government and departmental priorities, areas of significant materiality, risk and those which could be useful to many areas so as to conduct horizontal/cross-cutting evaluations. ADMs provided feedback and as a result, modifications were made to the proposed evaluation project list. Additional input from these consultations identified key issues facing the Department and areas where synergies in evaluation could be formed, e.g., progress in achieving results identified for the transformation agenda could be an aspect of a number of evaluations.

5.3 Preliminary Scoping Exercise

The Evaluation Division undertook a preliminary scoping exercise at the end of FY 2008-2009 for the purpose of determining budgets and resource requirements for the evaluations being undertaken in FY 2009-10. The preliminary scoping exercise involved an initial document review, identification of strategic considerations including linkages to past, current and future evaluations, suggestions for proposed methodologies, initial assignment of time allocations to various evaluation tasks, and proposed financial and human resources for each evaluation including internal and external resources (e.g., consultants). This exercise was also utilized to allocate workloads to the evaluation team in the division and to plan an appropriate schedule of evaluations for the fiscal year to meet deadlines and priorities identified by senior management as well as distribute the workload in a reasonable fashion.

6.0 Current and Anticipated Resource Requirements

The Evaluation Division (ZIE) is currently resourced for 12 A-based positions with the possibility to staff four additional positions through GC program resources for a total complement of 16 employees. Currently, 12 positions are staffed permanently with the anticipation of two more positions being filled in FY 2009-10. Since receiving an A-based reference level budget in FY 2007-2008, increments have been received over the last fiscal year to help the division meet its requirements for evaluation coverage according to the new Evaluation Policy. The additional support received annually starting in December 2007 as part of the Fed AA Treasury Board Submission to strengthen the evaluation function across federal departments has been well utilized toward sufficiently resourcing the division's FTEs. As well, the necessary funding has been received in the division's A-base from Grants and Contributions programs such as the Global Partnership Program (GPP), the Global Peace and Security Fund (GPSF), the Counter Terrorism Capacity Building (CTCB) program and the Global Commerce Strategy (GCS). Funding is also anticipated in relation to various Gs Cs program that are currently being considered.

The last fiscal year has involved a dedicated effort to building the capacity of the Evaluation Division and to planning for the future. Efforts have included developing a learning plan for the team that focuses on both current and anticipated competencies. A small internal working group was established that conducted an analysis of divisional needs and prepared a proposed approach for the divisional learning plan including surveying staff to identify their needs and priorities.

Resourcing for the future has included hiring and bridging Co-op students from universities and preparing associated training plans to help them develop over the coming years from junior officers to eventually attaining positions at the evaluation manager level.

Equally important, the Evaluation Division at DFAIT has focussed on building and maintaining a solid infrastructure of the evaluation function and evaluation professionals by putting in place a vision, a logic model for the evaluation function in DFAIT, guidelines, directives, and tools to ensure quality and consistency of evaluations and the work undertaken in this regard.

7.0 Five-Year evaluation Work Plan

8.0 Appendix A: Evaluation Division Logic Model

Logic Model (PDF Version, 1.35 MB) ^*

9.0 Appendix B: Five-Year Evaluation Work Plan Aligned to the Program Activity Architecture

Work Plan Aligned to PAA (PDF Version, 90 KB)*

10.0 Appendix C: Gap Analysis based on Past Evaluation Projects (FY03/04-FY08/09) as per the Program Activity Architecture

Gap Analysis (PDF Version, 90 KB)*

11.0 Appendix D: Evaluation Planning Approach and Considerations Framework

Evaluation Framework (PDF Version, 140 KB)*

Office of the Inspector General

Footnotes:

¹ As per Program Activity Architecture version approved by TBS June 7, 2007

² Evaluations shaded in grey are evaluations started in 2008-2009 to be completed in 2009-2010

³ Evaluations shaded in tan are evaluations conducted by OGDs that may have recommendations for and request management responses from DFAIT

⁴ Evaluations shaded in blue are evaluations that will start in FY 2009/2010 but will be completed in FY 2010/2011

⁵ As per Program Activity Architecture version approved by TBS June 7, 2007.

⁶ As per Program Activity Architecture version approved by TBS June 7, 2007.

⁷ As per Program Activity Architecture version approved by TBS June 7, 2007.

⁸ As per Program Activity Architecture version approved by TBS June 7, 2007.

^* If you require a plug-in or a third-party software to view this file, please visit the alternative formats section of our help page.

Primary navigation

Date Modified:

2013-02-22