Evaluation of the Client Service Fund and the Integrative Trade Strategy Fund
Prepared by the Diplomacy, Trade and Corporate Evaluation Division (PRE)
Strategic Policy Branch (PFM)
Global Affairs Canada
PRE would like to thank the Evaluation Advisory Committee (EAC) and members of the following divisions and bureaus who provided their support and contributions to this evaluation: Sector Practices (BBD) and Trade Commissioner Service – Operations (BTD) bureaus; Geographic, Functional, Financial Management, Mission Support, and Trade and Economic Analysis divisions; and Senior Trade Commissioners (STCs) and Trade Commissioners (TCs) at missions abroad and Regional Offices (ROs) in Canada.
This report provides findings and recommendations from the 2016-2017 evaluation of the Client Service Fund (CSF) and the Integrative Trade Strategy Fund (ITSF).
The Trade Commissioner Service (TCS) relies on the CSF and ITSF to fund strategic initiatives, projects and services that help Canadian businesses to participate and expand into global markets. As a proportion of total trade budgets, the CSF and ITSF represent an increasingly important source of funding to Commercial-Economic sections due to a decline in funding from Other Government Departments and partners.
The ITSF serves as a highly targeted and strategic source of funding aligned with the Sector Analysis and Priorities for the TCS, while the CSF continues to be a discretionary source of funding that allows the TCS to respond to local, market-driven opportunities and client demands. Access to discretionary CSF and targeted ITSF funding enhances TCS capacity to serve Canadian businesses in their efforts to expand into global markets.
Trade initiatives funded in part through the CSF and ITSF support the advancement of Departmental international business development objectives by contributing to the expansion of Canadian business participation in global markets, increasing interactions between Canadian clients and global business partners, and generating new business opportunities and results for Canadian clients.
Strategic guidance provided by the Trade Sectors Bureau for the ITSF was found to be clear for all of the TCS’s priority sectors, however, greater clarity and consistency in the provision of operational guidance across all sectors is needed. Further, stakeholders at missions and headquarters expressed a need for greater access to hospitality to support CSF and ITSF- funded initiatives since hospitality is considered an essential cost of doing business.
The CSF and ITSF-funding mechanisms are well-managed and have effective governance structures that provide oversight and support the achievement of results through sound project planning and implementation. The financial monitoring and reallocation systems in place are highly effective and contribute to low lapse rates. However, uncertainty of the ITSF budget and timing of ITSF disbursements to posts was found to adversely impact on program delivery, and the transaction costs of administering the ITSF was reported to be high at missions, headquarters and Common Service Delivery Points.
Finally, the database review demonstrated a lack of interoperability between CSF and ITSF planning (Strategia) and reporting (TRIO2) systems which increases the administrative burden on posts, introduces a risk that data will be entered inaccurately, and limits effective performance measurement.
- The Program should review the guidelines and expense eligibility criteria which govern the use of funds for hospitality with the view to making funds more accessible to support CSF and ITSF- funded initiatives.
- The Program should consider systems to provide more consistency within the Trade Sectors bureau in the provision of operational guidance and feedback to STCs for the ITSF.
- The program should consider options to release ITSF funds earlier in the year, increase the predictability of funding, and reduce the administrative burden associated with the ITSF at missions and HQ.
- The Program, in consultation with the relevant responsibility centres, and aligned with the Departmental Data Strategy, should explore options to improve data quality and reporting on CSF and ITSF initiatives and their results to help make informed, evidence-based decisions.
Acronyms and Abbreviations
- Trade Sectors
- Clean Technology, Infrastructure and Life Sciences
- Multi-Sectors Practices
- Aerospace, Automotive, Defence and ICT Practices
- International Business Development and Innovation Branch (Chief Trade Commissioner)
- Business Intelligence
- Investor Services
- Trade Planning, Coordination and Responsible Business Practices Division
- Systems and Analysis
- Trade Commissioner Service - Operations
- Canadian Commercial Interests List
- Canadian Direct Investment Abroad
- Commercial Economic (section)
- Commercial Economic Plan
- Canadian Food Inspection Agency
- Canadian Manufacturers and Exporters
- Fund Commitment
- Client Relationship Management
- Common Service Delivery Point
- Client Service Fund
- Corporate Social Responsibility
- Canadian Technology Accelerator
- Economic Outcome Facilitated
- Foreign Policy and Diplomacy Service
- Fiscal Year
- Global Affairs Canada
- Government of Canada
- Global Value Chains
- Head of Mission
- Human Resource Management System
- International Business Development
- International Commerce Coordination Board
- Information and Communication Technologies
- Integrated Management System
- Integrative Trade Strategy Fund
- Key Performance Indicator
- Multi-National Enterprise
- Management Response Action Plan
- North American Partnership Program
- Other Government Department
- Organisation for Economic Co-operation and evelopment
- Operational budget
- Opportunity Pursued (KPI)
- Opportunity Identified
- Opportunity Referred
- Post Initiative Fund
- Q1 (2,3,4)
- First Quarter (second, third, fourth)
- Regional Office
- Return on Investment
- Small and Medium-Sized Enterprise
- Sector Practice Approach
- Senior Trade Commissioner
- Trade Commissioner
- Trade Commissioner Assistant
- Trade Commissioner Service
The CSF was launched in April 2002 as the Department’s principal fund for supporting proactive initiatives undertaken by the Trade Commissioner Service (TCS) to assist Canadian industry to compete globally and influence foreign decision-makers to select Canada as a source of goods and services.
On April 1, 2009, the Integrative Trade Strategy (ITS) was introduced whereby 50% of the total CSF allocation was devoted to funding strategic initiatives aligned with the Department’s priority markets and sectors through a competitive application process.
In 2013-2014, the Global Value Chains (GVC) Fund and the Canadian Direct Investment Abroad (CDIA) Fund were amalgamated under the ITS, establishing a single, larger fund known as the Integrative Trade Strategy Fund (ITSF).
The Corporate Social Responsibility (CSR) and Edu-Canada funds are allocated and tracked through the same processes as ITSF funds, but are administered separately from the ITSF funding envelope.
CSF and ITSF Governance Structure
|Governance Body||Roles and Responsibilities|
Chief Trade Commissioner (BFM)
Ultimate accountability for the utilization of the CSF and ITSF
International Commerce Coordination Board (ICCB)
Strategic direction and oversight
CSF and ITSF Steering Committee
Delivery of the program and allocation of financial resources
Trade Planning, Coordination and Responsible Business Practices Division (BTA)
Commercial program planning, financial and administrative management
Trade Sectors Bureau (BBD), informed by Private Sector Advisory Groups
Provides strategic and operational guidance to missions for the ITSF, and adjudicates and distributes ITSF funds through a competitive process
The CSF began with annual funding levels of $8.7M in fiscal year (FY) 2003-2004, which was reduced to $6.3M in 2004-2005, and $5.9M per year from 2005-2006 to 2007-2008. Since 2008-2009, the yearly base level budget has been maintained at $4.9M, divided between the CSF and ITSF components. Due to departmental priorities and/or supplemental injections of funding, actual annual budgets have fluctuated from $4.76M in 2013-2014 to $5.88M in 2015-2016.
|FY||Fund||Budget ($)||Initial Allocation ($)||Final Allocation ($)|
Geographic Disbursement of CSF and ITSF Funds
Each year, CSF and ITSF funds are disbursed to approximately 140 missions abroad and Regional Offices (ROs) in Canada.
Following consultations with senior management in the International Business Development and Innovation Branch (Chief Trade Commissioner) (BFM), the CSF and ITSF were identified for evaluation in fiscal year 2016-2017.
- the CSF has not been evaluated in six years, since 2009-2010;
- the ITSF has not undergone an evaluation since its creation.
The authority to proceed with the evaluation was approved by the Deputy Minister of Foreign Affairs as part of the Global Affairs Canada (GAC) Five Year Evaluation Plan.
The focus of the evaluation is on the relevance, performance and efficiency of the CSF and ITSF, as well as stakeholder perceptions on the funds’ usefulness. The Corporate Social Responsibility (CSR) and EduCanada programs were not part of the evaluation. EduCanada will be subject to a separate evaluation (expected in 2018), and CSR was recently evaluated in 2014.
The evaluation reference period is from 2013-2014 to present, to reflect recent CSF and ITSF practices and processes.
The evaluation objectives were to assess the following:
- The continued relevance of the CSF and ITSF to the TCS community in support of Canadian business clients.
- The extent to which CSF and ITSF have contributed to departmental international business development objectives and results for Canadian businesses.
- The efficiency and effectiveness of the governance structure, program practices and processes.
The evaluation utilized a mixed method approach that consisted of quantitative and qualitative data collected using the methods and data sources described below to respond to the evaluation questions. Data was collected between October 2016 and February 2017.
- Standardized protocols were used to obtain information on the full list of evaluation questions.
- Interviews included representatives from headquarters (HQ) divisions (e.g. financial management geographic mission support and trade and economic analysis) and STCs and TCs at missions abroad.
- An online survey was sent to all STCs at missions abroad and regional offices in Canada.
- Majority of questions used Likert Scales, and each question had a comment section for participants to provide feedback
- In total, 164 surveys were returned of which 114 were fully completed and 50 partially completed; 78 STCs and 86 TCs returned a survey
A database review was conducted based on the following data:
- Strategia data generated by BTA on the CSF and ITSF-funded initiatives at 8 missions (BEJING, BERUT, BGOTA, GJARA, LDN, NROBI, TOKYO, and WSHDC) in fiscal year 2014-2015, and TRIO2 results data generated by the Systems and Analysis division (BTB)
- Financial data provided by BTA
- TCS Client Survey results linked to the database sample
- Program staff were interviewed in order to obtain information on the full list of evaluation issues and questions.
- Program staff includes members of BTA, the Sector Bureau (BBD, BBR, BBH, BBI), Systems and Analysis (BTB), and Investor Services (BIS).
Documents reviewed included:
- Program guidelines – CSF, ITSF, CSR, EduCanada and ExportUSA
- CSF and ITSF Wiki, e.g. governance, performance and financial reports
- Trade Horizons, e.g. performance measurement
- Sector Analysis and Priorities documents and Government of Canada Trade Strategies
1) The CSF and ITSF-funding mechanisms are internal to GAC, and clients for the most part are not aware that the funds exist. As a result, no direct client feedback was collected.
Since the TCS deliver CSF and ITSF-funded products and services to clients, results from 144 TCS Online Client Feedback surveys linked to the CSF and ITSF-funded initiatives in the database sample were used as a proxy measure of client satisfaction. Published research reports on the needs of Small and Medium-Sized Enterprises (SMEs) were also used in the analysis.
2) A representative sample was not possible due to: i) system limitations and ii) the high number of ITSF and CSF funded initiatives that took place over the evaluation reference period (e.g. over 1,500 initiatives for the ITSF alone).
The evaluation used a sample of CSF and ITSF initiatives at 8 missions. Observations from this sample cannot be inferred for the total population.
3) CSF and ITSF resources are frequently used by STCs in conjunction with other funding sources which complicates the attribution of outcomes.
The evaluation linked outcomes to their respective source where possible.
4) For the database review of CSF and ITSF-funded initiatives at 8 missions (see Finding 4), the ability to link key performance indicators (KPIs) in TRIO2 to initiatives in Strategia was dependent on officers making reference to the initiative when entering results in TRIO2. The Case tab in TRIO2 facilitates the linkage between key events and KPIs, but was newly introduced in 2014-2015 (the year of the database sample) and may not have been fully used in its first year.
For the database sample, the evaluation linked KPIs to the initiatives using the TRIO2 Case tab and KPI descriptions, however, KPIs may not represent all the results achieved.
5) The total number of TCs that received the survey is unknown due to the lack of email distribution list for TCs.
STCs were asked to share the online survey with the TCs that they considered to be knowledgeable on the CSF and ITSF. This approach resulted in 86 surveys returned by TCs, representing an even balance in responses between STCs and TCs.
Finding 1: The CSF and ITSF represent an increasingly important source of program funding to the TCS.
As illustrated in Figure 1, the CSF and ITSF represent an increasing proportion of the total funds available to the TCS due to a decline in external funding from other government departments (OGDs) and other partnersFootnote *.
Figure 1. Mission Trade Budgets by Funding Type, 2013-2014 to 2015-2016
As illustrated in Figure 2, since 2013-2014, total mission trade budgets (including CSF, ITSF, operational budgets (OPs), other GAC programs, and funding from OGDs and other partners) have declined by $6M or 25%. With increasingly limited program funds, particularly for travel and hospitality, STC and TC survey respondents said that without the CSF and ITSF, their ability to organize events or travel to support Canadian client’s needs would be limited.
Figure 2. Total Mission Trade Budgets, 2013-2014 to 2015-2016
Survey respondents strongly agreed that the CSF and ITSF represent a significant and meaningful contribution to the available resources of the Commercial-Economic (C-E) section at their post:
- For the CSF, 96% of STC and TC respondents agreed or strongly agreed with this statement, and 84.6% agreed or strongly agreed for the ITSF.
- Highest levels of agreement were recorded in Europe (92%), Asia (92%), the Middle East (96%), South America and the Caribbean (100%). Lowest levels of agreement were recorded in Africa (71%)
Interview data: 100% of stakeholders interviewed agreed that the CSF and ITSF are vital sources of funding to the TCS.
Finding 2: Access to discretionary and targeted funding supports TCS capacity to serve Canadian businesses in their efforts to participate in global markets.
STCs, TCs, and stakeholders affirmed the advantages of each fund and the niche that they occupy:
- Targeted and strategic source of funding aligned with the Sector Analysis and Priorities for the TCS
- Consultations between the Sector Bureau (BBD) and Private Sector Advisory Groups, OGD partners, industry and other key stakeholders inform the Sector Practice Strategies and help the TCS to align proposals with Canadian capabilities and interests
- Competitive nature encourages more strategic and focused initiatives
- Costlier activities that may not otherwise be possible with limited C-E budgets, i.e. supporting medium to large-scale delegations to key, regional, multi-post events (e.g. Farnborough Air Show) which are crucial to building the TCS network and promoting Canadian capabilities
- Customized business-to-business (B2B) activities are encouraged to bring Canadian clients and local buyers into direct contact
- Discretionary source of funding that allows the TCS to respond to local, market-driven opportunities and client demands
- Stable source of funds outside of the operational budget (OPs)
- Allows for cross-sectoral programming and emerging sectors, or sectors not covered by the ITSF (e.g. Agriculture / Agri-food)
- Complements and covers deficits from other sources of funding and foreign exchange deficiencies
- Provides STCs with the flexibility to respond when changes to initiatives occur and to emerging, mid-year demands
Achievement of expected outcomes
Finding 3: The CSF and ITSF support Government of Canada International Business Development (IBD) objectives.
CSF and ITSF contributions to the expansion of Canadian business participation in global markets:
The Report on Plans and Priorities reported that in 2015-2016 the TCS supported Canadian companies by conducting 1,220 trade initiatives funded through the CSF and ITSF (and EduCanada) which contributed to 963 commercial agreements concluded as compared to 934 commercial agreements concluded in 2014-2015.
Research conducted by the Organisation for Economic Co-operation and Development (OECD)Footnote 1 and Canadian Manufacturers & Exporters (CME)Footnote 2 indicate that the types of initiatives and activities funded by programs such as the CSF and ITSF are aligned with the needs of Canadian Small and Medium-Sized Enterprises (SMEs).
The CSF and ITSF were highly rated by STCs and TCs in terms of their effectiveness at contributing to the following objectives:
- Contribute to the expansion of Canadian business participation in global markets by funding strategic initiatives, projects and services within the scope of the TCS.
- Increase interaction and strategic alliances between Canadian clients and global business partners.
- Generate new business opportunities, leads and results for Canadian clients.
- Provide detailed market information and intelligence to specific Canadian clients.
- Support strategic initiatives / partnerships as defined in the commercial plans of C-E sections.
However, the CSF and ITSF received lower ratings in terms of their effectiveness at contributing to the following objectives, for the reasons provided below:
- Provide information and advisory resources to TCS clients wishing to pursue foreign investment.
- May be due to respondents failing to recognize that CDIA is included under the ITSF.
- Expand Canadian business participation in, and awareness of, Multinational Enterprise (MNE) Global Value Chains (GVC).
- GVC is no longer viewed as a discrete programming domain.
- Promote Canada as a competitive location and partner for investment and innovation.
- Funded under Invest in Canada program, Tier III, and Champions program rather than ITSF.
Overall, the CSF (83%) received a higher ranking compared to the ITSF (76%) in terms of its effectiveness at contributing to CSF and ITSF objectives.
Survey respondents and stakeholders interviewed at HQ identified the following types of initiatives/activities as the most effective at generating outcomes for clients: workshops/seminars featuring Canadian capabilities; matchmaking events; trade missions in market; participation in trade shows; market studies; and reverse trade missions to Canada.
For the CSF specifically, support for core TCS work, such as local travel to conduct outreach and gain contacts, market intelligence and hosting events with local contacts were also noted to be highly effective initiatives/activities.
Finding 4: The project sample demonstrates that the CSF and ITSF contribute to results for Canadian businesses.
Project sample description
The evaluation relied on performance data from a sample of 123 CSF and ITSF-funded initiatives at 8 missions (Beijing, Beirut, Bogota, Guadalajara, London, Nairobi, Tokyo, and Washington). Of this sample, 79 (65%) initiatives were traceable to KPIs in TRIO2.
Based on mission expenditure data in Strategia, the project sample represents 80% of the ITSF funds ($343.8K) and 53% of CSF funds ($213.3K) allocated to these missions in 2014-2015. The sample spans across different sectors including Aerospace, Automotive, Clean Technology, Information and Communication Technologies (ICT), Infrastructure and Life Sciences. Key events in the sample include: Farnborough Air Show, Mobile World Congress, and Game Developers Conference.
Other sources of funding comprised 60% ($880K) of the total funding for the project sample, with federal and provincial partners contributing the highest share (31%).
KPI data reported in this finding includes KPIs achieved up to January 2017.
As noted in the Methodology section, performance data must be regarded with caution for the following reasons:
- A representative sample was not possible, therefore, the results of the evaluation project sample are indicative of the sample only and do not represent the total population. Different results may be observed among a different sample of projects.
- Within the project sample, all KPIs may not be captured.
- The CSF and ITSF initiatives in the project sample were frequently jointly funded, or funded in conjunction with other sources of funding including OPs, other internal GAC funding, OGD, provincial, private industry & associations, and/or foreign governments, which complicated outcomes attribution.
KPIs generated by funding source:
The project sample was linked to a total of 1,537 KPIs including: 847 services delivered, 493 outcalls, 164 opportunities identified/referred (OPI & OPR), 22 opportunities pursued (OP) and 11 economic outcomes facilitated (EOF).
Key observations from the project sample based on average number of KPIs generated by funding source type (see Table 2):
- Cost effectiveness: On average, the CSF used alone or in conjunction with other funding sources, was more cost-effective in terms of the amount spent on the initiative vs. the number of KPIs generated ($666/KPI).
- Highest average number of KPIs: On average, the CSF used alone or in conjunction with other sources of funding generated a slightly higher average number of KPIs per initiative (20.6) including the highest average number of outcalls per initiative (8.2) and the highest number of opportunities identified/referred per initiative (2.5).
- Success KPIs: Both the CSF and ITSF (used alone or in conjunction with other sources of funding) resulted in approximately the same average number of successes (OPs and EOFs combined) per initiative (0.4-0.5/initiative).
Key observation on leverage (not shown in Table 2): Key events, funded primarily through the ITSF, leveraged the greatest amount of funding from external partners ($423.9K in total). Funds were primarily leveraged from federal or provincial partners (e.g. $250K from Atlantic Canada Opportunities Agency and Industry Canada to support the Farnborough Air Show and $93K from the Government of Alberta for the 2014 Expo Oil and Gas Congress & Exhibition).
|Funding source||# of Initiatives in Sample||Total Cost||Avg. # of Outcalls||Avg. # OPI and OPR||Avg. # Services Delivered||Avg. # OP||Avg. # EOF||Total Avg. # KPIs||Avg. Cost/KPI|
|CSF (including with other sources of funding)||35||$479.8K||8.2||2.5||9.4||0.3||0.2||20.6||$666|
|ITSF (including with other sources of funding)||28||$679.4K||5.5||2.4||10.3||0.3||0.1||18.5||$1,311|
|CSF & ITSF (including with other sources of funding)||16||$277.3K||3.4||0.4||14.4||0.2||0.2||18.6||$930|
Key observations from the project sample based on KPIs generated by type of activity/initiative (see Table 3):
- Outreach, events and core services to clients generated the highest average number of KPIs per initiative (24.7), but at the highest average cost per KPI ($1,289).
- On average, incoming and outgoing trade missions generated the highest average number of OPs (0.6) and EOFs (0.4) per initiative.
- Trade missions were the most cost effective in terms of the average amount spent per KPI generated ($341/KPI). This was followed by seminars / workshops featuring Canadian capabilities ($538/KPI).
Additional details on trade missions:
- While there were fewer outgoing trade missions (6) in the sample than incoming missions (9), outgoing trade missions generated on average 1.5 successes per initiative (9 OPs and EOFs) which was more than double the number of successes generated from incoming trade missions (4 OPs and EOFs).
|Type of activity / initiative||# of Initiatives in Sample||Total Cost||Avg. # of Outcalls||Avg. # OPI and OPR||Avg. # Services Delivered||Avg. # OP||Avg. # EOF||Total Avg. # KPIs||Avg. Cost/KPI|
|Trade show / fair / conference||40||$843.8K||5.7||1.3||13.4||0.3||0.1||20.8||$1,017|
|Trade mission (incoming and outgoing||14||$112.2K||4.7||5.7||12.1||0.6||0.4||23.5||$341|
|Seminar / workshop featuring Canadian capabilities||6||$47.9K||6.5||0.7||7.7||0||0||14.8||$538|
|Outreach, events and core services to clients||11||$350.5K||13.8||2.6||8||0.2||0.1||24.7||$1,289|
Of the 33 successes (OPs and EOFs) linked to the project sample, 11 success stories reported in TRIO2 contained details on the actual or estimated value of the contract or agreement. The total combined actual/estimated value to clients was greater than $40M.
Beyond the performance data examined above, client satisfaction is also an important indicator of performance. Based on the 144 online client feedback surveys linked to the project sample, the satisfaction rate with the overall quality of TCS service delivery was 86%.
Between the two funds, client satisfaction was 6% higher for CSF-funded initiatives than the ITSF. However, with regards to specific client satisfaction drivers (such as helping clients to overcome barriers to overseas business; improve client profile or credibility; improve client knowledge of the competitive environment in a foreign market; gain access to information or intelligence; connect with customers/partners or other contacts; avoid mistakes, delays or costs; and improve business development strategy), the ITSF scored higher than the CSF.
Efficiency and economy
Finding 5: The CSF and ITSF governance structure provides effective guidance and oversight over the use of the funds.
1. The Steering Committee is a representative and inclusive body that engages the right stakeholder groups in decision making at the appropriate level
Chaired by the Director of BTA, the steering committee includes Director-level representatives (or designated appointees) from the sector and geographic bureaus, the regional network, and functional divisions.
BTA reports annually on the performance of the CSF and ITSF to the ICCB - the ultimate trade governance body.
Geographic representatives interviewed indicated that the Steering Committee provides a forum that supports effective consultation on key CSF and ITSF decisions.
2. Roles and responsibilities are clear
All members contribute to key decisions relating to expense eligibility, how to allocate the CSF reserve, and CSF and ITSF reallocations.
The Geographics determine the distribution of CSF resources among the missions in their regions based on a number of factors and considerations including the Canadian Commercial Interests List.
BBD distributes ITSF funds to missions based on a competitive process supported by strategic guidance provided in part through Sector Practice Strategies and private sector advisory boards.
3. Steering Committee members provide good guidance and oversight in the delivery of CSF and ITSF resources
Steering Committee meetings are held 3 to 4 times per year timed to correspond with key milestones.
CSF and ITSF Guidelines and Sector Practices Strategies and Guidelines provide direction on program objectives, allocations and usage criteria.
Ongoing monitoring ensures the funds are used according to guidelines and expense eligibility criteria, and that unused funds are returned and repurposed.
Successful reallocation of returned funds contributes to high burn rates/low lapse rates.
Finding 6: There is general satisfaction with the eligible activities and expenses in the CSF and ITSF guidelines, however, stakeholders conveyed a need for greater access to hospitality.
77% of STC and TC survey respondents are either satisfied or neutral with the eligible activities and expenses in the CSF and ITSF Guidelines and 23% are dissatisfied.
Survey and interview feedback demonstrates dissatisfaction with the following ineligible expenses and activities in the CSF and ITSF Guidelines:
- Lack of access to hospitality funds under the CSF, and limited access under the ITSF
- Overtime expenses for ITSF events
- Limitations to French translation of market reports
- Client travel to market using CSF base funds
Stakeholders agreed that hospitality is an essential cost of doing business, particularly for networking and B2B events and in regions where providing hospitality is the cultural norm (e.g. China, India).
Access to OPs budget for hospitality at missions is not guaranteed or sufficient to support the CSF and ITSF for the following reasons:
- OPs budgets for hospitality are subject to ceilings
- competition for the OPs hospitality budget between C-E sections and other players (Head of Mission (HOM), Foreign Policy Diplomacy Service (FPDS)) for non-trade related expenses
- access to the OPs budget for hospitality requires HOM approval. Some HOMs view OPs hospitality funds solely for local networking and not for larger trade events/activities.
Similar limitations and ceilings on OGD and partner hospitality budgets limits partner’s ability to cost-share.
For these reasons, stakeholders conveyed the view that the CSF and ITSF warrant dedicated hospitality resources. Stakeholders proposed granting missions the discretion to use hospitality within specified limits and conditions as permitted under the following GAC trade programs:
- North American Partnership Program (NAPP) Base and Pooled funds: hospitality is permitted, but cannot be the only cost within a project.
- Export USA/Mexico: hospitality is permitted to a limit of 20% of the total cost of an initiative, and is limited to basic refreshments (i.e. no meals or alcohol).
- EduCanada: hospitality is eligible in support of education promotion activities, however, cost-sharing with mission budgets is strongly advised.
Events can take place outside of regular working hours thereby requiring significant amounts of overtime to fully participate. Lack of eligibility for overtime expenses under the ITSF means that missions may have to reconsider participating in certain components of events if they involve overtime which may result in missed opportunities to serve clients.
Finding 7: Strategic guidance provided for the ITSF is clear, however, operational guidance was found to be less clear.
1. Demonstrated strategic and operational guidance provided by BBD
Analysis and sector priorities documents are provided to posts to coincide with the launch of the Strategia commercial planning process to help STCs prepare their ITSF applications. In 2016-2017, new guidance was provided on the creative industries/arts & culture sector, an e-commerce guide for consumer products, and a detailed sub-sector guide on the digital media/video-game industry.
The Trade Sectors Wiki contains tactical guidance, fact sheets and detailed background on all of the TCS’s priority sectors, as well as organizational charts and contact information for all sectors to encourage missions to proactively reach out to the sector practices for guidance on their ITSF proposals.
2. STC and TC perceptions on ITSF strategic guidance
93% of survey respondents are satisfied or neutral with the strategic guidance provided by BBD.
Satisfaction with the strategic guidance provided by BBD appears to have improved since 2014 when only 70% of STC and TC respondents to the 2014 Sector Practice Approach (SPA)Footnote 3 Survey agreed or were neutral that the Sector Practices provided timely and meaningful sector intelligence and information.
3. STC and TC perceptions on ITSF operational guidance
Despite noted improvements in project approval rates (from 59% in 2013-2014 to 67% in 2015-2016), survey and interview feedback from STCs and TCs indicates different levels of satisfaction with the operational guidance and advice provided by sector teams and officers in the following areas:
- lack of clear benchmarks and scoring criteria which contributes to a perception that decision making is arbitrary and subjective;
- levels of transparency and feedback provided, in particular for rejected proposals and partial funding, differs among sector officers;
- 21% of STC and TC survey respondents do not agree that ITSF funding decisions are transparent
- different application of rules concerning hospitality and travel;
- inconsistent approaches among sector teams and officers concerning the repurposing of funds in response to changes to initiatives;
- some sectors/officers are flexible when changes to initiatives occur (e.g. change of client or location), while other sectors/officers are less flexible and changes require that a post re-compete for funding.
These issues were previously flagged in the 2014 SPA survey.
Finding 8: High burn rates demonstrate efficient financial monitoring and reallocation systems.
1. High burn rates / low lapse rates
As shown in Figure 3, for the last 3 years, CSF and ITSF burn rates were between 95% and 98.8%.
Figure 3: CSF and ITSF Burn Rates, 2013-14 to 2015-16
2. Effective financial monitoring and reallocation systems
BTA and other colleagues at HQ continually monitor the status of CSF and ITSF initiatives, and conduct on-going follow-up with missions and ROs on funds utilization.
For CSF, missions are asked to move uncommitted funds to a hard commitment. For ITSF, BTA committed the funds to hard commitments for all approved initiatives to help reduce the administrative burden on missions, and to better track funding to results.
Two annual re-allocation phases take place to ensure that unused and returned funds are repurposed toward new initiatives.
Phase 1 in Q3 (October - December)
Phase 2 in Q4 (January – March)
3. Year-end accountability for all CSF and ITSF expenditures
BTA provides detailed, year-end accounts of all expenditures on the departmental Intranet site and presents annual highlights to the ICCB.
Finding 9: Lack of stable funding, and late release of funds to missions, impacts ITSF program delivery.
1. Timeliness of ITSF approvals and disbursements
Survey feedback demonstrates overall satisfaction (70% agree and 25% are neutral) with the timeliness of the ITSF adjudication process (survey question D2(iii)):
70% of respondents agree with the timeliness of the ITSF adjudication process, 25% are neutral and 5% disagree
70% of respondents agree with the timeliness of the ITSF adjudication process, 25% are neutral and 5% disagree
However, satisfaction with the timeliness of ITSF allocations to posts is lower (survey question D02(vi)):
47% agree with the timeliness of ITSF allocations to posts, 29% are neutral and 24% disagree.
47% agree with the timeliness of ITSF allocations to posts, 29% are neutral and 24% disagree.
2. Lack of stable ITSF budget
Each year, CSF and ITSF budgets are subject to reductions if other BFM branch pressures, such as salary deficits, arise. For example, in 2013-14 and 2014-15, initial CSF and ITSF budgets were reduced by 5% and 10%.
For the ITSF, this resulted in reduced allocations to posts below amounts notionally approved in March. Partial allocations is one of the main reasons cited by posts for cancelled ITSF initiatives and returned funds.
In addition to reductions at the start of the FY in 2013-14 and 2014-15, in 2016-17, ITSF funds returned by missions were frozen from reallocation by BFM from April to October. Uncertainty regarding the Phase 1 reallocation process impeded planning at HQ and posts.
3. Timing of ITSF disbursements to posts
The CSF and ITSF budgets are generally communicated by BFM to TCS – Operations (BTD) and BBD in April. BTA reconciles the funding decisions on ITSF in February-March against sector/function budget allotments. Upon receipt of the budget confirmation, BTA prepares financial coding for each mission. CSF and ITSF allocations are usually sent to missions and ROs in May. Funds can be accessed early in the first quarter (Q1) upon request.
The timing of the release of ITSF funds to posts in late May impacts on ITSF program delivery in the following ways:
- Discourages STCs from submitting ITSF proposals for spring/Q1 events due to the need to fund ITSF events from other budgets which leads to complicated accounting problems. This may result in missed opportunities.
- Travel and event related fees cannot be paid for as far in advance to obtain the lowest prices.
- Foreign currency fluctuations can result in insufficient funds by the time funds are received at mission.
- Uncertainties relating to actual funding impacts on posts’ ability to leverage partner funding, especially for spring/Q1 events.
Finding 10: Organizational restructuring has led to administrative efficiencies, however, the cost of administering the ITSF remains high.
1. Organizational restructuring and adoption of financial management practices
Amalgamation of the GVC, CDIA, CSR and EduCanada funds with the ITSF has resulted in more streamlined financial, submission and reporting processes, reducing the administrative burden on missions and HQ.
The funding guidelines of the GVC and CDIA funds are now the same as the CSF and ITSF, reducing the number of guidelines and rules that officers have to follow.
The Funding Unit creates individual fund commitments (COs) for each ITSF initiative, a practice that has reduced the administrative burden on missions and that is highly appreciated by officers.
For the last two years, missions received 100% of their allocations at one time, as opposed to receiving it in two tranches.
2. Level of effort invested in the annual Strategia commercial planning process
Despite administrative efficiencies gained, interview and survey respondents indicated that the number of transactions required to manage the ITSF (e.g. proposal development and submissions, adjudication, cash management, cancellations, returned funds, and reallocations) contributes to high administrative costs for missions, Common Service Delivery Points (CSDPs), and HQ, which is not necessarily commensurate with the amount of funds received.
While the ITSF approval rate for projects submitted as part of the annual Strategia commercial planning process has increased from 59% in 2013-14 to 67% in 2015-16, when cancellations are factored in, the time and effort involved in this process results in projects being implemented only half the time.
Figure 4. Number of ITSFFootnote 4 proposals submitted, approved, cancelled and added, 2013-14 to 2015-16
Additionally, as noted in Finding 9, ITSF funds for approved projects are disbursed to posts in late May, requiring missions to “cash manage” to pay for ITSF events taking place in the first quarter using funds from other budgets. Cash management creates complicated accounting problems and high transaction costs because funds have to be reallocated to the appropriate budgets later in the fiscal year. Further, the replacement of in-house financial staff with CSDPs has placed the administrative burden for all financial transactions previously done by the accounting section on STCs, TCs and Trade Commissioner Assistants (TCAs).
3. ITSF cancellations and returned funds
Of the approved projects, roughly 20% are cancelled and funds returned for repurposing in the third and fourth quarters. The reasons cited for ITSF project cancellations and returned funds include:
- lack of participation for an event/trade show
- initiatives completed under budget
- partial funding below amounts initially requested and/or notionally approved which can render the funds unusable if the mission is unable to revise the scope of the initiative, or find other sources of funding to cover the ITSF deficit
- changes to the initiative
While the reallocation process has contributed to low lapse rates of ITSF funds, personnel at both HQ and missions have noted that the process of returning funds, resubmitting proposals, adjudication and re-disbursement entails significant transaction costs.
Further, roughly half of ITSF annual expenditures occur in the fourth quarter which imposes significant administrative burden on personnel at post and HQ. Stakeholders indicated the need to spread expenditures and activities more evenly throughout the year.
The CSF, unlike the ITSF, provides STCs with the flexibility and discretion to repurpose funds to respond to changes to initiatives or cancellations. For example, as illustrated in Figure 5, returns of CSF base funds over the last 3 years were 87% less ($332.7K) than the ITSF returns which totalled $2.07M.
Figure 5. CSFFootnote 5 and ITSFFootnote 5 Funds Returned by Missions, 2013-14 to 2015-16
STCs have noted that the lack of flexibility to repurpose ITSF funds to respond to changes to initiatives does not reflect the planning cycle since initiatives can change between February, when the Strategia commercial planning process takes place, and the actual execution of initiatives which can be up to a year later.
Finding 11: General agreement that the existing KPIs accurately capture the outcomes of CSF and ITSF funded activities and initiatives, however, there is room for improvement.
1. General satisfaction with KPIs
In response to the survey question “Existing performance indicators (KPIs) accurately capture the outcomes of CSF and ITSF activities/initiatives/projects”, 82% of survey respondents conveyed satisfaction with the KPIs (only 18% disagreed).
However, survey feedback and stakeholders interviewed at post and HQ identified the following concerns and challenges with regards to using the Trade Commissioner Service (TCS) KPIs, which were not specifically designed to measure the performance of CSF and ITSF initiatives:
- too much focus on quantity and lack of focus on quality
- KPIs don’t measure value, e.g. a $1M dollar economic outcome facilitated (EOF) is weighted the same as one worth $10K
- KPIs do not reflect the level of effort or resources invested
- KPIs incentivize/drive short-term transactional behaviours rather than long-term ones
- officers have little control over ultimate outcomes (OPs and EOFs)
- KPIs do not cover the full range of TCS tasks/work
- frustrations that KPI definitions, especially successes, are too rigid
- KPIs can be manipulated
- results/KPIs can take longer than the current one-year timeframe for reporting and performance measurement, e.g. requirement to close initiatives in Strategia by the end of the fiscal year, before some results, particularly successes, have materialized
- officers may not make the linkages between the KPIs and the initiative and funding source
Finding 12: Lack of interoperability between CSF and ITSF planning and reporting systems creates administrative burden and limits effective performance measurement.
1. System developments since the last evaluation have improved departmental capacity to monitor performance
Strategia, which replaced the Commercial Economic Plan (CEP), is a comprehensive, one-stop, mission planning tool. The TCS uses it to enter their commercial action plans and initiatives, and to report on expenditures, outcomes and KPIs.
TRIO2, originally intended to be a Client Relationship Management (CRM) tool, is now used to track the performance of the TCS based on KPIs. It is also used to record and approve successes (OPs and EOFs). The Case tab, introduced in 2014, has increased interoperability by linking results to key TCS events.
A Business Intelligence (BI) pilot project is currently under development to link trade data from Strategia, the Integrated Management System (IMS), Human Resource Management System (HRMS), and TRIO2. The objective is to better align KPIs and outcomes to trade initiatives, and to measure the return on investment based on the human and financial resources invested. The tool will also be used to inform future ITSF funding decisions. Since FY 2015-2016, COs are created by BTA for each approved ITSF funded initiative to support this project.
2. Despite the noted improvements, challenges still remain
Due to limitations of the Strategia tool, trade officers are required to manually enter CSF and ITSF expenditures (IMS) and performance data (TRIO2) into Strategia. This increases both the administrative burden on posts and introduces a risk that data will be entered inaccurately, evidenced by data discrepancies for both expenditure data and KPIs in Strategia.
Despite KPI data in Strategia, TRIO2 is considered the only authoritative source for validated KPI data.
Outside of the Case tab and BI pilot project, there is limited ability to generate reports that link initiatives and funding sources with KPIs in TRIO2, which limits the ability to compare performance at the initiative/activity and funding levels.
A previous version of the IBD planning tool contained activity, product and service “categories” to extract reports on where CSF funds went, and to facilitate comparison for performance measurement purposes.
The sector bureau, however, has demonstrated the ability to track performance at a sector level to inform decision making
The sector bureau has conducted detailed TRIO2 data analysis to determine the scope and scale of TCS activity within the ICT and creative industries sectors. This was achieved by using TRIO2 performance data for the sector, including all funded and unfunded initiatives/activities.
Information presented in the sector analyses includes:
- Client profile, number of clients served, location of clients served, client organization type (e.g. SME), client data by sub-sectors, etc.
- KPI trends by markets covered, and successes by post and by region. However, KPIs are not disaggregated by funding source(s) or amounts.
- Types of initiatives/activities that lead to successes
In addition, on an annual basis, sector officers review Strategia and KPI performance data in TRIO2 to determine the results achieved from previously funded ITSF activities to inform funding decisions.
Best practice: WSHDC
WSHDC has developed their own performance measurement system using a standardized naming convention, e.g. ICT0 (ICT=sector and 0 = general outreach) that links all their commercial initiatives to KPIs in TRIO2.
KPI descriptions in TRIO2 reference the initiative name, making it simple to link Strategia data (initiatives, funding sources) to TRIO2 data (results/KPIs).
WHSDC plans to use the TRIO2 Case tab for all initiatives, not only key events, to track results and to inform future STC funding decisions.
|CI15||ICT0. Commercial Activities (ICT)||Sector – Information & Communication Technologies (ICT) – Cybersecurity and Wireless|
|CI10377||ICT1. FOSE (Federal Office Systems Expo.)||Sector – Information & Communication Technologies (ICT) – Cybersecurity and Wireless|
Access to discretionary CSF and targeted ITSF funding has become increasingly important for the Commercial-Economic sections of the Department and enhances TCS capacity to support Canadian businesses in their efforts to participate and expand into global markets. Trade initiatives funded in part through the CSF and ITSF support the advancement of Departmental international business development objectives by contributing to the expansion of Canadian business participation in global markets.
Strategic guidance provided by the Trade Sectors Bureau for the ITSF was found to be clear for all of the TCS’s priority sectors, however, there is a need for greater clarity and consistency in the provision of operational guidance across all sectors. With respect to eligibility criteria, stakeholders at missions and HQ expressed a need for greater access to hospitality to support CSF and ITSF- funded initiatives.
Mechanisms are in place to support sound project planning and implementation, and the system of reallocation contributes to low lapse rates. However, uncertainties over the ITSF budget and timing of disbursements to posts adversely impact on program delivery. Further, the transaction costs of administering the ITSF were found to be high at missions, headquarters and CSDPs. Finally, lack of interoperability between planning (Strategia) and reporting (TRIO2) systems increases the administrative burden and limits the ability of the Program to monitor performance and report on results.
Recommendation 1: BTD and BBD review the guidelines and expense eligibility criteria which govern the use of funds for hospitality with the view to making funds more accessible to support CSF and ITSF funded initiatives.
Access to hospitality is deemed critical to the success of initiatives, both under the CSF and ITSF.
According to the guidelines, ITSF funds can be used for hospitality however, the criteria for using the funds for this purpose remain unclear and the requirement to seek Director General level approval may be a barrier to requests for authorization.
Past experience has resulted in a disincentive amongst STCs to submit proposals containing hospitality to BBD which could result in missed opportunities.
Establish clear guidelines and parameters for access to the funds for hospitality purposes and consider delegation of authority to Director level.
Linked to Finding 6
Recommendation 2: BBD adopt a more consistent approach within the branch and divisions in the provision of operational guidance to STCs.
Clear benchmarks and criteria against which ITSF proposals are scored and evaluated are needed in all sectors.
- Example: See EduCanada “How to apply for Education Sector funds”. Criteria are listed by “high impact” and “other factors”.
Consistent feedback and explanations for rejections and partial funding across all sector practices:
- feedback adjusted to the post / market / proposal /sector - avoid repetitive feedback
- constructive feedback that makes it clear to posts whether or not they need to make changes to a proposal
Sector specific “best practices” or “cheat sheets” to illustrate which types of activities work best/deliver results in each sector.
Linked to Finding 7
Recommendation 3: BTD and BBD explore options to increase the predictability of funding, release ITSF funds earlier, and reduce the administrative burden at missions and HQ.
Greater certainty in the amount of funds available for ITSF programming purposes would facilitate planning and leveraging with partners at missions.
Earlier disbursement of ITSF funds would increase expenditures in the first quarter of the fiscal year, and reduce cash management and admin burden at posts for ITSF.
Explore options to address the issues that contribute to high administrative costs at posts and HQ.
Stakeholders interviewed and surveyed proposed five options to improve program delivery and/or reduce the administrative burden associated with the ITSF. See Annex A.
Linked to Findings 9 and 10
Recommendation 4: BTD and BBD, in consultation with the relevant responsibility centres, and aligned with the Departmental Data Strategy, explore options to improve data quality and reporting on CSF and ITSF initiatives and their results to help make informed, evidence-based decisions.
BTD work to streamline expenditure and KPI reporting requirements to a single tool or a common interface to reduce risk for discrepancies/inaccuracies and to strengthen CSF and ITSF performance monitoring and reporting.
BBD and BTD, in consultation with the relevant responsibility centres, work to strengthen results reporting for the ITSF by leveraging existing capacities within Strategia and TRIO2 (e.g. use of the Case tab to link initiatives to results for all ITSF events and the BI Pilot Project).
Examine best practices at missions such as WSHDC that are leading the way.
Linked to Finding 12
Management Response and Action Plan (MRAP)
BTD and BBD review the guidelines and expense eligibility criteria which govern the use of funds for hospitality with the view to making funds more accessible to support CSF and ITSF funded initiatives. (linked to F6)
|Management Response (MR) & Action Plan (M-AP)||Responsibility Centre||Time Frame|
MR: Agree, subject to review of GAC benchmarks (ceilings) on hospitality and establishment of limits and conditions. There are, however, important trade-offs that will need to be considered before determining whether adjustments are appropriate, i.e., whether accessibility to one fund (CSF and ITSF) and less to another (operating budget) will provide an effective and appropriate risk-balanced solution for both HQ and missions abroad.
In consultation SWCT (Trade Financial Management) and approval by the Steering Committee.
BBD adopt a more consistent approach within the branch and divisions in the provision of operational guidance to STCs. (linked to F7)
|Management Response (MR) & Action Plan (M-AP)||Responsibility Centre||Time Frame|
MR: Agree but need to take into consideration the particularities of each sector.
M-AP: Building on the progress that has already been made to improve the timeliness and quality of strategic guidance provided to posts, management will continue to explore new ways to increase coordination across all sector teams to ensure that consistent operational guidance, notably on funding proposal evaluation, and constructive feedback is given to STCs. Management will, for example, review and seek to align processes and evaluation criteria used by the different sectors, where appropriate, to evaluate ITSF requests. Management will also review how information on operational guidance is made available to TCs and STCs to engage in an ongoing open communication.
BTD and BBD explore options to increase the predictability of funding, release ITSF funds earlier, and reduce the administrative burden at missions and HQ. (linked to F9 and F10)
|Management Response (MR) & Action Plan (M-AP)||Responsibility Centre||Time Frame|
MR: Agree. The CSF and ITSF budgets are not fenced, and the program has historically been susceptible to being taxed if there are Branch pressures. As such, there is a trade-off between early release of predictable amounts of money and the actual amount of money up for release that needs to be examined. Management also welcomes any opportunities to reduce the administrative burden at missions and HQ.
M-AP: Management will examine the feasibility of the proposed options (Annex A, Evaluation Report). Management will also review the trade-offs between levels of funding, early funding release and predictability of funding as well as other potential options (e.g. fencing CSF/ITSF). Management will present a proposal to the CSF and ITSF Steering Committee for discussion and approval with a view to implementing in alignment with the 2018/19 planning cycle.
BTD and BBD, in consultation with the relevant responsibility centres, and aligned with the Departmental Data Strategy, explore options to improve data quality and reporting on CSF and ITSF initiatives and their results to help make informed, evidence-based decisions. (linked to F12)
|Management Response (MR) & Action Plan (M-AP)||Responsibility Centre||Time Frame|
MR: Agree, to explore options to improve reporting on CSF and ITSF results, subject to system technical capabilities, availability of resources and costs. To enhance linkages/connections between IMS/TRIO/HRMS/Strategia which would allow the commercial program to better use corporate data sources for evidence based decision making.
M-AP: Discussions are currently underway with the technical development team to identify plausible options to push data to Strategia in order to streamline expenditure and KPI reporting requirements. Further, management agrees to revive and fully implement the objectives of the Business Intelligent (BI) project based on the availability of dedicated resources. Working in consultation with BBD, BTD management will examine best practices (e.g. WSHDC’s approach) for viable elements that may be incorporated in the BI project to better link CSF/ITSF initiatives in Strategia to TRIO data.
- Date Modified: