This report presents the results of an evaluation of the Client Service Fund (CSF) conducted by the Evaluation Division (ZIE) - Office of the Inspector General (ZID). The study was commissioned by the Policy and Partnership Division - International Business (TBX), International Trade Canada (ITCan). The evaluation was formative, reporting on the relevance, results achievement/success and effectiveness of the CSF after its first three years of operation, and contributing to its ongoing efforts of continuous improvement.
The CSF was launched in April 2002 subsequent to a review of the Program for International Business Development (PIBD). It was designed to be the Department's principal fund for supporting proactive initiatives undertaken through Trade Commissioner Service (TCS) to assist Canadian business and partner clients to compete globally, and to influence foreign decision-makers to select Canada as a preferred source of goods and services. It was intended to enable Trade Program Managers (TPMs) and Commercial Officers (COs) to develop business contacts, test the market, identify agents and distributors, acquire Market Information and Intelligence (MII) and broaden the awareness of Canadian capability in foreign markets. The Fund was established to contribute to the overall objectives of the International Business Development (IBD) Branch.
The primary purpose of this formative evaluation was to determine the extent to which the CSF is making progress toward meeting its strategic and operational objectives through an assessment of short- and medium-term expected outcome achievement since its inception in April 2002. In addition, it also examined evaluation issues related to program relevance and cost-effectiveness. The methodology involved relatively distinct data collection activities which included: the extraction and analysis of existing performance data from the IBD Planning and Reporting system; face to face interviews with individuals from TBX, and ITCan personnel representing Geographic Divisions, Investment Partnership Branch (IPB), Science and Technology (TBR) and Market Research Centre (TMR); and, on-line survey of trade program managers.
The CSF has remained highly relevant to the Government of Canada (GOC) IBD objectives and priorities. Its policy guidance continues to be updated to remain current with changes in strategic priorities and key results. Nevertheless, the realignment of the Department's organisational structure and Strategic Framework warrants further articulation and emphasis of the role of 'two-way international investment' and 'science and technology linkages' as critically important IBD programming areas. Advancing this international commerce approach is a Corporate issue that requires policy guidance and emphasis in the IBD business plans. While the CSF continues to meet the needs and expectations of its Canadian business and partner clients, its allocations are not well aligned with the Canadian Commercial Interests List (CCIL) priority markets and Trade Team Canada Sectors (TTCS). In both of these cases, Corporate guidance is required on their use as policy instruments to bring about stronger alignment between IBD plans, including CSF initiatives, and priority markets and sectors. The level of CSF funding remains significant and meaningful for Commercial-Economic Sections (C-E Sections) in Posts located in relatively high priority markets (e.g., 2003-2004 CCIL top 23 markets) however, funding to small C-E Sections in lower priority markets is no longer adequate for meaningful proactive IBD programming.
CSF initiatives generate a wide range of products and services to respond to clients needs for either trade promotion, direct sales support or MII. However, the lack of funding for official language translation of market information products has become an obstacle to Post/C-E Sections wanting to transition from trade promotion to providing MII services to Canadian business and partner clients.
The achievement of short-term outcomes for Canadian business clients as a result of the use of these products and services has remained quite high and relatively unchanged in terms of: overall awareness and appreciation of TCS services; awareness and knowledge of market opportunities; and, expanded network of contacts. Business client satisfaction with respect to the accuracy of key contact lists and the reliability of local company information was somewhat down in 2004. A valid and reliable determination on the increased awareness for foreign contacts could not be made, similarly for enhanced guidance and support by Canadian partner clients due to a lack of available data.
Medium-term outcome achievement for business clients has seen a significant improvement in almost all areas since the launch of the CSF. The perception of Canadian business clients is that the TCS/CSF services can be attributed with positively influencing or contributing to various aspects of their business. Thirty-six percent (36%) of business clients attributed their financial results averaging $7.6 million each over the past 12 months to the TCS/CSF services. On the other hand, the perception of Canadian partner clients regarding the positive contribution of the TCS/CSF to their business initiatives has continued on a downward trend since the 2002 Client Survey, especially with respect to saving time and money, market access and opportunities, making better decisions, finding the right people and business image/credibility.
While the CSF has not had any significant unintended negative consequences, it has had a very positive influence on IBD programme management practices, especially in terms of strategic planning, financial management and results-based management.
The CSF is a cost-effective programme and represents good value for the resources invested as evidenced by the leveraged revenues, high client satisfaction levels, as well as the short-term financial results achieved by business clients. Further cost-efficiencies could be realised by fully integrating other ITCan programmes and sources of funding (e.g., Investment Partnerships, Science and Technology, Enhance Representation Initiative (ERI) and Branding Canada) into the same management structure and processes created for IBD programming by the CSF.
It is widely recognised that TPMs/C-E Sections select a large number of priority sectors for IBD programming based on local market demands with little reference to TTCS sectors. While the TBX/CSF Unit has created an on-line IBD Planning and Reporting system with considerable potential, the annual revisions to the Strategy, Action Plan and Reporting templates are not completed in a sufficiently timely manner to meet the needs of Post/C-E Sections. There is little indication that the captured performance data in the Reporting template is being analysed and the information used to identify areas for client service improvement. The biannual Client Surveys were the only source of valid and reliable data available on outcome achievement for this evaluation.
In conclusion, the usefulness of the IBD Planning and Reporting system for tracking and reporting on outcomes accrued to Canadian business and partner clients will remain very limited until additional on-line reporting templates for this purpose are developed.
The following recommendations address issues that are beyond the scope of the CSF. They are presented for consideration by ITCan management.
The following recommendations address issues that are within the scope of the CSF management. They are presented for consideration by TBX.
There were essentially two types of recommendations that arose from the evaluation of the Client Service Fund:
Following is a summary of the key themes/issues that emerged and the recommended action:
TBX management responses to the specific recommendations are incorporated in Section 6 of this Evaluation Report.
The Evaluation Division (ZIE) from the Office of the Inspector General (ZID) was commissioned by the Policy and Partnership Division, International Business (TBX) - International Trade Canada (ITCan) to conduct a formative evaluation of the Client Service Fund (CSF). The result of this evaluation is intended to serve as an important element in the ongoing strategic development of the Client Service Fund.
The primary purpose of this formative evaluation was to determine the extent to which the CSF is making progress toward meeting its strategic and operational objectives through an assessment of short- and medium-term expected outcome achievement since its inception in April 2002. In addition, it also examined evaluation issues related to program relevance and cost-effectiveness. The following definitions of these three evaluation standards are provided to ensure a common understanding among all of those concerned.
Relevance: The extent to which the CSF is consistent with departmental and government-wide international trade priorities and whether it realistically addresses the actual needs of its clients. Specific sub-issues included consistency and complementarity with the international trade interests of its Canadian partner clients and co-deliverers.
Results Achievement/Success: The extent to which the CSF is effective in meeting its objectives, within budget and without unwanted outcomes. Specific sub-issues included the attribution of outcomes to the activities of the program, as well as the identification of unintended consequences and repercussions, both positive and negative, of program implementation.
Cost-Effectiveness: The extent to which the CSF uses the most appropriate and efficient means to achieve the expected results, relative to alternative design and delivery approaches. Specific sub-issues included whether resources were allocated to initiatives that fully met the eligibility criteria and were most likely to yield the best results.
The client for this formative evaluation is TBX, whose interests in this evaluation are of paramount importance, given its strong formative or learning orientation. As such, a participatory evaluation process was employed to ensure maximum flexibility to address priority evaluation issues of interest.
There were five relatively distinct data collection activities undertaken to implement this formative evaluation. The integrity of the following sequence was maintained in order to optimise the "snowball" effect of the methodology, which resulted in an increasingly better understanding and a sharper focus on the evaluation issues of importance to the client. The completion of each data collection activity informed the implementation of the next, refining and opening new lines of enquiry in an interactive process of discovery and learning.
The evaluation work plan defined key evaluation issues to be addressed.
All available documentation related to the CSF were reviewed including: Results-Based Management Framework (RMAF); Guidelines; 2002-2003 Report Analysis; various surveys (e.g., Employee Survey and Client Survey); as well as other relevant ITCan documentation available on the Horizons web site.
Since the 2002-2003 fiscal year (FY), a considerable amount of tombstone and performance data on CSF initiatives were collected from Trade Program Managers (TPMs) through an on-line reporting database available on the Horizons web site referred to as the IBD Planning and Reporting system. While some analysis was previously conducted using the 2002-2003 Reporting data, TBX/CSF Unit staff participated in the analysis of the 2003-2004 Reporting data. Much of this data was used to assess the types of CSF products and services and, particularly, TPM's perceptions of outcome achievement. The findings were triangulated with other data sources containing similar performance data (e.g., client survey and employee survey), wherever possible.
The CSF is an integral part of the services delivered by the Trade Commissioner Services (TCS), in particular the C-E Sections at Posts, and cannot be realistically distinguished from the New Approach@Work (NA@W) six core services which is described in section 2.9 of this report. Consequently, the TCS Client Survey findings are highly relevant to the CSF, since many of the Canadian businesses and partners interviewed were involved in a CSF initiative, although they may not have been aware of the provenance of the funding which supported their particular activity. As such, the findings of the 2004 Client Survey(1) provided valid and reliable indicator data on CSF results achievement, especially at the short- and medium-term outcome levels, while the 2002 Client Survey data was used as a baseline reference.
The 2003 Employee Survey was also valuable secondary data source that was used to assess a number of important evaluation issues. While specific questions were asked in the survey questionnaire related to the CSF it also contained relevant data on the staff workload at Posts, thus contributing to the assessment of cost-effectiveness.
Finally the ITCan Integrated Management System (Systems Analysis and Program Development (SAP)-Financial Module) was used to extract relevant financial data on the CSF as a whole, at Bureaus and initiative levels. This data was used to assess the "cost" part of the cost-effectiveness equation, and also shed some light on the efficiency of CSF financial management.
Face-to-face interviews were conducted with some of the staff involved in the initial design and launch of the Fund, as well as those who are currently assigned to the TBX/CSF Unit. Each interview was customised to the individual, given their varying roles and involvement in the design, launch and subsequent management of the Fund.
The purpose of the on-line survey was to solicit the views of TPMs regarding specific evaluation issues on the relevance, results achievement and cost-effectiveness which were not addressed in the past. An invitation was issued to 135 TPMs by the TBX/CSF Unit to complete the on-line survey and 42 were completed, for an acceptable response rate of 31%. It should be noted that the sample was not adjusted for geographic balance, although the data was nevertheless analysed by Geographic Bureau in order to draw some comparisons.
|Africa and Middle East Regional Bureau (AMEB)||10||23.8%|
|Caribbean and Latin America Regional Bureau||8||19.0%|
|North America Regional Bureau (NA)||6||14.3%|
|Asia-Pacific Regional Bureau (A-P)||2||4.8%|
|Europe Regional Bureau (EU)||11||26.2%|
Subsequent to the analysis of the survey data, 17 follow-up telephone interviews were conducted with TPMs who had completed the survey questionnaire and volunteered to participate, representing 40% of survey respondents. The purpose of the telephone interviews was to explore the underlying reasons for opinions and judgements made in response to the survey questions. It permitted to validate the survey responses and further examine some evaluation issues raised.
Face-to-face interviews were conducted with ITCan Geographic Division and Functional Unit staff. Geographic Divisions manage CSF reserve funds to be used for CSF initiatives proposed by Posts during the Fiscal year, or to support regional type CSF initiatives. Ten interviews were held with at least one key contact from every Geographic Bureau. In light of the recent departmental restructuring, it was important to tap into the views and opinions of ITCan personnel from other branches and divisions (e.g., Investment Partnership Branch (IPB), Science and Technology Division (TBR), Market Research Centre (TMR)) to determine if unintended positive or negative results or consequences of the CSF have affected them.
As for most evaluations, there were constraints that have influenced the methodology and conduct of this evaluation. It should be noted that the methodology did not include focus group sessions with Geographic Division staff nor telephone interviews with Canadian business clients, as stipulated in the Terms of Reference. The reason for this departure is two-fold, first the findings of the 2004 Client Survey were considered highly relevant as an assessment of the CSF in terms of results achievement, obviating the need to collect additional primary data directly from clients. The second reason lies in the importance placed by the TBX/CSF Unit on the views and opinions of TPMs and a select number of other ITCan Branches and Divisions.
Therefore, some of the constraints to the implementation of this formative evaluation included:
In 1995, the Department of Foreign Affairs and International Trade (DFAIT) established the Program for International Business Development (PIBD) through the consolidation of a variety of formerly separate trade, investment and science and technology promotion programs, such as, the Program for Export Market Development/ government-initiated activities, the World Market Trade Development Program, the Post-Initiated Promotional Projects for the US, the Access North America Program, the National Trade Strategy, Going Global, the Investment Development Program, and the Technology Inflow Program. Subsequent to a program review undertaken in FY 2001-2002, the PIBD was replaced by the newly created CSF, which was launched in April 2002.
The CSF was designed to be the Department's principal fund for supporting proactive initiatives undertaken primarily overseas (i.e., borders-out programming) through the TCS to assist Canadian industry to compete globally and to influence foreign decision-makers to select Canada as a preferred source of goods and services. It was intended to enable Commercial Officers (COs) to develop business contacts, test the market, identify agents and distributors, acquire market information and intelligence (MII) and contribute to increased awareness of Canadian capability in foreign markets. The Fund was established to contribute to the overall objectives of DFAIT's IBD Branch.
Since the launch of the CSF, the DFAIT was divided into two separate departments: Foreign Affairs Canada (FAC) and International Trade Canada (ITCan). Created in December 2003, ITCan brings together specialists in trade, investment and international business development. International Business Development, including the TCS's CSF, is one of this new department's four business lines, along with Trade Policy, Investment Promotion and Corporate Services.
DFAIT had the mandate to foster the expansion of Canada's international trade, as set out in the Department of Foreign Affairs and International Trade Act, RSC 1985, c. E-22. The purpose of the IBD Business Line (2) was then:
As part of the IBD business line, the CSF was to provide funding to Canadian missions abroad, through the TCS, to carry out a range of intelligence-gathering and promotional activities in support of Canadian exports of goods and services, foreign investment inflow and science and technology development. To ensure that the CSF remained relevant over time, TBX was to monitor changes in Canada's Foreign Policy, the Departmental Planning, Reporting and Accountability Structure and the T-Branch Business Plan so as to periodically adjust the CSF objectives and scope of activities accordingly.
The CSF strategic objective established a clear up-link with the purpose of the IBD business line and oriented the long-term performance expectations of the Fund:
Strategic Objective: To assist Canadian businesses in expanding or diversifying export sales, attracting foreign investors and pursuing science and technology development in an increasingly competitive global marketplace.
The operational objective articulated the specific purpose of the Fund and oriented its medium-term performance expectations:
Operational Objective: To contribute to the establishment or strengthening of trade and investment-oriented business relationships among Canadian businesses, institutional partners and their overseas counterparts by providing timely, consistent, and high value-added services throughout the Trade Commissioner Service.
The results-based logic model featured in Exhibit 1 was used to graphically illustrate the internal logic of the Fund. This internal logic was based on a sequence of cause and effect relationships perceived to exist between the investment of resources in selected CSF initiatives, the subsequent production of outputs, and the achievement of short-term outcomes that were viewed as a logical consequence of output use by Canadian business and partner clients. This chain of causality from CSF initiative outputs to key result commitments (i.e., short-term, medium-term and long-term outcomes), served as a program delivery model.
Short-term outcomes were to consist of immediate benefits accrued to Canadian business and partner clients in terms of expected changes in knowledge, attitudes and perceptions derived from the utilization of products and services delivered by CSF initiatives.
Short-term outcomes for Canadian businesses were to consist of:
Enhanced awareness and appreciation by Canadian business groups regarding access to, range and quality of TCS services, including:
Improved awareness and knowledge of foreign business environment for trade, investment and S&T development opportunities in existing and new markets, including:
Expanded network of Canadian clients' key contacts in foreign markets, including:
Short-term outcomes for foreign contacts were to consist of:
Increased awareness and knowledge of Canadian business environment for trade, investment and S&T development opportunities, including:
Short-term outcomes for Canadian partner clients were to consist of:
Enhanced capacity of partners and co-deliverers to provide guidance and support to Canadian businesses, including:
Medium-term outcomes were to consist of behaviour changes in the clients and partners as a result of their use of the products and services delivered by the program.
Medium-term outcomes for Canadian businesses were to consist of:
Increased number of Canadian businesses better positioned to identify and explore opportunities to enter foreign markets, including
Increased number of Canadian businesses better positioned to sign sales agreements and joint ventures with foreign contacts, including:
Medium-term outcomes for foreign contacts were to consist of:
Increased interest by foreign contacts in investing in Canadian businesses, as well as purchasing Canadian goods and services, including:
Long-term outcomes are the end results, or consequences of the effects of all CSF initiatives on the Canadian economy. They were to consist of macro-level changes such as:
Increased Canadian market share and diversification of world exports by product and destination, including:
Increased Canadian market share of foreign direct investment by country, including:
|Resources||Activities/ Outputs||Reach||Short-Term Outcomes (0 -1 year)||Medium-Term Outcomes (1-3 years)||Long-Term Outcomes (beyond 3 years )|
The total resources available to the CSF vary from year to year. Allocations to the Geographic Bureaus will be determined by an allocation formula agreed upon each year by the RAC.
|Complementary to NA@W core services:|
- business/liaison meetings and events attended
- business, trade, investment, and S&T leads forwarded to Canadian clients
- business development and promotional events sponsored or organised
- media and marketing initiatives completed
- coordination and integration of international business support services of partners
Team Canada Inc.
Other federal departments/ agencies
The CSF was designed as a decentralized funding mechanism that posed special management challenges, both within the ITCan Headquarters geographic and functional bureaus, as well as with missions abroad and C-E Sections within them. A summary of the intended role and responsibilities of the various departmental entities is provided below.
The Assistant Deputy Minister (ADM), International Business Development and Chief Trade Commissioner is accountable to ITCan's Executive Committee and Parliament for the utilization of the CSF, and the achievement of results. In consultation with the Resource Allocation Committee, the ADM establishes the strategic orientation, priorities and key results commitments for the CSF, approves for dissemination the CSF Annual Performance Report, and approves annual financial allocations to geographic branches and the International Cultural Relations Bureau.
The Resource Allocation Committee (RAC) is a departmental committee composed of directors general from the eight geographic bureaus and the International Cultural Relations Bureau, and co-chaired by the two directors general of the IBD business line. TBX acts as the Secretariat for the RAC. The RAC's principal responsibility is to provide the ADM International Business Development with policy advice and strategic guidance. The specific responsibilities of the RAC are:
TBX is responsible for establishing the general guidelines for the utilization of the CSF, including the formulation and implementation of policy framework changes, strategic orientation and future performance reporting. In consultation with geographic and functional divisions, TBX manages the design and implementation of procedures, practices and processes for CSF initiatives, from the initial stage of developing mission CSF work plans through to performance measurement and reporting. TBX is to act as the Secretariat for the RAC, be involved in the monitoring and evaluation of on-going performance of the CSF, and the preparation of the Annual Performance Report.
This Committee was to be created following the roll-out of the CSF in April 2002 and regroup the responsibilities of the PIBD Working Group, PIBD Budget Reporting and Area Management Unit System (TAMS) Committee. Its role and responsibilities remain to be determined in detail.
A CSF Unit was created following the roll-out of the CSF. It is responsible for policy and planning, as well as day-to-day management issues arising from the RAC or related to queries on the use of the Fund. It establishes general guidelines for the utilisation of the Fund, chairs the CSF Management and Reporting Committee, liaises with the geographic divisions, provides training and support, manages operational aspects including information management systems and coordinates financial reporting with TAMS and the geographic bureaus. In addition, the Unit is responsible for compiling and analysing performance information to produce the Annual CSF Performance Report.
The Geographic Bureaus are responsible for CSF allocations to the missions abroad. They establish global, regional and/or sectoral priorities, approve annual international business development strategy and action plans, review the CSF work plans submitted by missions under their responsibility, recommend the amount of CSF funds to be allocated to individual missions abroad at the beginning of the fiscal year, and make any adjustments that may be required to mission allocations up to the third quarter of the fiscal year. They also consult with key Canadian partner clients involved in IBD, including OGD and provincial and territorial governments to ensure alignment and coordination of market development strategies.
The C-E Sections located in FAC missions abroad are the key delivery agents of international business development programming. TPMs and COs are responsible for the analysis of economic and business market conditions, local needs and opportunities, and the preparation of the Mission's annual international business development plan. They are the front-line decision-makers and the managers of CSF initiatives from the establishment of the overall IBD Strategy and Action Plan, the design of CSF initiatives through to their implementation and achievement of results. Their specific responsibilities are:
The Area Management Office (TAM) for ITCan is responsible for the transfer of financial resources to geographic and functional bureaus to missions abroad, and overall financial management and reporting. At the beginning of each fiscal year, TAM will transfer directly to missions abroad the CSF funding allocated to the C-E Section on the basis of recommendations formulated by geographic bureaus and adjustments required by the previous year's performance results.
In line with other departmental budgetary constraints and cutbacks, the annual amount set aside for the PIBD had decreased significantly, from $34 million in 1994/1995 to $9.5 million in 2000-2001. In FY 2001-2002 the allocation to geographic and functional bureaus again decreased to $8.7 million, excluding the allocation for training. At the time of its launch in April 2002, it was expected that CSF funding would stabilize at $8.7 million annually over the subsequent five years. However, the FY 2004-2005 budget allocation was again reduced to $6.3 million.
The CSF was not intended to replace or augment the reference-level operations, maintenance and procurement budgets allocated to geographic bureaus/divisions or C-E Sections in missions abroad. It was therefore not meant to be utilized for:
Canadian business operators, particularly small- and medium-sized enterprises (SMEs), are the drivers of economic growth. Therefore, the success of the CSF was predicated, to a large extent, on the ability to reach Canadian business clients and to influence business decisions which ultimately lead to the creation of employment and prosperity in Canada. Consequently, the primary targeted clients of the CSF are described in the box below.
Canadian business clients were to include:
The overseas counterparts or foreign contacts of Canadian businesses were also considered to be targeted clients to the extent that they were potential foreign innovators, investors and buyers of Canadian products in both the private and public sectors.
Foreign clients were to include:
Canadian business and foreign clients are the primary beneficiaries of the products and services delivered under the CSF as they were capable of generating downstream outcomes that were expected to occur as a result of CSF initiatives (e.g., investments in productive capital, sales, exports, and creation of employment).
Other groups to be reached in the context of the CSF were co-delivery agents or partner clients involved in influencing the primary target clients to adopt or change their behaviour. Such organizations were viewed as partner clients of the CSF, as they assisted and/or complemented ITCan's efforts to stimulate changes among Canadian business clients.
A key Canadian co-delivery partner is Team Canada Inc., the partnership of 23 federal government departments and agencies involved in international business development.
Team Canada Inc. were to include:
Other partners were to include:
CSF funds were to be allocated by the RAC in February/March of each year to global and/or regional business promotion opportunities, as well as to geographic branches and the International Cultural Relations Bureau. In the past, the actual transfer of funds to geographic and functional bureaus was staggered throughout the fiscal year. Effective April 1st, 2002, CSF allocations were to be transferred by TBX directly to missions abroad in a lump sum at the beginning of the fiscal year. In subsequent years, the precise financial allocation formula was to be adjusted annually by the RAC based on mission performance. Allocations to C-E Sections in missions abroad was to be determined by geographic managers at Headquarters based on past performance.
The CSF was intended to provide C-E Sections with the required flexibility to respond quickly to client needs and/or rapidly changing market conditions. Hence decision-making authority for the use of CSF funds was delegated to front-line Trade Program Managers in missions abroad. The extension of such decision-making authority had to be accompanied by an appropriate accountability for diligence in the management of the allocated funds, results achievement and performance reporting.
The Trade Commissioner Service (TCS) is the backbone of Canada's international business strategy in the global marketplace. It is a 100-year old network of more than 500 trade professionals located in over 130 offices abroad who help the Canadian private and public sectors to take full advantage of international business opportunities.
The role of the TCS is to promote Canada's economic interests in global markets. COs facilitate a wide range of business activities including, but not limited to: export market development; foreign market access; trade policy; investment in Canada; international business contacts; foreign business leads and strategic alliances; licensing and franchising; joint-venture partnering and subsidiary development; and, technology transfer.
The TCS introduced the NA@W six core services in 1999-2000 to Canadian business clients and to non-private sector partner-clients around the world.
The six core services include:
In addition to these core services, COs engage in a number of strategic activities to assist Canadian industry to compete globally and to influence foreign decision-makers to choose Canada as a business partner. These initiatives are typically custom-tailored to the conditions and trends of the local business environment and to the specific needs of the primary Canadian business clients who are interested or active in that market. The CSF was designed in conjunction with the NA@W six core services to support COs in undertaking these and other proactive activities when additional funding was required.
This chapter represents the findings on the relevance of the Client Service Fund.
In its design and planning, the CSF was closely linked to the mandate of DFAIT's IBD Business Line as stated in its 2001-2002 Business Plan. The long term performance expectations of the CSF were to be oriented by its strategic objective, as follows:
CSF Strategic Objective: To assist Canadian businesses in expanding or diversifying export sales, attracting foreign investors and pursuing science and technology development in an increasingly competitive global marketplace.
In 2002-2003, the DFAIT Executive Committee developed a Strategic Planning and Priorities Framework which was to set the direction for the development of 2003-2004 Business Plans. The framework contained five strategic objectives and corresponding key results, as well as twelve Departmental priorities. An overview of this framework was included in the 'Reference' section of the IBD Business Planning and Reporting Templates for FY 2003-2004. Policy guidance was provided to C-E Sections on how to identify strategic objectives and priorities for their business plan in the instructions, stating, "[T]he objectives and priorities should be guided primarily by the priorities for your geographic area and the departmental key results...". The departmental key results related to IBD were extracted from the Strategic Planning and Priorities Framework and listed within the body of the instructions. To further strengthen the policy uplink, the Action Plan template also required that specific departmental key results be identified for each CSF and non-CSF initiative.
In December 2003, the former DFAIT was split into two separate organizations: International Trade Canada (ITCan) and Foreign Affairs Canada (FAC). Currently, ITCan centralizes the federal government's trade policy, investment promotion strategies and coordination of its international business development activities. Its four business lines provide the framework for the Department's managerial accountability as well as its planning and performance reports: Trade Policy, International Business Development (and Chief Trade Commissioner), Investment Promotion, and Corporate Services. Subsequent to the reorganisation, a new Strategic Framework(3) specific to ITCan emerged which represents the current policy orientation for IBD/CSF programming. Two ITCan strategic outcomes are presented as follows:
Understanding and advancing Canada's commercial interests internationally. Canada and Canadians are more prosperous as a result of a modern, open and rules-based international economic order.
Enabling Canada to succeed in the global market place. Canada's international competitiveness is maintained or improved through increased international commerce, two-way international investment flows, and science and technology linkages.
While the first strategic outcome may be relevant for the international trade policy business line, the second appears to have more relevance for the International Business Development and Investment Promotion business lines. While not substantially different from the IBD purpose statement contained in the 2001-2002 DFAIT Business Plan, which also made reference to investment and technology, this strategic outcome places considerable importance on the role of 'international commerce, two-way international investment flows, and science and technology linkages' as the means to achieving Canadian competitiveness and prosperity.
In comparison, the CSF strategic outcome presented above remains quite consistent with this policy orientation, although the CSF strategic outcome statement itself and the accompanying narrative text in other documents (e.g., RMAF, CSF Guidelines) could benefit from further articulation of the role of 'two-way international investment' and 'science and technology linkages' as important areas of IBD programming. Nevertheless, instructions for the Strategy template of the 2004-2005 IBD Planning and Reporting system has already been adjusted by requiring C-E Sections to identify key results and priorities with respect to all three IBD program areas (i.e., trade development, investment and science and technology).
As mentioned above, the 2003-2004 IBD Planning and Reporting Template did require C-E Section staff to identify in the Action Plan departmental key results to which the CSF-funded initiative would be expected to contribute and to which it actually contributed. Based on the fourteen departmental key results, the following were identified the most frequently in the 2003-2004 CSF reports:
The Canadian Commercial Interest List (CCIL) points to the relative importance of markets to Canada's international business interests. The list is a guide to the level of financial and human resources required to provide Trade Commissioner Service services in markets of a similar level of actual or potential interest to Canadian businesses.
The final year end total expenditures for FY 2003-2004 as at March 31, 2004 for the relevant Posts were plotted against the 53 markets on the 2003-2004 Canadian Commercial Interest List. The percentage was based on the 2003-2004 total CSF expenditure of $8,788,137. It should be noted that the CSF allocations to Geographic Divisions were calculated as a separate category. As the highest ranked market, the USA C-E Sections collectively received 25.43% of the total, while the markets ranked 2-7 collectively received 22%. Among the next group of markets ranked 8-25, the percentages of total were generally 1% or lower, however there were some notable exceptions (e.g., Italy (2.92%), Brazil (2.17%) and India (2.63%)). The rate of decline in the CSF funding allocations to markets ranked 26-53 was also gradual, remaining below 1% of total. What is of particular note however is that approximately $708,000, or 8% of total CSF expenditures, was spent in markets that did not make the CCIL in 2003-2004.
|Markets/Country||2003-2004 Actual CSF Expenditure||% of Total|
|USA - 1||2,235,156||25.43%|
|China - 2||319,271||3.63%|
|United Kingdom - 3||269,999||3.07%|
|Japan - 4||438,149||4.99%|
|France - 5||225,594||2.57%|
|Germany - 6||351,200||4.00%|
|Mexico - 7||331,900||3.78%|
|Total CSF Expenditures||$8,788,137||100.00%|
The Trade Team Canada Sectors (TTCS) bring government and industry together to coordinate national trade development planning and activities in key industries and markets. The key role of the TTCSs, composed primarily of industry associations and private sector representatives, is to develop sectoral trade development strategies and annual action plans. Instructions to the 2003-2004 IBD Planning and Reporting Template included guidance on accessing and using the TTCSs as a source of information during the IBD strategic planning process when priority sectors were selected. A three step analysis was undertaken to determine if the guidance was effective. First, the top 23 markets on the 2003-2004 CCIL were used in the sample and then cross-referenced with the TTCS priority industries/markets list (Table 1).
Second, an analytic report was prepared for each eligible market to determine the percentage of CSF initiatives that were coded against a given sector. Third, the TTCS priority industries were then plotted on the matrix according to the corresponding TTC priority market (shaded cells).
|Priority Sectors and Markets||All||USA||China||UK||Japan||France||Germany||Mexico||Korea||Australia||Italy||Spain||Brazil||India||Russia|
|Aerospace & Defence||4||4||6||4||2||17||3||2||6||6||3||1||2||4|
|Agriculture Tech. & Equip.||3||2||3||2||4||4||4||3||3||3||4||10|
|Agric. Food and Beverages||9||9||9||7||7||9||5||12||4||8||9||6||4||14||10|
|Arts & Cultural Industries||3||6||1||4||4||9||5||8||4||1||12||6|
|Building Products & Const.||5||5||8||4||7||4||3||6||8||4||8||3||4||8|
|Electric Power Equipment/Services||3||1||2||4||2||4||2||2||8||3||8|
|Fish & Seafood Products||3||3||8||3||5||9||4||2||4||6||8||3||4||2|
|Information Comm. Tech. (ICT)||11||12||14||3||7||9||5||14||8||8||14||6||15||12||6|
|Multi-Sectoral or Cross-Promotional||14||11||6||25||13||13||17||23||4||15||6||18||13||9||4|
|Metals, Minerals & related ...||2||1||2||2||3||2||4||6||2||3||4||3||8|
|Oil & Gas Equipment & Services||3||1||1||7||3||2||8||2||1||9||4||8|
|Rail and Urban Transit||2||2||2||3||1||3||4||2||3||6||2|
|Service Industries & Capital||4||3||5||2||2||8||2||5||3||4||2|
* Please note that discrepancies in the total percentages are due to n/a answers and rounding errors.
Table 1 raises some issues regarding IBD sector priorities. For example, the most frequently coded sector is 'Multi-sectoral or Cross Promotional' indicating perhaps that a good number of CSF initiatives are not, in fact, focussed on a priority sector and may represent broad trade promotion activities. In general, CSF initiatives are spread out across a large number of sectors in any given market. While this may be understandable for large multi-post markets (e.g., USA, China, Japan and Germany), it also seems to hold true for a number of smaller markets (e.g., Australia, Italy, Spain, Brazil). The data suggests that CSF funding is spread across a large number of 'priority sectors' in almost every market.
A closer look at the correlation between priority sectors and the frequency against which CSF initiatives are coded suggest that CSF funds are not concentrated to any great degree on TTCS priority sectors either. For example, the China market is a priority for eight of the TTCS industries, but only three had 9% or more of CSF initiatives allocated to those sectors, while three others had less than 2% of CSF initiatives allocated to them. The Japan, Germany, Australia, Italy and Spain markets have CSF initiatives similarly dispersed across many sectors and with little in the way of concentration in the TTCS industries. In fact, several of the TTCS priority industries (i.e., Electric Power Equipment and Services, Oil & Gas Equipment and Services and Plastics) had no 2003-2004 CSF initiatives programmed in their TTC priority markets. The data suggests that TTCS priority industries/markets are not a determinant factor in IBD/CSF programming.
When asked if CSF initiatives are generally focussed on priority sectors and in line with strategic priorities, 81% of TPMs responded 'definitely yes', and another 16.7% 'probably yes'. However, it is clear from the explanations that priority sectors for IBD programming are selected by TPMs based on local market factors and demand with little reference to TTCS priority sectors/markets. Also, in an effort to be responsive to an important Canadian business client or partner, C-E Sections also undertake CSF initiatives that do not address their selected priority sectors or may be multi-sectoral in nature.
The CSF is an integral part of the services delivered by the TCS, in particular the C-E Sections at Posts, and cannot be realistically distinguished from services delivered using other sources of funding. This is because the CSF was designed, not as a stand alone funding program, but rather as an integrated and complementary adjunct to the TCS- NA@W six core services. Since, the CSF business clients and partners are the same as the TCS as a whole, a client profile can be established by using data from TCS Client Surveys, as well as from the IBD Planning and Reporting system.
Based on the 2004 Client Survey, the majority of the TCS/CSF Canadian business clients are relatively small companies that employ 50 or fewer people (59%) and have annual worldwide revenues of less than $5 million (54%). The mid-size business client employs between 50-500 people (27%) and has world-wide annual revenues between $5-$25 million (28%). The large sized business client employs 500 or more people (14%) and has annual revenues of $25 million or more (18%).
Based on data extracted from the 2003-2004 IBD Planning and Reporting system, a wide variety of Canadian partner organisations are involved in TCS/CSF initiatives. The most frequently cited among the Team Canada Inc. members are: Industry Canada (23%), International Trade Centre (16%) Agriculture and Agri-Food Canada (11%), Export Development Canada (8%), National Research Council Canada (6%) and Natural Resources Canada (6%). Other government departments most frequently cited are: Investment Partnership Canada (26%), Canadian Food Inspection Agency (9%), National Defence (6%), Technology Partnerships Canada (6%) and Health Canada (6%). All of the provinces and territories were involved in varying degrees with Ontario (21%), Quebec (17%), Alberta (13%) and British Colombia (10%) being cited most frequently. Large municipalities, Canadian Associations and Institutions round out the list of TCS/CSF partner clients.
Consistent with the 2000 Client Survey, 60% of Canadian business and partner clients have used TCS/CSF services for five years or less. Large-sized business clients are more likely to have been using the TCS/CSF services for a longer period (6.9 years) compared to mid-size (6.2 years) and smaller business clients (4.5 years). Partner-clients have been using the TCS/CSF services on average nearly two years longer (7.5 years) than business clients (5.6 years). One key difference in the 2004 Client Survey is that new clients (i.e., less than one year experience with the TCS) represent less than two percent of the total population compared to 18% in 2002, otherwise the general client profile has remained the same.
Canadian business and partner clients were active in targeting the following foreign markets at the time of the 2004 Client Survey: USA, China, Japan, Mexico, United Kingdom, Australia, France and India. In terms of their expectations, 79% of business clients and 57% of partner clients were targeting exports or direct sales at the time they used the TCS/CSF service. This is somewhat lower on average (76%) than the 2002 Client Survey where 80% shared this objective. Other reasons for their business activities abroad included: service or assistance to other Canadian organisations; seeking R&D or technology partnerships; licencing or franchising; foreign acquisition, equity investment or joint venture; attracting foreign investment; and, seeking international project financing. Partner clients tended to cite service or assistance to other Canadian organisations, seeking R&D or technology partnerships, and attracting foreign investment significantly more frequently than their Canadian business client counterparts.
Based on the 2004 Client Survey findings, the vast majority of Canadian business and partner clients would use TCS services again (92%) and would recommend TCS services to their associates (88%). A large majority (85%) of clients felt that their investment was worthwhile and that the TCS helped them advance their business interests (78%); there have been no changes in these measures since the 2002 Client Survey. Overall, more than eight out of ten (82%) clients agreed that the quality of the service provided by the TCS was impressive.
Based on the on-line survey, 36% of TPMs estimate that requests for services had increased and 31% estimate that it had somewhat increased, with less than a quarter of the responses indicating no change. Broken down by geographic region, the highest increases are in Europe and North America, with approximately 50% of TPMs estimating somewhat of an increase in the remaining regions. Based on the explanations provided by TPMs in the survey responses and during follow-up interviews, changes in the macro-economic conditions in the local market account for either the increase or decrease in the number of requests for services, while proactive IBD initiatives by the C-E Sections is the other principal reason cited for the increased demand in the Europe and the North America regions.
|Has Not Changed||10||23.8%|
Since the financial data for this analysis was not available from the ITCan Information Management System (IMS), the 2004-2005 IBD Planning and Reporting system was used to extract figures for Table 2. All figures are based on the planned budgets submitted by C-E Sections as part of their IBD Strategic Plan. The only exception is the 'Estimated 2004-2005 Revenue', representing funds leveraged from partners which is based on the previous year's actual revenue recorded in the IMS. It should be noted that, Enhanced Representation Initiative (ERI) funds are only allocated to Posts in the US, 'IBD Travel and Hospitality' funds are reserved for IBD programming and 'IBD Other Funds' include contributions from other government departments, Branding Canada, and others.
|Regional Bureau||2004-05 CSF Allocation||Estimated 2004-05 Revenue||Total CSF Related Funding||2004-05 ERI Budget||2004-05 IBD Hospitality||2004-05 IBD Travel||2004-05 IBD Other Funds||Total Non-CSF Funding Sources||2004-05 CSF % of Total|
Assuming the adequate accuracy of the 2004-2005 IBD Planning and Reporting financial data, the above analysis indicates that the 'Total CSF Related Funding' represents 50-57% of all funds available to C-E Sections for all Geographic Bureaus, with the exception of Europe where it represents 71%. This exception can be explained by the relatively lower IBD travel and hospitality budgets. Interestingly, if the comparison were made between just the '2004-2005 CSF Allocation' and only the IBD travel and hospitality budgets, the figures for North America and Europe Bureaus would be 70%, while the other bureaus represent < > 50%. Therefore, Europe and North America Bureaus receive similarly high CSF allocations relative to their IBD travel and hospitality budgets.
When asked if there are sources of funding other than the CSF available to their C-E Sections which could be used for the same purpose as the CSF, 55% of TPMs responded 'Yes' and 45% 'No'. When examined on a regional basis, almost 80% of the 'No' responses were from the Africa & Middle East and Caribbean & Latin America regions. Close examination of the 'Yes' responses reveals that Posts located in the USA have access to ERI and Branding Canada funds, while the Posts in the Asia and Europe regions cited funds from other government departments, especially Agriculture and Agri-Food Canada (AAFC), as well as travel and hospitality budgets as their alternate funding sources.
When asked what effect the reduction in CSF funding has had on the ability of the C-E Section to maintain the quality of services to priority clients, 38% of TPMs responded negative and 19% responded very negative for a total of approximately 57%. Somewhat surprising is that the reduction of CSF funding apparently had no effect on the quality of client services for 40% of the TPMs who responded. An analysis of TPM explanations by geographic region provided considerable insight. The Africa and Middle East and Caribbean and Latin America regions account for more than half of these "no effect" responses. Explanations provided by TPMs to the on-line survey and during follow-up interviews cited the already minimal CSF funding received and the negligible effect that the 2004-2005 reductions have had on client services.
Regarding the 57% of respondents who viewed the CSF budget reductions as negative, their explanations can be summarised by the expression, "We can no longer do more with less, ... only less". Explanations provided by TPMs to the on-line survey and during follow-up interviews indicate that the effect of the reductions on IBD programming over the past few years have restricted the ability of small C-E Sections to undertake market studies on emerging sectors, to gather relevant market intelligence, to encourage qualified contacts to visit Canada and to engage in IBD activities as a lead partner. These TPMs have seen their CSF budgets reduced to a point where it begins to jeopardise even their ability to leverage funds as this respondent attests: "[T]he CSF is the only source of funding directly managed by the Posts, which we can use to leverage contributions from partners in other government departments, from local contact partners and from clients. Without CSF we are very much handicapped partners." The reductions in CSF funding have handicapped smaller C-E Sections to the extent that TPMs wonder whether the reductions are part of an unacknowledged intent to abandon proactive operations in lower ranked markets on the CCIL (e.g., markets 24-53).
On the other hand, responses from TPMs in the North America and Europe regions are remarkably different. For the most part, they either cited the absence of any CSF funding reductions, or the ability to adjust with access to other funding sources (e.g., ERI, travel and hospitality budgets).
To test the internal consistency of responses to the on-line survey, TPMs were also asked if the current level of CSF funding was sufficient or appropriate to carry out work to provide services to priority clients. 53% of all TPMs responded either probably not or definitely not, and 60% of the TPMs in the Africa & Middle East and Caribbean & Latin America regions responded 'probably not' or 'definitely not', which correlated quite strongly with the 57% who felt that the recent CSF budget reductions had a negative effect on the quality of services to priority clients. Again, the reverse was true in the case of the North America region where 67.7% of TPMs felt that the current level of CSF funding was sufficient or appropriate to carry out work to provide services to priority clients, while in the more diverse Europe region, the responses were split at 45% for 'probably yes' and 45% 'probably no'.
It was suggested by several TPMs during the follow-up interviews that there should at least be a minimum amount of CSF funding per full time equivalent (FTE) to ensure an optimal utilisation of C-E Section staff, irrespective of the Post's location. The underlying principle to this suggestion was that without sufficient CSF funding as working capital for proactive IBD initiatives, these FTEs would not be appropriately utilised and perhaps ITCan's expectations of a NA@W style of client service in these Posts should be reconsidered.
When asked if their C-E Section would be able to maintain the current quality of services to priority clients without any CSF funding, 77.5% of TPMs responded definitely not. Most explained their responses in terms of the inability to be proactively engaged in IBD activities and to leverage additional funds from OGDs, local contacts and business clients. While clients already active in the market would continue to be provided with quality services upon request, there would be little in the way of business development and attracting new clients or local partners, which would be contrary to the philosophy of the NA@W. Without CSF funding, COs in many of the smaller Posts/C-E Sections with limited travel budgets would be virtually desk bound, forced to rely on the local print media and the internet for MII, local contacts and business opportunities.
This chapter represents the findings on the results achievement /success of the CSF.
Table 3 presents types of products delivered to clients by CSF initiatives based on reporting data contained in the 2003-2004 IBD Planning and Reporting system. Percentages are based on the total number of products generated by all CSF initiatives. The two most common products are a list of qualified contacts and sector market documents, which would include market information products (e.g., briefs, reports, country sector profiles) generally used in a MII approach to trade promotion. The next two most common products are marketing and promotional documents and brochures which are more characteristic of the trade promotion type activity. It should be noted that there were no significant differences when the data was analysed by Regional Bureau.
Table 3: Types of Products Delivered by CSF Initiatives
Table 4 presents the types of services delivered to clients by CSF initiatives based on reporting data contained in the 2003-2004 IBD Planning and Reporting system. Percentages are based on the total number of services delivered by CSF initiatives, since any given initiative could provide clients with several types of services. The most common service is to 'identify/expand the network of key qualified contacts', which is similar to the most common product 'list of qualified contacts'. The next three are also clearly focussed on providing clients with MII services on local markets, market opportunities and local companies. A number of other services also figure in this list (e.g., making appointments with local contacts, business travel advice, group briefings and debriefings). It should be noted that there were no significant differences when the data was analysed by Regional Bureau.
Table 4: Types of Services Provided to Clients
Based on data extracted from the 2003-2004 IBD Planning and Reporting system, it was possible to correlate the key CSF products and services with the extent of results achievement. The frequency with which products and services were identified as having 'greatly' or 'considerably' contributed to the achievement of each short-term result determined their nominal ranking in terms of value added. Using this technique, the highest ranked on the list was viewed to have approximately eight times more value-added than the lowest ranked. It should also be noted that the data for this analysis is based on the opinions and judgement of C-E Section staff when they complete their CSF initiative reports. The following is the nominal rank order for key CSF products and services with the most value added.
Of note is the large number of services on the list and their relatively high ranking in relation to the products. Products such as marketing, promotional documents and brochures figure lower on the list, with approximately 1/5 the value-added to results achievement compared to the intelligence gathering services.
Based on the 2003-2004 IBD Planning and Reporting system, and keeping in mind that the data for this analysis is based on the opinions and judgement of C-E Section staff, 18% of CSF initiatives were thought to have contributed 'greatly', 40% 'considerably' to the more effective delivery of TCS six core services. It should be noted that these findings correlate very strongly with the effect of the reductions in CSF funding on the quality of services to priority clients (i.e., very negative (19%) and negative (38.1%)) indicating high internal consistency in TPM responses on this issue.
The 2002 Client Survey served as a useful secondary data source, since it asked Canadian business and partner clients if they were aware that the TCS offered six standardised core services. The following is a summary of the findings:
While these figures establish a useful baseline for this short-term outcome, there was no equivalent question asked in the 2004 Client Survey to draw a comparison. Since the TCS's six core services pre-date the launch of the CSF the data demonstrates that considerable progress had already been made to educate clients on their service offering.
Based on Client Survey findings, TCS clients have remained appreciative of the services they received. The overall satisfaction rating out of 5 was 4.02 in the 2002 Client Survey, unchanged from the 2000 Client Survey, and was somewhat higher at 4.06 in the 2004 Client Survey when averaging the aggregate ratings for accessibility (4.11), timeliness (4.07) and quality (4.02) of the TCS services.
TSC/CSF initiatives provide market prospect and face-to-face briefings to both business clients and partners in order to improve their awareness and knowledge of opportunities for trade, investment and S&T in the foreign business environment. The performance indicators for measuring achievement of this second short-term outcome focus on the volume and quality of MII acquired, as well as level of confidence to identify and pursue opportunities(4). The 2002 and 2004 Client Surveys collected data with questions relevant to the specific attributes of these indicators. Table 5 presents a comparative analysis of this performance data.
|Specific Attributes||All Clients||Business Clients||Partner Clients|
|The market assessment rationale was clearly explained||70%||66%||69%||65%||77%||66%|
|The market information was up-to-date and accurate||83%||80%||82%||78%||89%||94%|
|The market information was relevant to my organisation||83%||85%||82%||85%||92%||96%|
|The market information was helpful in my decision-making||81%||85%||80%||83%||88%||96%|
|The market information helped me decide whether or not to pursue a market||71%||73%||69%||71%||85%||89%|
|The market intelligence provided was useful||82%||81%||82%||79%||83%||79%|
|The intelligence provided identified both opportunities and difficulties||80%||72%||77%||70%||90%||80%|
|Advice concerning the next steps was helpful||70%||70%||69%||69%||84%||76%|
The level of satisfaction with the quality of MII has remained relatively high since the 2002 Client Survey. However, the scores are approximately 10% lower for questions relating to decision-making support on whether or not to pursue a market and possible next steps, as well as for explaining the rationale for market assessments. Client perception regarding the provision of MII services has remained consistent since the 2002 Client Survey.
TSC/CSF initiatives provide key contact search and local company information to both business clients and partners in order to assist them in expanding their network of qualified local contacts. The performance indicators for measuring achievement of this third short-term outcome focus on the volume of key contacts acquired and confidence in market intelligence on these local companies. The 2002 and 2004 Client Surveys collected data with questions relevant to the specific attributes of these indicators. Table 6 presents a comparative analysis of this performance data.
|Specific Attributes||All Clients||Business Clients||Partner Clients|
|The key contacts were relevant to my organisation||86%||85%||85%||84%||93%||98%|
|The information provided on the key contact list was up-to-date and accurate.||87%||75%||85%||74%||98%||89%|
|The contacts were helpful to my organisation||81%||76%||80%||74%||86%||91%|
|The information about the company was relevant to my organisation||82%||78%||80%||76%||95%||100%|
|The local company information was valid and reliable||85%||78%||84%||77%||95%||92%|
|The market intelligence about local companies and organisations was useful.||74%||84%||71%||83%||95%||99%|
Since the 2002 Client Survey, the level of satisfaction with the quality of key contact and local company information has remained relatively high and steady for partner clients, while declining somewhat for business clients. The scores are approximately 15% lower for business clients in comparison to partner clients in this area.
The reporting template for IBD/CSF initiatives requires C-E Section staff to assess the extent to which CSF initiatives contribute to short-terms outcomes using five-point nominal scale. Although there are some technical issues with the scale itself (i.e., positively skewed), the data does provide some insight into the opinions and judgements of C-E Section staff and are presented in Table 7. Approximately 70% of CSF initiatives contribute either 'greatly' or 'considerably' to the CSF's short-term outcomes, with the exception of short-term outcome #5 where the contribution is not as 'great'.
|Short-Term Outcomes||Greatly||Considerably||Somewhat||A Little||Not at all|
|1. Effective delivery of TCS core services||21.82%||49.12%||24.36%||4.46%||0.24%|
|2. Increased awareness for Canadian clients||23.48%||45.94%||26.05%||4.21%||0.31%|
|3. Expanded network for Canadian clients||27.04%||46.98%||22.00%||3.74%||0.23%|
|4. Increased awareness for foreign contacts||24.29%||45.90%||20.60%||7.20%||2.01%|
|5. Enhanced guidance / support by Cdn partners||16.14%||42.83%||33.07%||6.87%||1.10%|
The reporting template also asks C-E Section staff to specify the details of the outcomes achieved. For increased awareness and knowledge by foreign contacts of opportunities in Canada (#4), responses mention the acquisition of new information on Canadian products, services and technology (27%), contact with potential Canadian suppliers and strategic partners (24%), increased awareness and knowledge of Canada's policies, regulations and emerging trends in priority sectors (21%), collaboration opportunities identified with Canadian S&T/R&D development community (12%), and Canada considered as a preferred site for foreign direct investment (10%). While this may serve as an indication of the outcome achievement, it cannot substitute for data collected directly from foreign client contacts.
For enhanced guidance/support by Canadian partners (#5), the responses cite the improved identification of matching opportunities for Canadian firms (38%), strengthened linkages with foreign governments, multilateral organisations or foreign associations and institutions (22%), enhanced bilateral cooperation agreements and exchanges between Canadian partners and their foreign counterparts (13%), increased participation by Canadian partners and corresponding foreign organisations in joint international cooperation projects (12%), and new regulatory policies, programs, practices and incentives introduced to support the development of international business capacity (8%). Again, while this may provide an indication of short-term outcome achievement, it cannot substitute for data collected directly from Canadian partners and co-delivers.
The most relevant data source for assessing progress made toward the achievement of CSF medium-term outcomes are the TCS Client Surveys, since they ask clients to assess the contribution which TCS services have had on different aspects of their business. From the other perspective, the client surveys ask clients to attribute changes in their business to TCS services they received. A comparative analysis of the findings of the 2000, 2002 and 2004 Client Surveys presented in Tables 8 and 9, provide some insight into the progress made toward the achievement of CSF medium-term outcomes.
|Business Aspects||2000 Client Survey||2002 Client Survey||2004 Client Survey|
|1. Saving time and money||79%||79%||85%|
|2. Market access||69%||70%||85%|
|3. Finding the right people||69%||69%||85%|
|4. Making better decisions||73%||74%||80%|
|5. Business image/credibility||73%||70%||79%|
|7. Organisation's financial results in the past 12 months||35%||36%||47%|
|8. Signing of other agreements||38%||36%||41%|
|9. Organisation's financial results over the next 24 months||60%||52%||41%|
Three main observations can be made about the above data. First, the findings for the 2000 and 2002 Client Surveys were consistent, showing relatively little change. Second, the relative rank order in terms of positive impact has been relatively stable across all three surveys, with only a couple of exceptions. Third, and most importantly, the 2004 Client Survey findings show significant improvements in all business aspects in comparison to the two previous survey findings, with only one exception (i.e., 'financial results over the next 24 months').
Two main observations can be made about the data for partner clients in Table 9. First, the relative rank order in terms of positive impact has been relatively stable across all three surveys, with only a couple of exceptions. Second, and most importantly, the 2004 Client Survey findings confirms a downward trend in positive impact on partner clients that began with the 2002 Client Survey. While there are slight improvements in a few business aspects(i.e. #6-9), there have been significant declines of 6-10% in business aspects #1-5.
|Business Aspects||2000 Client Survey||2002 Client Survey||2004 Client Survey|
|1. Saving time and money||87%||81%||75%|
|2. Market access||79%||76%||72%|
|3. Making better decisions||80%||79%||71%|
|4. Finding the right people||81%||79%||69%|
|5. Business image/credibility||83%||75%||69%|
|7. Organisation's financial results over the next 24 months||65%||38%||52%|
|8. Organisation's financial results in the past 12 months||56%||35%||38%|
|9. Signing of other agreements||52%||33%||35%|
A valuable addition to the 2004 Client Survey is the inclusion of questions regarding the achievement of financial results attributable to the TCS/CSF. Over half of all business clients (55%) claimed to have been involved in the achievement of financial results abroad. Overall, two-thirds (66%) of these clients attributed, at least in part, the financial results to the TCS/CSF services, with some variance between large companies (73%) and small companies (59%). The types of financial results achieved abroad were export sales (91%), partnership or alliance leading to revenue (28%), Canadian investment abroad leading to revenue (12%) and foreign direct investment in Canada (8%). The average value of the financial results attributable to the TCS/CSF for large companies was $21.7 million, mid-sized companies $4.2 million and small companies $.5 million, while the average for all business clients was $7.6 million. Unfortunately, the IBD Planning and Reporting system does not track business client numbers in these categories, so it was not possible to calculate the total financial results attributable to the TCS/CSF.
From the perspective of TBX/CSF Unit staff, the most significant benefit of the CSF was to make incremental resources available through delegated authority directly to TPMs/C-E Sections to provide services to Canadian business clients without going through Geographic Divisions. The CSF has also been used as "seed money" to leverage additional funds for client-focussed services in priority sectors. This was a significant departure from how the predecessor programme was managed. The Geographic Division staff view the CSF primarily as a source of funds to support client focussed IBD programming, and secondarily as "seed money" used to leverage additional funds from Canadian business and partner clients.
A qualitative analysis of the statements and examples provided through the on-line survey by the TPMs in answer to this question reveals the following typology of IBD activities most frequently cited as the most significant:
|Market information and intelligence||12||25%|
|Science and technology promotion||1||2%|
|Other (flexibility, networking, etc.)||6||13%|
From the perspective of TBX/CSF Unit staff, the CSF turned out to be a catalyst for a reform of trade program management in C-E Sections in terms of planning, decision-making and reporting. The IBD strategic planning process has in many cases resulted in positive team-building, improved staff understanding and commitment to shared IBD programming objectives. TPMs and COs have a better understanding of the principles and concepts of Results-Based Management (RBM), and have made surprising efforts to achieve high performance ratings. It has also encouraged them to evaluate their performance, and to make adjustments in programming where required.
According to the Geographic Division staff, the CSF has introduced a well structured planning and reporting system for use by Geographic Divisions and C-E Sections at Posts. It has resulted in more focussed strategic planning on priority sectors and greater transparency and accountability for the use of financial resources. The delegation of decision-making authority to TPMs for the management of CSF funds has had the unexpected benefit of making financial management of the funds more efficient.
A qualitative analysis of statements and examples provided by TPMs regarding the most significant benefit or result which was not planned or expected generated the following responses:
|Unexpected benefit or result||Count||Percentage|
|Better expected results than planned||12||36%|
|Able to do more activities than planned||4||12%|
|More results-oriented management||4||12%|
|More flexibility on short notice||3||10%|
|More leveraged funds than expected||2||6%|
|No unexpected benefits or results||8||24%|
From the perspective of TBX/CSF Unit staff, it may have frayed relations with Geographic Divisions because of the direct delegation of the CSF management to TPMs/C-E Sections. Geographics lost control over the allocation of CSF funds and financial management because funds were transferred from TBX directly to Posts. This meant that the Geographic Division staff responsible for financial management of the CSF had to be reassigned. On the other hand, Geographic Divisions were asked to provide more value-added support services to Posts (e.g., strategic guidance, regional coordination, funds reallocation). The reduced flexibility in the use of the CSF has also been noted as a negative consequence, since it could no longer be used to subsidize overall Mission expenses.
The negative consequences of the CSF that were identified by the Geographic Division staff were articulated in terms of both the effects on working relations with C-E Sections and on their respective workload. While some Geographic Divisions provide sectoral expertise and guidance to Posts, they do not feel that they have the human resource capacity to review each Post's IBD Strategic Plan from a holistic perspective as expected by TBX and their C-E Sections. In larger more well-funded Posts, C-E Sections were not provided with more resources to handle the additional planning, administration and accounting responsibilities that have come with the delegated authority over the use of the CSF, resulting in an increased staff workload. On the other hand, the common perception among less-well funded geographic programmes with smaller Posts is that the cost of planning, administration and accounting in terms of C-E Section staff time exceeds the value of the results achieved with the CSF allocation. Many TPMs in small Posts in the Africa and Middle-East, and Caribbean and Latin America regions generally confirmed this observation in their on-line survey responses and during follow-up interviews. While they are appreciative of any amount of funding they receive, the administrative overhead has become burdensome in relation to the CSF allocations received. Due to the 2004-2005 reductions in CSF allocations, some Geographic Divisions are currently contemplating pooled funding mechanisms for their Posts/C-E Sections, or have negotiated additional "reserve funds" for in-Canada activities.
Almost 90% of TPMs stated that the CSF had no negative consequences for either their business clients and partners.
Geographic Division staff almost unanimously recognise the positive influence the CSF has had on their IBD strategic planning processes. There has been a greater inclusion of all C-E Section staff in strategic and action planning, leading to better quality CSF initiatives and enhanced staff commitment to results achievement. Similarly, they also acknowledge the positive influence the CSF has had on their financial planning management processes in terms of greater transparency and accountability. While the CSF was not attributed with changes in human resource management and planning per se, it is nevertheless generally credited with influencing the IBD management culture and practices in Geographic Divisions and C-E Sections in Missions.
Based on the survey findings, most TPMs have either definitely (61%) or probably (27%) changed IBD management practices in their C-E Section as a result of the CSF. While improved strategic, financial and human resources planning and management were cited most frequently as the reasons, several other reasons were mentioned: more delegated responsibility to staff, better relations with partners, and more results-based management being practised.
More details on the influence the CSF has had on Geographic Divisions and C-E Sections is presented below in the context of the finding on cost-effectiveness.
This chapter represents the findings on the cost-effectiveness of the CSF
The figures presented in Table 10 are based on ITCan's IMS financial data and the ratios calculated allowing for rounding-up. The Africa & Middle East Bureau has consistently generated over the two years $1 of leveraged revenue for every $4 of CSF funding it received. Asia-Pacific and Europe Bureaus have shown the most improvement, while the remaining bureaus have remained consistent in generating $1 of leveraged revenue for every $7 of CSF funding received. Although the data may suggest that less CSF funding generates proportionately more leveraged revenue, it also indicates that quite large amounts of leveraged revenue can be generated when working with a larger CSF allocation. For example, in 2002-2003 the North American Bureau generated as much leveraged revenue as the Africa and Middle East Bureau's CSF expenditures. Some TPMs are confident that the CSF has not yet reached its potential, and still see room for improvement, as reflected in this quote. "As time goes, we could be surprised by our own ability to leverage CSF funds."
|Regional Bureau||2002-2003 CSF Expenditures||2002-2003|
Cost Sharing Revenue
Ratio CSF to Cost Sharing Revenue
Cost Sharing Revenue
Ratio CSF to Cost Sharing Revenue
|AMEB||$464,956||$122,190||4 : 1||$514,061||$133,814||4 : 1|
|C&LA||$575,941||$78,968||7 : 1||$668,730||$97,544||7 : 1|
|A-P||$1,905,306||$118,397||16 : 1||$1,856,709||$153,357||12 : 1|
|EU||$2,463,236||$48,791||50 : 1||$2,805,785||$290,728||10 : 1|
|NA||$2,931,143||$458,895||6 : 1||$2,923,667||$436,989||7 : 1|
|ALL||$8,340,582||$827,241||10 : 1||$8,768,952||$1,112,432||8 : 1|
5.1.2 Ratio CSF/O&M Budgets
Since the financial data for this analysis was not available from the ITCan Information Management System, the 2004-2005 IBD Planning and Reporting system was used to extract the figures for Table 11. All figures are based on the planned budgets submitted by C-E Sections as part of their IBD Strategic Plan. Assuming the accuracy of the available data and allowing for some rounding-up in the calculation, the findings show a 1:1 ratio for the first three geographic bureaus, while the Europe and North American Bureaus have a substantially higher 3:1 ratio.
Cost Sharing Revenue
CSF Related Funds
IBD Section Hospitality
IBD Section Travel
|2004-2005 Estimated IBD Funding||2004-2005|
Ratio CSF to O&M Budgets
|AMEB||$315,121||$133,814||$448,935||$144,365||$185,076||$329,441||1 : 1|
|C&LA||$427,837||$97,544||$525,381||$162,772||$265,305||$428,077||1 : 1|
|A-P||$1,199,497||$153,357||$1,352,854||$439,839||$740,715||$1,180,554||1 : 1|
|EU||$1,899,824||$290,728||$2,190,552||$349,305||$480,229||$829,534||3 : 1|
|NA||$2,094,271||$436,989||$2,531,260||$261,545||$615,707||$877,252||3 : 1|
|ALL||$5,936,550||$1,112,432||$7,048,982||$1,357,826||$2,287,032||$3,644,858||2 : 1|
The figures presented in Table 12 are based on ITCan's IMS financial data and the percentages calculated based on CSF Revised Allocations(5). C-E Sections are encouraged to closely monitor their CSF expenditures during the course of the year. The unused portion of funds is to be returned to the respective Geographic Division or to TBX for reallocation. This reallocation process optimises the use of available CSF funds by C-E Sections that have the capacity to implement previously unfunded CSF initiatives late in the fiscal year. Consequently, the percentage of lapsed funds is calculated against the final reallocation figures, as opposed to those determined at the beginning of the year.
CSF Revised Allocation
% Lapsed Funds
CSF Revised Allocation
% Lapsed Funds
It is clear from the very significant improvement in the year-over-year percentages that the reallocation process has been effective in reducing the total amount of lapsed CSF at the end of the fiscal year.
Geographic Division staff almost unanimously recognise the positive influence the CSF has had on their financial planning management processes in terms of greater transparency and accountability. The use of the CCIL to guide the allocation of the CSF to Posts has removed some ad hoc decision-making by Geographic Divisions, and increased the transparency of the financial planning process. The delegation of decision-making authority to TPMs for the management of the CSF, combined with their increased accountability through the application of performance criteria, has also encouraged more frequent monitoring of disbursements and subsequent programming adjustments. C-E Sections can choose to send any outstanding CSF balance to their Geographic Division where it is pooled and reallocated, so as not to lapse scarce funds at the end of the year.
When asked specifically if the CSF contributed to better financial planning and management, TPMs responded probably yes (32.5%) or definitely (42.5%) yes, which demonstrated a high level of internal consistency with the similar survey question regarding changes in IBD management attributable to the CSF. Posts with smaller C-E Sections, especially in the Africa and Middle-East Bureau, tended however to respond to the contrary seeing no change and citing small CSF allocations as the main reason. Many of them are dependent on the Mission's financial administrators and accounting systems to track and report on expenditures. They do not have access to an integrated financial planning and accounting system, and the IBD Planning and Reporting system cannot be used for that purpose. COs in these Posts have little direct financial management responsibility.
C-E Sections in larger Posts tend to have set up separate and parallel accounting systems, such as, spreadsheets, to keep track of expenditures, whether CSF, travel, hospitality, or other sources. This allows for closer financial monitoring and justifies the reallocation of the CSF on a timely basis, which has become critical for larger IBD programmes with larger CSF allocations. Some constraints have made financial planning and management more difficult for all Posts.
Since the "split" of FAC and ITCan it has become increasingly more complex and difficult to get accurate accounting on expenditures against the CSF budget. There are frequent mis-codings between CSF and Mission budgets; identifying these mis-codings is extremely difficult. Leveraged revenue for example, has also become more difficult to track. In the past, Geographic Divisions were able to post revenue themselves in the IMS. They can no longer track revenue by CSF initiative because FAC/Post lumps financial transactions together. This requires very complicated series of transactions between FAC and ITCan.
The other problem has been the lateness of the notification of the CSF allocation amounts well into the fiscal year, which hampers good financial planning. In addition, C-E Sections were notified in November whether they would actually receive the full allocation. For these reasons, financial planning and management have recently become more time consuming and administratively burdensome.
While it was not possible to obtain data on the overall percentage of C-E Section staff time spent on CSF initiatives, the 2003-2004 IBD Planning and Reporting system contained data in the reporting templates on the amount of time that IBD staff spend on each CSF initiative. Table 13 presents this data by Regional Bureau.
|Regional Bureaus||Over 6 Months||4-6 Months||1-3 Months||1 Month or Less||1 Week or less||Not Indicated|
For 'All' Regional Bureaus combined, the data is skewed to the short end of the time continuum (i.e., in 83% of the cases C-E Section staff spend less than three months working on a CSF initiative). Or, in 57% of the cases they spend less than one month working on a CSF initiative. The North America Bureau figures are particularly skewed to the short time frame, where 68% of cases staff spend less than one month working on a CSF initiative. Given this time frame, the data suggests that CSF initiatives can be organised and implemented in a short amount of time, which is typical of trade promotion events, recruitment for trade fairs and missions, as opposed to the longer term MII gathering activities.
The 2003 Employee Survey served as a useful source of secondary data. It asked the C-E Section staff at Posts to "allocate the percentage of time, including overtime, receptions and official function, that you spend in a typical month, performing or supporting the following activities." Table 14 presents the data collected.
|Activity Type||Average % of Time in a typical Month||Change from 2000 Employee Survey||Change from 1998 Employee Survey|
|Trade development activities initiated by your Post||25%||-3%||0.6%|
|Trade development activities in response to inquiries or requests||23%||-1%||-5.5%|
|Investment development activities initiated by your Post/IPC||7%||-1%||0.3%|
|Investment develop. activities in response to inquiries/requests||4%||-2%||-1.2%|
|S&T transfer activities initiated by your Post||3%||0||0.1%|
|S&T transfer activities in response to inquiries or requests||3%||0||-0.4%|
|Trade policy and market access activities||6%||0||N/A|
|Planning and prioritizing workload||10%||N/A||N/A|
Of interest are the sum totals for trade development (48%), investment development (11%) and science and technology transfer (6%), irrespective of whether the activities were considered proactive or reactive. It should also be noted that IBD staff at Posts reported spending 29% of their time on planning and prioritizing workload or other activities, which are non-client focussed activities.
Using the 2003-2004 IBD Planning and Reporting system data on CSF initiatives by type of IBD activity, the figures are remarkedly similar for trade (33%), investment (12%) and science & technology (8%). It should be noted that the IBD activity types are not mutually exclusive, so there is some overlap between IBD Program Delivery (33%) and the aforementioned trade, investment and S&T figures. Nevertheless, in order of magnitude it is clear that C-E Section staff spend considerably more time on trade development than investment and science and technology related IBD activities.
The 2003 Employee Survey served as a useful source of secondary data. It asked the C-E Section staff at Posts to "allocate the percentage of time, including overtime, receptions and official function, that you spend in a typical month, supporting the following clients or groups." Table 15 presents the data collected on this question.
|Activity Type||Average % of Time in a typical Month||Change from 2000 Employee Survey||Change from 1998 Employee Survey|
|Capable and committed Canadian business clients||24%||0||2.5%|
|Unprepared and uncommitted Canadian business contacts||9%||0||-1.2%|
|Non-Canadian business contacts (local foreign firms and government partners)||17%||2%||.8%|
|Other Federal Government Departments and agencies, provinces, municipalities and industry associations.||11%||0||-.4%|
|Investment Partnerships Canada||3%||-1%||N/A|
|Management of Post C-E Section Program||11%||0||2.8%|
|Other Post Programs||11%||-2%||-1.1%|
When the above data is consolidated in terms of the three target client groups identified in the CSF RMAF, the following percentages of time spent in a typical month are: Canadian business clients (33%), foreign contacts (17%) and Canadian co-delivery partners (11%). Approximately 40% of C-E Section staff at Posts spend time on non-client-focussed support activities. It should be noted that these figures have remained fairly constant over the past three employee surveys.
While the CSF has not had a widely recognised influence on human resource management and planning per se, it is nevertheless credited with influencing the IBD management culture and practices in Geographic Divisions and in C-E Sections in Missions. Based on interview responses, the Geographic Division staff and TPMs believe that there has been a shift in their focus from short-term event-driven activity planning to high value-added client-focussed programming. When TPMs were asked in the on-line survey if the CSF contributed to better human resource planning and management, they responded probably (44.1%) or definitely (38.2%) yes, which demonstrated a high level of internal consistency with a similar question regarding changes in IBD management attributable to the CSF. However, C-E Sections in smaller Posts with only one or two COs have not experienced the same changes as their colleagues in larger Posts with more IBD staff.
The delegation of IBD/CSF programme management to TPMs/C-E Sections has had a very positive effect on human resource planning and management in larger Posts. Development of the IBD Strategy now involves the entire C-E Section staff in a team exercise as opposed to a top-down planning approach. This has had a positive effect on the staff sense of self-worth and performance. With the ability to set priorities and reorganize initiatives to take better advantage of staff skills and time, TPMs have shifted from activity to portfolio management, some going as far as establishing performance management agreements with their staff. COs have been increasingly delegated responsibility for sector portfolios, day-to-day management and monitoring of CSF initiatives and, in some cases, the latitude to decide how and where to spend the CSF allocation while being held accountable for the achievement of results.
From the perspective of TBX/CSF Unit staff, the guidance on eligible and ineligible expenses is clearly presented in the CSF Guidelines, RMAF and FAQ which are all available on the Horizons web site. Nevertheless, further guidance and clarification on this issue is sought by e-mail or telephone to which TBX responds. The basic principle is that funds are to be used to support client-focussed activities. CSF initiatives requesting funds inclusive of ineligible items would not have been approved.
Geographic Divisions are generally of the opinion that clear guidance is now being provided on what expenses are considered to be ineligible for the CSF. This guidance is provided in considerable detail on the Horizons/CSF web site, as well as in CSF Guidelines. The approach used is to delineate what expenditures the CSF is not meant to support, rather than establish an incomplete list of what would be eligible.
The CSF web site also includes guiding questions and answers to determine the eligibility of expenses and ten examples of appropriate CSF activities for further guidance. The TBX/CSF Unit has been proactive in disseminating a one pager outlining CSF eligible and ineligible expenses, and is considered by TPMs to provide good advice when called upon with questions.
Not withstanding these efforts, some Geographic Divisions have advocated for greater flexibility in the use of the CSF (e.g., to replace the Post Initiatives Fund, the Public Diplomacy Fund or other mission funds which are no longer accessible to the C-E Sections at Posts). While an overwhelming majority of TPMs (93%) are also satisfied that clear guidance has been provided, this does not mean that they are all in agreement with the stated funding restrictions. For example, the exclusion of funds for the translation of sector reports and/or market information products into Canada's official languages was the most frequently raised issue. The rationale for inclusion as an eligible expense is based on their pivotal role in a MII approach to trade development. Market information products are widely recognised as a key value-added component to any sector development strategy. However, if they cannot be made available to Canadian business and partner clients through the Virtual Trade Manager, Posts web-site or other official GOC media, then their overall usefulness is considerably diminished. The limited CSF available to commission market research studies and the inability to have them translated becomes an impediment to advancing an MII sectoral strategy.
Since it was not possible to analyse the ITCan IMS data related to financial transactions, reliable data cannot be presented on this question. However, anecdotal evidence was noted from time to time during interviews and in the survey responses that pointed to some common uses of the Fund that were perhaps not in keeping with the spirit and intent of "working with priority clients in priority sectors". For example, it seems to be quite common that C-E Section staff will attend regional trade fairs on their own to establish/maintain their networks with Canadian business clients and foreign contacts, or to stay abreast of developments in the sector. Similarly, several interviewees cited examples of IBD staff going on mission to Canada for the same reasons. The question is not whether these are examples of misuse, but rather the best use of the CSF, since in many ways they have a stronger rationale from a professional development perspective than a business development perspective. Other examples of questionable expenditures cited by TPMs are:
It is widely recognised by Geographic Division staff that priority sectors are selected by TPMs/ C-E Sections based on local market demands and business opportunities, and that the TTCS priority sectors are not an important consideration especially for those Posts in markets that do not figure on the TTCS list. Based on the on-line survey responses, 81% of TPMs responded 'definitely yes', and 16.7% 'probably yes' when asked if CSF initiatives are generally focussed on priority sectors and in line with their strategic priorities. Based on their explanations and follow-up interview responses, they select the priority sectors for IBD programming based on local market demands and business opportunities, and implement their CSF initiatives in accordance with their Strategy and Action Plans. Nevertheless, from time to time, and in an effort to be responsive, they may undertake initiatives that do not address the priority sectors per se. These exceptional initiatives are often multi-sectoral, or are undertaken in support of an important Canadian business client or partner.
Geographic Division staff almost unanimously recognise the positive influence the CSF has had on their IBD strategic planning processes. There has been a greater inclusion of all C-E Section staff in strategic and action planning, leading to better quality CSF initiatives and enhanced staff commitment to results achievement. The single template for strategic planning allows for easy review, cross-post comparison and dialogue between Geographic Divisions, their C-E Sections and among the latter within the same region or country. Consequently, IBD strategic plans are presumably more focussed than they were in the past, while operational plans are better coordinated within the country, region and with Canada-based activities.
When asked specifically if the CSF contributed to doing more strategic and operational planning, TPMs responded either probably (30%) or definitely (65%) yes, which demonstrated a high level of internal consistency with a similar question regarding changes in IBD management attributable to the CSF. Although several TPMs now refer to the CSF as only a component of the overall IBD planning process, it is clear from the explanations that the former was the catalyst for more systematic and rigorous IBD strategic and operational planning with a focus on priority sectors and client service. The following testimonial captures well the nature and importance of changes that C-E Sections have undergone.
"Until 2 years ago, consolidated IBD strategic planning for the program in this country as a whole was quite limited. The Embassy and the consulates operated more independently of each other. Now there is a multi-post planning and tracking system for IBD/CSF initiatives including all of the Posts. Planning is coordinated by the Senior Trade Commissioner with bottom-up input from Commercial Officers in all the Posts and across all lines of business (Trade, Investment, S&T). Tracking of expenditures is ongoing. Quarterly expenditure reviews allow for potential reallocation. Unfunded CSF initiatives form the basis for third and fourth quarter submissions to HQ for supplemental funding."
According to the TBX/CSF Unit staff, the IBD Planning & Reporting system has evolved considerably since its launch in fiscal year 2002-2003. It was initially designed to capture the planning and reporting information for CSF initiatives, but the following year incorporated broader IBD strategic plans and non-CSF funded initiatives. In 2004-2005, action plans for non-CSF initiatives represent approximately 25% of all ongoing initiatives. While the CSF has been a catalyst for IBD planning and reporting, the emphasis has been on strategic and operational planning. The application of performance criteria has helped in ensuring systematic planning by C-E Sections, and the quality of plans has improved with each year.
Since 2004-2005, the Strategy and Action Plan templates have stabilised, while the Reporting template of the IBD Planning & Reporting system will undergo changes as it has every year depending on feedback from Posts. It was initially designed as a performance self-assessment tool for TPMs and, as such, has inherent limitations in terms of assessing overall programme performance. While data fields on the reporting template have become increasingly more focussed and detailed, the TBX/CSF Unit has had to strike a balance between collecting desirable performance data and the increased administrative burden it places on C-E Sections staff.
The Geographic Divisions use the IBD Planning and Reporting system primarily in the same way as the C-E Sections of Posts (i.e., to fulfil the planning and reporting requirements for their CSF allocation). Depending on their level of involvement and role in the strategic planning process, they may also use the IBD system to review and comment on the Post Strategy and Action Plans. However, in most cases, the strategic planning process has been completed by the end of March, if not sooner, well before the planning templates are made available on-line, or due to be submitted. Consequently, the IBD Planning and Reporting system is not used by Geographic Divisions as a tool for coordination, sharing of planning information or quality control, but moreover as a repository of planning information which is accessed only after the Strategy and Action Plans are submitted by Posts.
The reporting template is the least used by Geographic Divisions and viewed as the weakest component of the IBD system for a number of reasons: 1) it doesn't link to WIN Exports, 2) has no means of attaching the actual products (e.g., market reports, participant lists), and 3) reports on very high level results, whereas Geographic Divisions require more detailed information on activities.
When asked if the IBD Planning & Reporting system has evolved into a useful management tool, TPMs responded probably (36.5%) or definitely (56.0%) yes. The overwhelmingly positive response to this question can be largely attributed to the well structured and standardised two-step IBD program planning and reporting templates used to coincide with the annual management cycle. While usage of the tool is encouraged through the application of performance disincentives, many respondents appreciate the management rigour and discipline that it requires of them.
Notwithstanding the above, an analysis of the response explanations also reveals a significant divergence of opinion on the usefulness of the planning versus reporting templates. While the relative merits of the planning and reporting templates are discussed in greater detail elsewhere in this report, suffice it to say that the Reporting template remains a work in progress. More direct linkages to action plans, better integration of financial data and a greater focus on medium-term performance indicators would be welcomed improvements by TPMs.
Tracking performance has not been a priority for the CSF/TBX Unit, strategic planning was. While the CSF RMAF did foresee the development of on-line reporting templates for Canadian business and partner clients, these were not developed because the CSF Unit did not want to duplicate WinExports database on clients, nor any client tracking that the electronic Client Relations Management (eCRM) was to develop. The reporting template was developed simply to help TPMs document their perception of results achievement. The completion rates of reports by C-E Sections has improved over the two years, and is markedly better than the predecessor programme. The application of performance criteria has helped in terms of ensuring systematic reporting by C-E Sections, but the quality of the reporting is still somewhat variable.
When asked if the CSF has contributed to better performance tracking and reporting, TPMs responded probably (43.2%) or definitely (32.4%) yes, however an analysis of the response explanations reveals a significant divergence of opinion with regard to performance reporting versus tracking. While the on-line web-based approach to performance reporting is problematic for some Posts due to narrow communications bandwidth, it is generally appreciated for its structured, standardised and relatively streamlined format. Fixed deadlines for report submission, regular reminders and the use of disincentives have all contributed to more regular and timely performance reporting. The reporting task itself provides COs with an opportunity to reflect on the performance of their CSF initiatives and reinforces the results-based management culture in the C-E Section at Post.
There is also the perception however that the reporting template is very short-term and event driven with a preponderance of questions focussed on collecting data on activities, products and NA@W services provided. It does not adequately capture performance data that is useful to COs to determine if the CSF initiative has been successful, or not, since this requires medium-term follow-up tracking with Canadian business and partner clients. One respondent stated: "It is difficult to track the success of events other than in terms of 'yes the event was held' and 'this is how many people [companies] participated.' Whether the event ultimately leads to increased sales is difficult if not impossible to track. Simply having the CSF does not really affect how we track and report on events."
The data in the IBD Planning and Reporting system has only been used sporadically by Geographic Divisions to enhance coordination of IBD activities, to avoid duplication of effort and optimise collaborative activities in the same sector throughout a region or country. For some, it also serves as a source of information for internal briefings for ministerial missions and senior managers going to the field, as well as for ideas and best practices in IBD programming.
When asked if the data captured in the IBD Planning & Reporting system has been used by C-E Section to improve the delivery of Trade Commissioner Services, TPMs responded probably (27.7%) or definitely (35.1%) yes. However, an analysis of the response explanations also reveals the same pattern, as discussed elsewhere in this report, in terms of a significant divergence of opinion on the usefulness of the planning data versus the reporting data, with the latter being the least useful.
When asked if their CSF initiatives have made a reasonable contribution toward the CSF programme goal, 100% of TPMs responded either probably or definitely yes.
When asked if the CSF represents good value to Canadian business and partner clients, 100% of TPMs responded either definitely yes (73.7%) or probably yes (26.3%). While most of the responses were based on a "gut feeling" and not credible return on investment data, most TPMS did and can point to the amount of leveraged funding that a CSF initiative can generate. Contributions from business clients, local contacts and other government departments are an unequivocal indication of the value that they place on the IBD initiatives lead by the staff of C-E Sections at Posts.
IBD strategic planning is HQ driven and led by TBX with Branch management. The CSF is one component of the plan along with other programs and divisions, so there has been an increasingly tighter alignment with T-Branch objectives and priorities. While there has been an attempt to align IBD resources with the Canadian Commercial Interests List, which the CSF uses to allocate funds to Posts, there has been no real influence on the allocation of FTEs or O&M budgets.
ITCan has a world class global information management system that is the envy of other foreign service agencies. Nevertheless, there is a perceived need for the technological convergence of its current electronic information systems (e.g., WinExports, Virtual Trade Manager and the IBD Planning and Reporting system) with those systems currently in development (e.g. TRIO and Corporate Dashboard). There is a need for greater consistency in the use of terminology, performance data fields and performance measures across these systems.
The conclusions of this formative evaluation are grounded in the findings presented in the previous three chapters. They are formulated to respond to the general evaluation questions for each of the three evaluation issues: relevance, results achievement/ success and cost-efficiency as per the evaluation Terms of Reference and approved Work Plan. The evaluation recommendations are based on the conclusions presented below and are intended to address only the most important issues raised in this report.
6.1.1 CSF consistency with Canada's IBD's objectives and priorities
The CSF RMAF was considered the definitive design document finalised prior to the CSF launch in April 2002. DFAIT's IBD mandate and objectives were clearly stated therein and the CSF objectives appropriately articulated to ensure policy coherence. The CSF/IBD Planning and Reporting Template for 2002-2003 included ample information and guidance to ensure that CSF initiatives were consistent with the IBD objectives and priorities. This information and guidance was subsequently updated on an annual basis as new strategic frameworks were developed, first for DFAIT and then for ITCan. A demonstration of policy coherence was required by CSF recipients at the planning and reporting stages of the management cycle. Nevertheless, changes in the Department's organisational structure and in its Strategic Framework warrant further articulation and emphasis of the role of 'two-way international investment' and 'science and technology linkages', as important IBD programming areas that are significantly under-represented in terms of the number of CSF initiatives and C-E Section staff time allocation. The integration of IBD programming in trade, investment and science and technology areas is first and foremost a Corporate issue that requires policy guidance and emphasis in the IBD business plans.
Therefore the evaluation makes the following recommendations:
That a CSF Programme Management Committee be established to provide policy guidance, and include members representing the Investment Partnerships Branch, as well as other important stakeholders (e.g., TTCS, TMR).
A more broadly based working group for resource related issues will be struck. Key issues to be addressed include further development of the Canadian Commercial Interests Lists (CCIL), priority sector identification with input from regional offices in Canada, resource needs and utilization at missions, and FTE allocation and re-allocation. The working group on resources would report to a re-invigorated Resource Allocation Committee (RAC) that would now be chaired at the ADM level.
That, further to changes already made, the IBD Planning and Reporting system include additional guidance on international commerce, to ensure policy coherence with ITCan new Strategic Framework.
Alignment with the current ITCan Business Planning Framework is well underway. The instructions sent to missions for the last two years, and currently to regional offices in Canada and HQ divisions, have specifically asked them to consider S&T and investment (in/out) as well as cover trade policy and advocacy matters in their business plans. Ongoing efforts in this regard will continue to be made to ensure and enhance policy coherence in ITCan plans and programs.
That CSF budget allocation targets be established for investment, and science and technology programming in markets where these two programming areas are considered a high priority.
Recognition that the CSF budget has dwindled has been reflected in recent submissions for new funding through the International Policy Statement (IPS) requests. Although no new funding has been provided, continuing efforts will be made in the next several months to augment the CSF to more appropriate levels but an important part of this exercise will include obtaining a full picture of all resources received and utilized at missions around the world from all funding sources. A full picture of resources including CSF budgets, travel and hospitality funds, OGD and other contributions will help in determining future CSF allocations and other operating fund requirements.
An examination of CSF expenditures by priority market and sector are also indicative of policy coherence. The CCIL and TTCS were used as indicative of government policy guidance for the identification of priority markets and sectors, respectively, for IBD programming. While CSF expenditures for FY 2003-2004 generally reflect the rank order of the 53 priority markets, approximately $708,000 (8%) of the funds were nevertheless disbursed in non-CCIL markets. This indicates a significant departure in the CSF allocation from an already long list of "priority markets". Also, the correlation between CSF sector coding and the TTCS priority sectors in 14 of the top 25 CCIL markets can be characterised as weak. C-E Sections are selecting sectors based on market conditions rather than priority sectors identified under the TTCS. In both of these cases, Corporate guidance is required on the use of the CCIL and TTCS as policy instruments to bring about stronger alignment between IBD plans, including CSF initiatives, and priority markets and sectors.
Therefore the evaluation makes the following recommendations:
That ITCan identify the priority markets based on the CCIL rankings that can be used to determine C-E Section eligibility to receive CSF funding.
The Canadian Commercial Interests List (CCIL) has been used over the last few years for allocation of resources to missions abroad in terms of the CSF and FTE determinations. Formerly the Priority Markets List, the CCIL has been a tool developed and used by the Trade Commissioner Service for a number of years to identify priority markets for Canada. Alignment of resources is steadily improving with emphasis being placed on data credibility and sources, appropriate weighting of each indicator, separating countries into tiers, adding more client demand factors as data becomes available and ongoing efforts to ensure transparency, high quality and integrity of the data used.
During the last exercise of revising the CCIL, agreement was reached that a working group on resource related issues would be struck to help examine issues like potential improvements to the CCIL, e.g., ways will be sought to forecast growth in markets.
That ITCan re-examine and revitalise as necessary the TTCS so that C-E Sections in priority and emerging markets can be given clear direction on sector choices.
This particular matter has been discussed separately from the CSF for a number of years and will continue to require clarity and should be given a high priority in the current fiscal year. With the addition of the Regional offices in Canada to the ITCan family, a tremendous opportunity exists for targeting priority sectors where Canada has a competitive advantage in North American and global markets.
Recent efforts with Industry Canada to re-engage with industry on sectors may provide insights in this regard.
In addition, new materials were developed over the last year and provided online for missions to assist them in the selection of priority sectors. The tools were also used in training related to the Global Learning Initiative.
Canadian business and partner clients have consistently expressed high levels of satisfaction over several years with the services of the TCS, which include CSF funded initiatives. High client retention rates is a key indicator that the needs and expectations of clients are being met. Furthermore, new clients have cited 'word of mouth', or referrals, as the most common reason for utilising TCS services abroad, which also indicates a high level of satisfaction among current clients. While there are measurable differences in the levels of satisfaction between Canadian business client and partners, as well between large, medium and small companies, the needs and expectations of clients in general are being met.
The CSF and the estimated revenue that it will leverage will account for approximately 56% of all funding available to C-E Sections during FY 2004-2005. Access to sources of funding for IBD programming other than the CSF is limited to Posts in the USA, Europe and Asia-Pacific regions, while those in the Africa and Middle East and Caribbean and Latin America regions depend on relatively modest IBD travel and hospitality budgets which cannot be used in quite the same manner, especially for proactive initiatives and to leverage additional funds from clients and partners. Reductions in funding to the current fiscal year allocations are generally viewed as negatively impacting the capacity of C-E Sections to undertake proactive IBD programming, or neutral in the case of TPMs who consider their Posts to be either already severely under-funded, or well enough funded from other sources to make the necessary adjustments. Given the funding reductions to a mere fraction of its previous levels, raises the question: What is an appropriate funding level? The evidence suggests that CSF funding remains significant and meaningful for C-E Sections in Posts located in relatively high priority markets (i.e., 2003-2004 CCIL top 23 markets). However, proactive and meaningful IBD programming requires substantially more funds than what small C-E Sections in lower priority markets (i.e., 2003-2004 CCIL 24-53 markets) are currently allocated. Again, this raises the following question: If the CSF is part of the 'basic tool box' for TPMs, what are the performance expectations of those who hardly receive anything?
Therefore the evaluation makes the following recommendations:
That ITCan reestablish the overall CSF budget to at least the same level as the 2003-2004 fiscal year expenditure level if proactive IBD programming remains a corporate priority.
Reinstatement of the CSF budget to earlier levels is challenging. Requests for additional funding under the IPS have been made but no new funding has been provided to the department as of yet. Further efforts will be made in upcoming resource requests.
That as the allocation of FTEs becomes increasingly aligned with the CCIL, CSF allocations be pegged to the C-E Section FTE allocation.
The issue of FTE management will continue to be discussed in relation to other resource related issues. At this point in time, it cannot be fully supported that FTEs and CSF resources should be in direct alignment. The working group on resources will examine this question in more detail and examine its validity.
There are two main categories of CSF products and services. First, there are those focussed on trade promotion and direct sales, and secondly those focussed on providing MII to clients. This distinction is useful in that the former is most often driven by client demand, while the latter reflects a longer term proactive strategic approach to international commerce. Market information and intelligence products and services are perceived by C-E Section staff to have higher value-added. However, the lack of available funding for official language translation of market information products has become an obstacle to C-E Sections wanting to transition from trade promotion/events to providing MII services to Canadian business and partner clients. The usefulness of locally commissioned market information products is diminished when they cannot be made available to target clients through official means of communication. The growth of pressures on Posts/C-E Sections translation budgets is a Corporate issue that requires attention to: a) ensure compliance with the GOC bilingualism policy; and, b) to remove this important barrier to C-E Sections providing high value-added MII services to Canadian business and partner clients.
Therefore the evaluation makes the following recommendation:
That ITCan provide Post/C-E Sections with adequate budgets for official language translation based on demonstrated need, including the translation of locally commissioned market information products that meet the Department's quality standards.
Official translation budgets were devolved from a centralized departmental source of funding to divisions at headquarters a few years ago. At this time, it could be argued that there was an increase in the demand for translated materials as a result of technology advancements, e.g., the Virtual Trade Commissioner (VTC) on the department's website is increasingly being used by clients. A related issue discussed by senior management is that some materials were being produced and/or translated unnecessarily. A small committee has been created by the Assistant Deputy Minister for International Business Development to review the increased demand for translation as well as to develop guidelines on when materials are required in both official languages.
In conjunction with the committee on translation issues, the "resources" working group will seek a solution to the ongoing challenge of producing and funding materials in both official languages.
Overall awareness and appreciation of TCS services has remained high, and improved slightly since the launch of the CSF. While the promotion of TCS services is the fifth most common CSF service, this slight improvement could be attributable to any number of other IBD initiatives. Awareness and knowledge of market opportunities resulting from the provision of MII, as perceived by business clients and partners, remains high and relatively unchanged. An expanded network of contacts, as perceived by business clients and partners, remains high and relatively unchanged for the latter, but somewhat down for business clients, particularly with respect to the accuracy of key contact lists and the reliability of local company information. A valid and reliable determination on the increased awareness for foreign contacts could not be made due to a lack of available data.
Therefore the evaluation makes the following recommendation:
That additional evaluation research be undertaken to determine short-term outcome achievement of foreign contacts, especially with respect to foreign direct investment and science and technology programming.
This recommendation falls outside the Client Service Fund's mandate but can be suggested for consideration by those groups responsible for these areas. The recommendation likely arose because of the department's interest in promoting these two areas as new priorities. With the addition of the Investment, Science and Technology Branch to ITCan, it is likely that work may be underway or will be done in this area. The recommendation will be passed to them for consideration.
The perception of Canadian business and partner clients is that the TCS/CSF services can be attributed with positively influencing or contributing to various aspects of their business. With the exception of 'projected future earning', business client responses to the 2004 Client Survey indicate a significant improvement over previous surveys in the positive contribution of the TCS/CSF to their businesses, especially in terms of market access and opportunities, finding the right people, competitiveness and financial results in the past 12 months. Thirty-six percent (36%) of business clients attributed their financial results averaging $7.6 million each over the past 12 months to the TCS/CSF services.
On the other hand, the perception of Canadian partner clients regarding the positive contribution of the TCS/CSF to their business initiatives has continued on a downward trend since 2002 Client Survey, especially with respect to saving time and money, market access and opportunities, making better decisions, finding the right people and business image/credibility. As mentioned above, this correlates with the perception of TPMs who rated the corresponding short-term outcome significantly lower than others.
Therefore the evaluation makes the following recommendation:
That additional evaluation research be undertaken to determine reasons as to the decline in satisfaction levels of Canadian partner clients, especially with respect to those who would play important roles in investment and science and technology programming.
Work in this area is covered by TCS who conduct the ITCan client survey and as such is not directly related to the Client Service Fund (CSF). Other related work is being done by TCE in terms of TRIO which may over time capture more information on partner clients. In addition, the Investment, Science and Technology Branch, a new branch in ITCan is focusing on programming in the areas of investment and S&T.
The CSF in its design and management has been a catalyst for the reform of IBD programming in Geographic Divisions, as well as in C-E Sections. It has had a very positive influence on IBD strategic planning processes, resulting in more and better quality strategic plans being completed in a timely manner. It has influenced financial planning and management in terms of greater transparency, accountability, efficient use of funds and also reporting in a timely manner. It has influenced IBD management culture through the application of results-based management processes and practices, including the delegation of decision-making authority to TPMs in conjunction with a nascent performance-based budgeting system. Since it does represent such a significant proportion of funds available to Post/C-E Sections for IBD programming, the client-focussed and results-oriented management practices it has introduced are now being applied to some extent in all IBD programming, irrespective of the source of funding (e.g., non-CSF and ERI initiatives). It is a testament to this influence that some TPMs view the CSF as only one component of 'their' IBD Planning and Reporting system.
During the CSF first two fiscal years, there has been a year-over-year improvement in both the total amount of cost-sharing revenue, but also in the ratio of leveraged funds. In 2003-2004, for every $8 in expenditures, the CSF generated approximately $1 in cost sharing revenue. The percentage of lapsed CSF has declined over the same period from 4.82% to .29%. It is generally acknowledged that the CSF is efficiently used as "seed money" to leverage other revenues, and efficiently allocated and reallocated during the year to avoid unnecessary wastage of scarce resources.
While it is generally thought that C-E Section staff have increasingly focussed their efforts on high value-added MII type initiatives for priority clients in priority sectors, as opposed to the short-term event driven multi-sectoral trade promotion and direct sales initiatives, the evidence does not support this assumption. The allocation of C-E Section staff time to IBD programming areas (i.e., trade, investment and S&T), or in support of clients (i.e., business clients versus partner clients), has not changed significantly for a good number of years. The percentage of their time spent on non-client support activities has also remained relatively high and unchanged. However, the CSF influenced the C-E Section team approach to strategic planning, the delegation of decision-making authority for portfolio management and greater results-oriented management.
The general consensus is that clear guidance has been provided by the TBX/CSF Unit on what is considered eligible and ineligible expenses through a variety of communication channels. Incidents of ineligible expenses being charged to the CSF remain anecdotal, since an analysis of end-of-year financial transactions could not be accomplished within the time frame of the evaluation. However, despite appropriate policy guidance provided by the TBX/CSF Unit, it is widely recognised that TPMs/C-E Sections select priority sectors for IBD programming based on local market demands and business opportunities with little, if any, reference to the TTCS sectors and markets. Furthermore, based on a sample of 13 of the top 25 2004-2005 CCIL markets, there is little strategic focus, or sector concentration of CSF-funded initiatives. So while there are no apparent abuses or misuses of CSF funding, they are not invested in a manner that could yet be characterised as "working with priority clients in priority sectors".
Therefore the evaluation makes the following recommendation:
That ITCan divisions establish a systematic review and approval process for IBD Strategy and Action Plans with a view to ensuring increased strategic focus on a more limited number of priority sectors for proactive initiatives.
A systematic process is in place for the review of the annual business plans prepared by Commercial-Economic Sections at missions around the world. During the last fiscal year, colleagues at headquarters with geographic responsibility were asked to review and evaluate the business plans for posts in their territory. While the volume of information remains heavy relative to the capacity for in-depth review, this enhanced the process and strategic focus of the plans.
The new fiscal year is providing further opportunity for improvement in the plans. Regional Offices in Canada which recently joined ITCan are now also part of planning process and significant opportunity exists for targeting the appropriate priority sectors particularly given their proximity to Canadian businesses.
The IBD/CSF Planning and Reporting system has evolved considerably since its launch in April 2002. It has introduced a results-oriented management structure and standardised planning and reporting formats appreciated by Geographic Divisions and C-E Section staff. Initially, the TBX/CSF Unit priority was to strengthen the IBD strategic and operational planning processes which made considerable progress, although earlier availability in the fiscal year of the planning and reporting templates would further enhance their usefulness. Performance tracking and reporting however has not been a priority for a number of reasons previously discussed. Consequently, the current performance reporting system is dependent on only one source of data (i.e., TPMs), and serves primarily as an annual self-assessment tool. It has had limited usefulness as a performance tracking tool for C-E Sections, since the reporting templates are revised and posted only at the end of the third fiscal quarter or thereafter.
Therefore the evaluation makes the following recommendations:
That Strategy and Action Plan templates for each new fiscal year be available on-line at the beginning of each calendar year shortly after the Corporate priorities and results are set.
This recommendation is fully supported and plans are underway to launch the templates well before the beginning of the next fiscal year. Although, it would not have been achievable in the last fiscal year when the new department was created, it is expected that maintaining similar planning templates over the next few years will be quite feasible.
That the Reporting template for each new fiscal year be available on-line at the beginning of the second fiscal quarter.
Earlier availability of the reporting templates will be a high priority particularly since minimal changes are expected in the next year. Reporting requirements will be reduced from previous years as the department rolls out its new system "TRIO" which will capture client information and therefore replace sections of the Reporting Template.
That the current Reporting template continue to be improved with a view to making it a useful performance tracking tool to assist TPMs / C-E Sections with their management decision-making and strategic planning.
Although full capacity and usage of the new system "TRIO" is unknown at this early date since it is currently being piloted, it is reasonable to expect that over time, TRIO will replace some of the reporting information captured by the "Reporting Templates." The shortened version of the reporting template will be more of a performance management tool versus for tracking or performance measurement.
Improvement of client service is one of the main objectives of the CSF, as well as the NA@W, but there was no indication that the performance data captured was used and analysed to identify areas for client service improvement. In fact, TBX/CSF Unit has not succeeded to produce an Annual CSF Performance Report based on the data captured in the reporting template. Without the biannual Client Surveys, no valid and reliable data was available on outcome achievement for this evaluation. In conclusion, the usefulness of the IBD Planning and Reporting system for tracking and reporting on outcomes accrued to Canadian business and partner clients will remain very limited until either complementary ITCan information systems (e.g., electronic Client Relations Management (eCRM) become fully functional, or additional on-line reporting templates are developed.
Therefore the evaluation makes the following recommendations:
That performance tracking and reporting data be analysed on a yearly basis, a performance report produced annually, and an executive summary circulated to senior management.
It has been recognized that the information gathered in connection to the CSF has been substantial and that the ability to analyze a significant amount of information by such a small unit is challenging and could result in higher transaction costs. In addition, it has been decided that efforts would not only concentrate on an annual report per se but rather on providing timely information from various systems that can be used in decision making throughout the year. New tools and systems such as the "IBD Dashboard" and "TRIO" will, over time, replace a significant portion of the data.
That ITCan support the continued development of the IBD Planning and Reporting system into a performance information management system, positioned between the electronic Client Relations Management (eCRM) and the Corporate Dashboard, focussed on Canadian business and partner client outcomes/results data.
An IBD Dashboard is being developed to capture performance measurement data. In addition, TRIO, a client relations management system is being rolled out and will allow missions to track client interest in services and efforts in service provision. These two tools will greatly assist the Client Service Fund in performance measurement. With the high priority senior management has given to this area, it is expected that department wide efforts will be made to track and measure performance.
While not a fee for service programme, the CSF formula for generating cost-shared revenues has considerable potential for augmenting the total available resources to C-E Sections for IBD programming, assuming a reasonable and stable level of CSF funding. Further cost-efficiencies could perhaps be realised by fully integrating other functional ITCan programmes and sources of funding (e.g., Investment Partnerships, Science and Technology, Branding Canada, and ERI) into the same management structure and processes created for IBD programming by the CSF.
Therefore the evaluation makes the following recommendation:
That the approach, processes and tools developed for the CSF be increasingly used to manage all IBD programme funding intended for the use of the Post/C-E Sections.
The processes and tools developed for the CSF have increasingly been used for other areas of IBD and recently for ITCan. The creation of the new department presents even further opportunities in this area with potential for collaboration on all resource related issues. Planning and reporting is already at an ITCan level so progress in that area has already been achieved. Future improvements will include examination of all of our resource needs and utilization in a more corporate context.
1 Since the first draft of the Final Report dated December 2, 2004 was used, the statistical data is subject to change.
2 As stated in the 2001-2002 DFAIT Business Plan.
3 Draft reference document is dated December 3, 2004.
4 See CSF RMAF Exhibit Four - Performance Measurement Tables
5 The 2003-2004 % of lapsed funds takes into consideration end-of-year financial adjustments.